Method and system for synchronizing structured data between relational databases

Abstract

The invention discloses a system and method for synchronizing structured data between relational databases. The method is used for realizing data synchronization between a source database and a targetdatabase and includes steps: step 1) according to practical business application demands, extracting key value pair information from to-be-exported structured original data in the source database andlog information of the source database, rewriting into intermediate data meeting with format requirements according to a data operating type and a data entry mark format, and storing as intermediateexport data in a json format; step 2) according to a data import strategy of the target database, mapping the intermediate export data in the json format from basic data to target data through a datacleaning operation and a database format conversion operation, converting into to-be-imported data meeting corresponding database import formats, and importing into the target database; step 3) subjecting the generated to-be-imported data to reverse parsing according to the target database type to generate backup recovery data for data version rollback.

Description

technical field [0001] The invention relates to the field of data synchronization processing, in particular to a method and system for synchronizing structured data between relational databases. Background technique [0002] In the field of data processing, processes such as data extraction, mapping conversion, and synchronization are usually required. The archiving and warehousing of original data is usually stored in traditional databases such as MySQL, MariaDB, SQLServer, etc. to support large-scale retrieval needs, and further data processing will store data in relational databases such as elasticsearch and solr that professionally serve data retrieval engines. Therefore, data needs to be mapped and synchronized between various relational databases. In addition, enterprises, companies, and data analysis institutions that provide large-scale data applications usually have security isolation settings for internal and external networks for the authority control of data res...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to propose a method for synchronizing structured data between relational databases and a method for synchronizing structured data between different relational databases, as well as data processing requirements such as synchronization of off-line structured data such as one-way communication. The system is suitable for solving the problem of data synchronization between databases, and can complete the synchronization of data between databases in a broad sense, that is, including complete consistency of data and incompletely consistent mapping synchronization after adding data cleaning operations; especially suitable for offline situations Under the data synchronization, the data reverse analysis operation is added to roll back the data version; to ensure the reliability and stability of the data

Images