Remote data increment synchronization method and device based on data package

Abstract

The invention relates to a remote data increment synchronization method and a device based on a data package. The method comprises a first step of obtaining data manipulation languages (DML) of a database, a second step of leading out the DML into the data package, a third step of sending the data package to a target database in a remote synchronization mode, a fourth step of analyzing the data package into structural query languages (SQL) and changing data, and a fifth step of updating the SQL and the changing data to the target database. According to the remote data increment synchronization method and the device based on the data package, increment records, deleted records and changed records in the database can be extracted in fixed time to form the data package including the increment records. According to network situations and data synchronization demands, the data package is transmitted to a target database system in a java message service (JMS) or a file transfer protocol (FTP) manner by utilizing message-oriented middleware or an FTP client side / server terminal product and being supported by a certain allocation management, and therefore consistency of data of a remote terminal and a near-terminal databases is kept.

Description

Technical field [0001] The present invention relates to the technical field of databases, in particular to a method and device for remote data incremental synchronization based on data packets. Background technique [0002] Remote data synchronization means that two database systems across network segments maintain data consistency. The source system is generally a business system database deployed at the front end, and the target system is generally an OLAP data mart or data warehouse system used to support decision analysis. The connection between the front-end business system database and the back-end data center needs to penetrate the firewall. The existing remote data synchronization system has the following problems: [0003] 1. The remote data transmission method is fixed and single, unable to cope with various complicated situations. [0004] Remote data transmission can use various methods such as Web services, JMS message middleware, and FTP. Web services can realize re...

AI Technical Summary

Problems solved by technology

The method of web service can realize the real-time synchronous transmission of data, but only a small amount of data can be transmitted, which is suitable for scenarios where the data changes sporadically, and the quality of data delivery cannot be guaranteed; JMS message middleware is generally suitable for transmitting a small amount of data, but it can guarantee the data It can also achieve the goal in a bad network environment; FTP can transfer large-scale data volume, and can support breakpoint resume

These three methods have their own advantages and disadvantages. The existing remote data transmission methods can only adopt one of them, which cannot cope with various complex situations under different network environments and data synchronization requirements.

[0005] 2. The incremental data collector used in the remote data transmission front end has certain limitations

However, this method requires the dependence of a specific DBMS (Database management system, database management system), and requires the source system to support flashback query, so the scope of application is narrow

Images