Incremental data real-time synchronizing device based on log management packet and incremental data real-time synchronizing method based on same

Abstract

The invention provides an incremental data real-time synchronizing device based on a log management packet, which comprises a first synchronizing unit and a second synchronizing unit, wherein the first synchronizing unit is used for duplicating initial data in a first data source into a second data source; and the second synchronizing unit is used for obtaining data manipulation statements executed by the first source by the aid of the logo management packet of the first data source and applying the data manipulation statements into the second data source so as to realize real-time synchronization of incremental data between the first data source and the second data source. The log management packet includes a required storage process for analyzing logos of the first data source so as to obtain data manipulation information from the logos, the data manipulation information comprises markers, manipulation objects and belonging transactions of the data manipulation statements, and the logos record all manipulations on the first data source. The invention further provides an incremental data real-time synchronizing method based on the log management packet. By the aid of the device, synchronism of the incremental data between different data sources is realized.

Description

technical field [0001] The invention relates to the technical field of data integration, in particular to a device and method for real-time synchronization of incremental data based on a log management package. Background technique [0002] With the rapid development of science and technology represented by computer software / hardware, information technology represented by business intelligence has also developed rapidly. At present, management and operation informatization is already one of the key strategies to realize the leap-forward development of an enterprise. Modern enterprises have higher and higher requirements for business intelligence analysis of their business data; senior managers of enterprises are also increasingly Soberly realize how important the valuable information hidden in massive data is to improving the scientific decision-making and management level of enterprises, and the era of formulating policies after taking heads will be gone forever! In this e...

AI Technical Summary

Problems solved by technology

The disadvantages of this technology are: first, it cannot reflect real-time data and does not meet the needs of new business intelligence; make the business overwhelmed

This replication technology can only be limited between databases of the same type, and it is helpless for current business intelligence in OLAP data sources that tend to be open source data sources or columnar data storage with better performance than traditional row data storage.

[0006] 3. Use the trigger mechanism of the database: this technology is relatively intrusive to the original system and will have a great impact on the performance of the business system

This method requires an in-depth understanding of the storage structure of the database log (the data structure stored in the log is not disclosed), and it is extremely difficult to implement; and it is not universal: once the storage data structure in the log changes, it needs to be recreated. Implement the log analysis module

Images