Incremental data synchronization method, device and equipment and readable storage medium

Abstract

The invention discloses an incremental data synchronization method, device and equipment and a readable storage medium. The method comprises the steps that a source database is controlled to start a CDC, and capture operation in the CDC is forbidden; obtaining table structure information corresponding to the current scanning from the source database, comparing the table structure information of the current scanning with the corresponding table structure information of the last scanning to obtain table structure change information, and sending the table structure change information to the target database; and scanning logs of the source database by using fndblog to obtain a log corresponding to current scanning, analyzing the log corresponding to current scanning to obtain incremental data, and sending the incremental data to the target database. According to the technical scheme disclosed by the invention, the CDC and the fndblog are combined to synchronize the incremental data, so that the log is prevented from being cut off, the table structure change information can be identified, and the incremental data synchronization can be effectively and reliably carried out without depending on the self-increasing field.

Description

technical field [0001] The present application relates to the technical field of data synchronization, and more particularly, to an incremental data synchronization method, apparatus, device, and readable storage medium. Background technique [0002] When database data changes, the changes are synchronized to another database, which belongs to incremental data synchronization. SQL Server is a relational database system launched by Microsoft, which has the characteristics of scalability and high performance. [0003] As a closed-source database, SQL Server generally adopts the following three methods for incremental data synchronization: 1) Based on auto-increment fields, this method depends on the auto-increment fields existing in the database table. The incremental records larger than the increment point are filtered out from the table. This method must rely on the auto-increment field. Not all tables have auto-increment fields, and this method cannot synchronize delete op...

AI Technical Summary

Problems solved by technology

[0003] As a closed-source database, SQL Server generally uses the following three methods for incremental data synchronization: 1) Based on the auto-increment field, this method relies on the auto-increment field in the database table. By setting the value of the increment point, from Incremental records larger than the incremental point are filtered out in the table. This method must rely on the auto-increment field. Not all tables have auto-increment fields, and this method cannot synchronize the delete operation, and cannot handle increments smaller than the setting 2) Based on CDC (Change Data Capture, change data capture), that is, using the CDC mechanism of SQL Server itself to capture activities such as insertion, update and deletion of database tables, incremental data will be recorded In the change capture table, incremental data can be obtained by querying the change capture table. However, this method cannot identify the table structure change and cannot synchronize the table structure change; 3) Based on fn_dblog, query fn_dblog to obtain the incremental data of the database Incremental information, and incremental data is obtained through incremental information, but this method cannot perform incremental data synchronization when the log is truncated, that is, once the database log is truncated, incremental data cannot be obtained through fn_dblog

Images