Parallel execution method based on log analysis synchronization and data synchronization system

An execution method and log technology are applied in the field of parallel execution methods and data synchronization systems based on log parsing and synchronization, and can solve problems such as slowing down synchronization and low replication efficiency.

Active Publication Date: 2020-10-30
WUHAN DAMENG DATABASE
View PDF7 Cites 3 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If the destination data replication software performs replication in strict accordance with the order of the transaction operation logs in the source log stream, the consistency of data replication can be effectively guaranteed, but the replication efficiency will be very low, because the commit operation requires the database to copy the transaction operation logs You can only return after flashing the disk to the log file. Waiting for the log file to flash will slow down the speed of synchronization

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Parallel execution method based on log analysis synchronization and data synchronization system
  • Parallel execution method based on log analysis synchronization and data synchronization system
  • Parallel execution method based on log analysis synchronization and data synchronization system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] In data synchronization, a synchronization system is deployed on the source database and the destination database. The source data synchronization system reads logs from the source database, while the destination data synchronization system is responsible for applying the synchronization operations sent by the source to the destination. terminal database.

[0056] Since existing data synchronization environments vary widely, in order to maximize synchronization efficiency, it is necessary to perform targeted optimization according to the characteristics of the current data synchronization environment, in order to optimize the synchronization performance in the current environment. The present invention customizes a synchronization optimization scheme specifically for a synchronization system that needs to strictly guarantee the order of transaction submission. In this system, when the transaction of the destination end is stored, the order of submission of the transactio...

Embodiment 2

[0128] Both the source database and the destination database have a table T (ID INT PRIMARY KEY, C1 INT). Among them, the source application has two transactions (TRX1 and TRX2) that operate on table T in parallel, and TRX1 inserts 10 rows of data into T table, ID from 1 to 10; TRX2 inserts 2 rows of data into the T table with IDs 20 and 21, and then updates a row of data with the updated ID 1, generating the following log:

[0129]

[0130]

[0131] The synchronous system at the destination end initializes two transaction execution threads, namely EXEC1 and EXEC2, and the above log execution process is as follows:

[0132] After the log receiving thread receives the above two transactions, it assigns TRX1 and TRX2 to two execution threads according to the order of transaction submission, assigns transaction TRX1 to transaction execution thread EXEC1, and assigns transaction TRX2 to transaction execution thread EXEC2. The following details this Two transaction execution ...

Embodiment 4

[0144] see Figure 5 , Figure 5 It is a schematic structural diagram of a data synchronization system provided by an embodiment of the present invention. The data synchronization system in this embodiment includes one or more processors 41 and a memory 42 . in, Figure 5 A processor 41 is taken as an example.

[0145] Processor 41 and memory 42 can be connected by bus or other means, Figure 5 Take connection via bus as an example.

[0146] The memory 42, as a non-volatile computer-readable storage medium based on a parallel execution method, can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, the methods of the above-mentioned embodiments and corresponding programs instruction. The processor 41 executes various functional applications and data processing by running non-volatile software programs, instructions and modules stored in the memory 42 to implement the methods of the foregoing embodiments.

[0147] Where...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a parallel execution method based on log analysis synchronization and a data synchronization system, and the parallel execution method comprises the steps that a log receivingthread distributes a to-be-executed transaction to which a submission operation belongs to a corresponding transaction execution thread after obtaining the submission operation; the method further includes taking out the to-be-executed operation from the to-be-executed transaction, and judging the operation type of the current to-be-executed operation by the transaction execution thread; if the current to-be-executed operation is the DML operation, judging whether the current to-be-executed operation conflicts with other to-be-executed affairs or not; and if so, adding the transaction execution thread to which the current to-be-executed operation belongs to the awakening linked list of the conflict transaction, and executing log storage processing after the conflict is removed. According to the invention, mainly through a conflict detection mechanism, the operation which can only be executed after other transactions are submitted is executed in advance, the non-conflict transaction operation is executed in advance as much as possible, and the number of times of waiting for submission of other transactions is reduced, so that the degree of parallelism is increased.

Description

technical field [0001] The invention belongs to the technical field of data synchronization, and more specifically relates to a parallel execution method and a data synchronization system based on log parsing and synchronization. Background technique [0002] The traditional database-based master-backup mechanism realizes real-time replication of database data, which is an important solution for data disaster recovery and backup to ensure data security. Currently, for example, Oracle's DataGuard and Dameng Database's DM7 master and backup products are mature commercial products that implement this solution. Under the database master-backup mechanism, the standby machine is generally used as the backup node, and usually only provides read-only access. Some read-only access applications such as report analysis and data mining can be performed on the standby machine, but cannot provide read-write access like the main machine. In addition, the database master-backup mechanism r...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/17G06F16/178G06F16/23G06F16/27
CPCG06F16/1734G06F16/178G06F16/2365G06F16/27Y02D10/00
Inventor 孙峰付铨彭青松刘启春
Owner WUHAN DAMENG DATABASE
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap