Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data synchronization method and device

A data synchronization and database technology, which is applied in the computer field, can solve the problems of inability to further improve data synchronization performance and long synchronization time, and achieve the effects of reducing data synchronization time, improving efficiency, and increasing concurrency

Pending Publication Date: 2021-02-02
BEIJING JINGDONG ZHENSHI INFORMATION TECH CO LTD
View PDF0 Cites 8 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] In view of this, the embodiments of the present invention provide a data synchronization method and device, which can at least solve the problems in the prior art that the data synchronization performance cannot be further improved and the synchronization time is long

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data synchronization method and device
  • Data synchronization method and device
  • Data synchronization method and device

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0066] Exemplary embodiments of the present invention are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present invention to facilitate understanding, and they should be regarded as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

[0067] It should be noted that the embodiment of the present invention is applicable to incremental data subscription and consumption components, such as Debezium and Canal, and the present invention is mainly described by taking Canal as an example.

[0068] The present invention is mainly aimed at the master-slave synchronization of incremental data. Incremental data co...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data synchronization method and device, and relates to the technical field of computers. One specific embodiment of the method comprises the steps of monitoring a first database, so as to obtain an updated log file from the first database when a log in the first database changes; analyzing the log file to obtain incremental data, and writing the incremental data into a message queue corresponding to a table name in the incremental data according to the table name in the incremental data; and writing the incremental data in the message queue into a second database through a queue consumption thread corresponding to the message queue. According to the embodiment, the whole data synchronization method has a clear architecture, and the table-level concurrent processing mode is adopted in the parallel implementation stage in the data synchronization process, so that the overall concurrency is greatly improved, the data synchronization efficiency is effectively improved, the data synchronization time is shortened, and meanwhile, the data synchronization consistency is ensured.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a data synchronization method and device. Background technique [0002] Database incremental subscription and consumption middleware, in practical applications, has a wide range of application scenarios, such as for data synchronization across computer rooms, reducing the delay of database synchronization; heterogeneously forming ES (ElasticSearch, search engine) data from mysql database data, improve query performance, etc. [0003] Taking Canal as an example, it adopts a multi-threaded division of labor and multi-stage processing mode. In order to ensure the consistency of the data after synchronization, Canal adopts a relatively conservative strategy: use binlog serial to pull, parse, store and consume data to ensure data consistency. [0004] In the process of realizing the present invention, the inventor finds that there are at least the following problems in the...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/23G06F16/27
CPCG06F16/2365G06F16/27Y02D10/00
Inventor 杨坤
Owner BEIJING JINGDONG ZHENSHI INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com