Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Universal multi-source heterogeneous large-scale data synchronizing system

A large-scale data, multi-source heterogeneous technology, applied in the direction of electrical digital data processing, special data processing applications, instruments, etc., can solve the problem of limited data throughput

Active Publication Date: 2018-02-23
GUANGDONG CONSTR INFORMATION CENT
View PDF5 Cites 32 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The technical problem to be solved by the present invention is to overcome the defect of limited data throughput of existing synchronization methods for massive heterogeneous data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Universal multi-source heterogeneous large-scale data synchronizing system
  • Universal multi-source heterogeneous large-scale data synchronizing system
  • Universal multi-source heterogeneous large-scale data synchronizing system

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0055] This embodiment provides a universal multi-source heterogeneous large-scale data synchronization system, such as figure 1 As shown, it includes: a synchronization network planning management unit 11 , an installation configuration unit 12 , a block pipeline processing unit 13 , a one-way synchronization unit 14 , a two-way synchronization unit 15 and a synchronization correctness guarantee unit 16 .

[0056] Synchronous network planning management unit 11, used to construct a synchronous topology of a tree structure, the synchronous topology includes a plurality of synchronous pairs formed by a pair of adjacent nodes, and the synchronous pairs include clients (relative lower-level nodes) Compared with the server (relative superior node), the intermediate nodes of the synchronization tree (nodes except leaf and root nodes) can be equipped with both roles of client and server, but they belong to different synchronization pairs. The client includes an intermediate layer so...

Embodiment 2

[0074] This embodiment provides a heterogeneous data synchronization method, which is applicable to the universal multi-source heterogeneous large-scale data synchronization system described in Embodiment 1. The following uses the client as an example to introduce the heterogeneous data synchronization method, as shown in image 3 shown, including the following steps:

[0075] S11: Synchronous configuration initialization stage. The client obtains the metadata information of the source data terminal participating in the synchronization data table and transmits it to the server. Then, the client creates several (preset value) data synchronization threads, and according to the heterogeneous data mapping rules and change capture rule templates obtained from the server, creates "delete", "insert" and "change" three threads on the source data side. Change capture triggers (collectively "Triggers").

[0076] S12: Change event trigger stage. When a change event is triggered, the t...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a universal multi-source heterogeneous large-scale data system. The system comprises a synchronous network planning management unit, an installation configuration unit, a blockassembly line processing unit, a one-way synchronizing unit, a two-way synchronizing unit and a synchronous correctness guaranteeing unit. The system works on a node database layer and on a middle layer below an application logic layer. In the data synchronizing process, a client transmits captured local change information to a server according to a synchronizing task plan; the server receives thechange information and transmits the change information to an asynchronous parallel information processing mechanism to store the change information in a corresponding information queue; the server polls the local information queues to read the to-be-processed change information, and then carries out subsequent data change according to an heterogeneous data mapping rule so as to maintain the consistency of synchronous data of a source data copy and synchronous data of a target data copy. The system independently runs parallel to a synchronous node local application, and provides a relaxationaffair guarantee mechanism of internet distributed multi-source heterogeneous data synchronization by means of loose coupling coordination.

Description

technical field [0001] The present invention relates to the technical field of Internet data processing, in particular to a loose computing environment oriented to a wide-area network connected through an Internet communication protocol, applicable to a variety of mainstream data system types, and capable of covering a large number of autonomously managed heterogeneous data source nodes Scale data synchronization middle layer system. Background technique [0002] Heterogeneous data refers to data with different structures. The heterogeneity of data is mainly reflected in: the heterogeneity of computer architecture, the physical storage of data comes from computers with different architectures; the heterogeneity of operating systems, the storage of data comes from different The operating system; the data format is heterogeneous, and the data storage management mechanism is different. It can be a relational database system, such as Oracle, SQL Server, DB2, etc., or a file-type...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F17/30
CPCG06F16/275G06F16/273G06F8/61G06F9/546G06F2209/541G06F9/4881G06F9/5061G06F2209/5018G06F2209/5011G06F16/2246
Inventor 杨海涛徐飞阮镇江
Owner GUANGDONG CONSTR INFORMATION CENT
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com