Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for synchronizing database increment to dynamic ES index database, and medium

An incremental synchronization and database technology, applied in relational databases, database updates, database models, etc., can solve problems such as high latency, complex operations, and inflexible configuration, and achieve zero latency and high availability.

Pending Publication Date: 2021-01-05
SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
View PDF8 Cites 5 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The technical task of the present invention is to address the above deficiencies and provide a method, device and medium for incrementally synchronizing the database to the dynamic ES index library, so as to solve the problems of complex operation, high delay and poor configuration in the existing business system when the data is updated in real time. flexible technical issues

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for synchronizing database increment to dynamic ES index database, and medium
  • Method and device for synchronizing database increment to dynamic ES index database, and medium
  • Method and device for synchronizing database increment to dynamic ES index database, and medium

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0047] Such as figure 1 As shown, the method for database incremental synchronization of the present invention to a dynamic ES index library includes the following steps:

[0048] S100. Using objects and attributes as core elements, perform dynamic data modeling on MYSQL data tables;

[0049] S200, open and read the BINLOG log of MYSQL, analyze and encapsulate the read MYSQL operation event into an event object, and then process the log of MYSQL;

[0050] S300. Send the event object message to the message queue, write the event object message into the ES index library through the message queue consumer, and the message queue is a distributed message queue;

[0051] S400. Create an index by calling the service of the ES index library through the message queue consumer, and read the data model configuration index from the MYSQL data table, and update the incremental and full amount of data to the ES index library.

[0052] In this embodiment, dynamic data modeling is performed...

Embodiment 2

[0071] The embodiment of the present invention also provides a device, including: at least one memory and at least one processor; at least one memory for storing a machine-readable program; at least one processor for invoking the machine-readable program, Execute the method for incrementally synchronizing the database to the dynamic ES index library disclosed in Embodiment 1 of the present invention.

Embodiment 3

[0073] An embodiment of the present invention also provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the processor executes the database incremental synchronization disclosed in Embodiment 1 The method to the dynamic ES index library. Specifically, a system or device equipped with a storage medium may be provided, on which a software program code for realizing the functions of any of the above embodiments is stored, and the computer (or CPU or MPU of the system or device) ) to read and execute the program code stored in the storage medium.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a method and a device for synchronizing database increments to a dynamic ES index database, and a medium, belongs to the technical field of real-time retrieval of data variables in a database, and aims to solve the technical problems of complex operation, high delay and inflexible configuration of an existing service system during real-time data updating. The method comprises the following steps: performing dynamic data modeling on an MYSQL data table by taking an object and an attribute as core elements; starting and reading a BINLOG log of the MYSQL, and analyzing andpackaging a read MYSQL operation event into an event object; sending the event object message to a message queue, and writing the event object message into an ES index library through a message queueconsumer; and calling the service of the ES index library to create an index through the message queue consumer, reading the data model configuration index from the MYSQL data table, and updating theincrement and the total amount of the data to the ES index library.

Description

technical field [0001] The invention belongs to the technical field of real-time retrieval of data variables in a database, in particular to a method, device and medium for synchronizing a database increment to a dynamic ES index library. Background technique [0002] In terms of incremental and full update processing of database data, the traditional solutions mainly include the following: [0003] (1) Timestamp method: add a timestamp field to the table, and when the data changes, use the timestamp field to control the data update operation; [0004] (2) Version control method: add a version number to the data record, do a good job of version control for each update of the data, and judge whether the data has changed by comparing the version number of the data; [0005] (3) Tool method: Use third-party data synchronization tools, such as ETL tools, LOGSTASH, CANAL, etc. to perform incremental data synchronization. [0006] The above tool method cannot dynamically configu...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06F16/23G06F16/27G06F16/28
CPCG06F16/2379G06F16/27G06F16/284
Inventor 谭伟
Owner SHANDONG LANGCHAO YUNTOU INFORMATION TECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com