Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Data updating method and terminal

A data update and data technology, applied in database update, structured data retrieval, electronic digital data processing, etc., can solve the problems of ES cluster pressure increase, aggravated load, and batch update tasks cannot be completed, so as to reduce pressure and improve The effect of data query efficiency

Pending Publication Date: 2021-12-07
FUJIAN TIANQUAN EDUCATION TECH LTD
View PDF13 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

If ES data is only updated for a single piece of data, the load of ES is relatively low, but there are still the following scenarios: For example, if a field needs to be added to a database table in a certain system, it is also necessary to add all the fields that have been previously stored in ES. Types of data are added with an index field, and then, there will be a batch of uninterrupted or occasional sleep for some time to update these data to the unified ES cluster
At this time, there may be such a problem: the ES cluster already has a high load, and then such batch data operations will further increase the load. At the same time, the batch update task cannot be completed, resulting in increased pressure on the ES cluster

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data updating method and terminal
  • Data updating method and terminal
  • Data updating method and terminal

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0063] Please refer to figure 1 , a data update method, including steps:

[0064] S1. Receive a full amount of update messages, and sequentially query a preset amount of data from the ES cluster to obtain a data set.

[0065] Wherein, before receiving the full update message, it includes:

[0066] Encapsulate the ES access service interface according to the type of data operation;

[0067] The data operation instruction sent by the sender is received, and the corresponding ES access service interface is invoked according to the data operation instruction to perform data operation on the ES cluster.

[0068] Specifically, please refer to image 3 with Figure 4 , in the unified ES access service of this embodiment, the agent encapsulates a layer of interfaces provided by ES, and interfaces mainly for functions such as adding, deleting, modifying, and checking are all encapsulated with a layer of proxy interfaces. Each service is directly connected to the ES cluster, and in...

Embodiment 2

[0086] Please refer to figure 2 , a data update terminal, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, and implementing the data update method of Embodiment 1 when the processor executes the computer program each step.

[0087]To sum up, a data update method and terminal provided by the present invention receive a full amount of update messages, sequentially query a preset number of data from the ES cluster, and obtain a data set, wherein the preset number is determined according to the resource amount of the ES cluster, In this way, it avoids a large query pressure on the ES cluster; according to the data set, the corresponding data of the sender of the full update message is called back, and the latest data information of the data set can be actively obtained, improving the efficiency of data query in the business, and the sender You only need to provide a callback interface, and you don’t need to write a set ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

According to a data updating method and the terminal of the invention, the full-amount updating message is received, the preset number of data is queried from the ES cluster in sequence, and the data set is obtained; according to the data set, corresponding data of a sender of the total update message is called back, so that the latest data information of the data set can be actively obtained, and the data query efficiency in business is improved; and before the newest data is written into the ES cluster, the load level of the ES cluster is inquired, if the load level is lower than a preset value, the newest data information of the data set is updated to the ES cluster through the ES access service interface, and otherwise, the quantity or the rate of updating the newest data information of the data set to the ES cluster by the ES access service interface is limited. Therefore, by actively callback the preset number of callback data of the sender, active control can be conveniently carried out according to the ES cluster resource condition, and the number or rate of the latest data written into the cluster is determined according to the load level of the ES cluster, so that the pressure of the ES cluster can be reduced on the premise of ensuring the own service.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a data updating method and a terminal. Background technique [0002] Elasticsearch (ES) is a Lucene-based search server. It provides a distributed multi-user capable full-text search engine based on a RESTful web interface. Elasticsearch, developed in the Java language and released as open source under the terms of the Apache license, is a popular enterprise-grade search engine. Elasticsearch is used in cloud computing to achieve real-time search, stable, reliable, fast, and easy to install and use. [0003] Many systems now use ES to store search data. A very common way is to create a public ES cluster, and many independent systems write data to this ES cluster. If ES data is only updated for a single piece of data, the load of ES is relatively low, but there are still the following scenarios: For example, if a field needs to be added to a database table in a certain sys...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/23G06F16/2458G06F16/21G06F16/25
CPCG06F16/23G06F16/25G06F16/2471G06F16/217
Inventor 刘德建林伟陈宏
Owner FUJIAN TIANQUAN EDUCATION TECH LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com