Data processing method based on memory data grid

A memory data and data processing technology, applied in the computer field, can solve problems such as high data processing load pressure and affecting data processing

Inactive Publication Date: 2020-06-16
BEIJING TONGTECH CO LTD
View PDF6 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The data processing method and system for in-memory databases in the above patents have the following disadvantages: when processing data with a large amount of data, the data processing load may be too large due to the huge amount of data, which will affect normal data processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method based on memory data grid
  • Data processing method based on memory data grid
  • Data processing method based on memory data grid

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0060] Data processing methods based on memory data grid, such as figure 1 shown, including the following steps:

[0061] S1: Determine the amount of data to be processed;

[0062] S2: Load the data to be processed in the database into the memory data grid;

[0063] S3: Establish the data grouping model to be processed, configure the memory data structure, and divide the data to be processed in the memory data grid into several data queues;

[0064] S4: Select the corresponding data grouping model for the string length of different types of data;

[0065] S5: Use the columnar storage management method to manage memory for different data types;

[0066] S6: transmit the data queue to different data processing nodes in the memory data grid, so that different data processing nodes process data on the data queue, obtain the processing results, and transmit the processing results to the memory data grid;

[0067] S7: Integrate the processing results transmitted by different dat...

Embodiment 2

[0098] Data processing methods based on memory data grid, such as figure 1 shown, including the following steps:

[0099] S1: Determine the amount of data to be processed;

[0100] S2: Load the data to be processed in the database into the memory data grid;

[0101] S3: Establish the data grouping model to be processed, configure the memory data structure, and divide the data to be processed in the memory data grid into several data queues;

[0102] S4: Select the corresponding data grouping model for the string length of different types of data;

[0103] S5: Use the columnar storage management method to manage memory for different data types;

[0104] S6: transmit the data queue to different data processing nodes in the memory data grid, so that different data processing nodes process data on the data queue, obtain the processing results, and transmit the processing results to the memory data grid;

[0105] S7: Integrate the processing results transmitted by different data ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a data processing method based on a memory data grid, and relates to the technical field of computers. The problem that data processing load pressure is too large due to huge data volume is solved. The method specifically comprises the following steps: determining the data volume of to-be-processed data; loading the to-be-processed data in the database into the memory datagrid; establishing a to-be-processed data grouping model, configuring a memory data structure, and dividing to-be-processed data in a memory data grid into a plurality of data queues; selecting corresponding data grouping models for the character string lengths of different types of data; and performing memory management on different data types by adopting a column type storage management method.According to the method and the device, the queue division result of the corresponding to-be-processed data when the time stability is minimum can be obtained, so that the to-be-processed data is intelligently divided, the time stability after division is minimum, the data processing time is short, and each data processing node is relatively stable.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a data processing method based on memory data grids. Background technique [0002] In-memory data grid IMDG is a new type of grid system for memory resource sharing on the wide area network. Its main goal is to improve the system performance of memory-intensive applications or IO-intensive applications in the case of insufficient physical memory. In-memory data grid Lattice is seen as a way to deal with fast, diverse and large amounts of big data, storing data in memory and distributing it to multiple servers, the purpose of this method is to make it easier to obtain data and improve its availability Scalability and better data analysis. [0003] After retrieval, the Chinese patent application number is CN201910203571.8, which discloses a data processing method and system for in-memory databases, including directly mapping keywords in data to storage addresses based on hash Ha...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F16/22G06F16/2457G06F12/02G06F9/50G06F9/54
CPCG06F9/5016G06F9/5022G06F9/546G06F12/0253G06F16/221G06F16/24578G06F2209/548
Inventor 李蕾于洋张春林李利军李春青
Owner BEIJING TONGTECH CO LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap