Data processing method, device, equipment and medium based on key-value database

A data processing and database technology, applied in special data processing applications, database indexing, structured data retrieval, etc., can solve the problems of large input and output bandwidth of storage devices, low data processing efficiency, etc., to reduce the amount of raw data and relevance high effect

Active Publication Date: 2022-06-24
锐掣(杭州)科技有限公司
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] The present invention provides a data processing method, device, device and medium based on a key-value database, which is used to solve the problem that the current compaction operation method processes the largest SST file, which takes up a relatively large input of the storage device in a single processing process. The problem of output bandwidth and low efficiency of data processing

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data processing method, device, equipment and medium based on key-value database
  • Data processing method, device, equipment and medium based on key-value database
  • Data processing method, device, equipment and medium based on key-value database

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0075] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with some aspects of the invention as recited in the appended claims.

[0076] The technical solutions of the present invention will be described in detail below with specific examples. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present invention will be described below with reference to the accompanying drawings.

[0077] In order to clearl...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

A key-value database-based data processing method, device, device, and medium provided by an embodiment of the present invention, the method includes: acquiring a plurality of first SST files in the target level to be processed and at least one second SST in the associated level file; determine the target SST file to be processed in the target hierarchy according to the key range and file size corresponding to each first SST file and the corresponding key range and file size of each described second SST file; for each first SST file in the target SST file For data blocks, determine the number of second data blocks whose key ranges overlap with the first data block in the second SST file; perform corresponding data processing on each first data block according to each number. The data processing method based on the key-value database in the embodiment of the present invention can reduce the amount of original data involved in the merging operation process, thereby occupying less input and output bandwidth of the storage device during the data processing process, and improving the efficiency of data processing.

Description

technical field [0001] Embodiments of the present invention relate to the technical field of data storage, and in particular, to a data processing method, apparatus, device and medium based on a key-value database. Background technique [0002] Log structure merge tree (full name in English: log structure merge tree, abbreviated in English: LSM-tree) is the current mainstream technology of key-value databases. Compared with the traditional multi-way search tree (abbreviation: B-tree) and B+ tree (abbreviation: B+tree), LSM-tree is optimized for random writing, and has better performance in data writing and deletion . The core of LSM-tree is the compaction operation in the background. The compaction operation can be called a merge operation or a compaction operation. The compaction operation is to select SST files with overlapping key ranges from adjacent levels (English: level) (full name in English: Sorted String Table, Chinese: Sorted String Table), and perform all key-v...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F16/22
CPCG06F16/2246
Inventor 郑宁许树堃王欢
Owner 锐掣(杭州)科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap