High-performance distributed key value storage method based on master-slave copy data decoupling

A distributed key-value, data decoupling technology, applied in special data processing applications, file system types, file access structures, etc., can solve problems such as aggravating system read and write amplification, achieve low overhead, speed up data recovery operations, improve Effects on write and read performance

Pending Publication Date: 2022-05-13
UNIV OF SCI & TECH OF CHINA
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] In order to solve the shortcomings of the above-mentioned prior art, the present invention proposes a high-performance distributed key-value storage method based on master-slave copy data decoupling, in order to improve the overall performance of the distributed key-value system and provide The mechanism of dynamically adjusting the read and write performance according to the performance requirements of the application, so as to solve the key problem that the unified multi-copy data management scheme in the existing distributed key-value storage system aggravates the read and write amplification of the system

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • High-performance distributed key value storage method based on master-slave copy data decoupling
  • High-performance distributed key value storage method based on master-slave copy data decoupling
  • High-performance distributed key value storage method based on master-slave copy data decoupling

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0040] In this example, if figure 1As shown, a high-performance distributed key-value storage method based on master-slave replica data decoupling, including master-slave replica data decoupling, master-slave replica data differential storage, storage order adjustment and parallelism data recovery. The present invention adopts hash calculation and combines the consistent hash ring to decouple the master-slave copy data, so that the master-slave copy decoupling operation is lightweight and low-overhead, and does not affect the data distribution mechanism and consistency of the upper layer. In addition, the present invention first batch-appends the slave replica data to the first-tier global log through a two-tier log architecture, and then the background thread divides it into multiple local logs of the second layer, thereby ensuring efficient slave replica write Ingress performance, as well as good read and restore performance from the copy; secondly, the present invention de...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a high-performance distributed key value storage method based on master-slave copy data decoupling, which is applied to a storage cluster comprising a plurality of nodes, if any node receives key value data for the ith time, decoupling operation and differentiated storage operation are carried out on the ith key value data, and writing the ith key value data into the log structure merging tree or the two-layer log architecture for storage and management, carrying out order degree adjustable operation on the key value data in the two-layer log architecture, and carrying out parallel data recovery operation on the key value data in the fault node. According to the method, the overall performance of the distributed key value system can be improved, and a mechanism capable of dynamically adjusting the read-write performance according to the performance requirement of the application is provided, so that the key problem that the read-write amplification of the system is intensified by a unified multi-copy data management scheme in the existing distributed key value storage system is solved.

Description

technical field [0001] The invention belongs to the technical field of computer storage systems, and in particular relates to a distributed KV store system (distributed KV Stores), based on master-slave replica data decoupling to construct a differentiated replica data storage mechanism, and on this basis, a A copy data management scheme with dynamically adjustable order, realizing a distributed key-value storage method with high performance and adjustable read and write performance. Background technique [0002] In order to meet the storage and access requirements of massive unstructured data, and overcome the shortcomings of traditional relational databases and file storage in terms of scalability and performance, key-value storage (referred to as KV storage) system provides a good solution , such as Google's LevelDB, Facebook's RocksDB and other stand-alone key-value storage systems, and distributed key-value storage systems derived to support large-scale data storage, su...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F16/13G06F16/18
CPCG06F16/134G06F16/1815G06F16/137
Inventor 李永坤张强许胤龙
Owner UNIV OF SCI & TECH OF CHINA
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap