Eureka AIR delivers breakthrough ideas for toughest innovation challenges, trusted by R&D personnel around the world.

Data storage method based on distributed hash table and erasure codes

A distributed hash table and data storage technology, applied in electrical components, transmission systems, etc., can solve problems such as single point of failure, and achieve the effect of ensuring consistency, high scalability and availability

Inactive Publication Date: 2014-02-05
RESEARCH INSTITUTE OF TSINGHUA UNIVERSITY IN SHENZHEN
View PDF7 Cites 39 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] Dynamo provides good scalability and availability, but it is more suitable for storing large-scale small data. GFS can provide file-level storage, but there may be a single point of failure. Therefore, how to design a system that ensures scalability On the basis of availability and availability, it is a technical problem to be solved for a distributed storage system with large-scale metadata and data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Data storage method based on distributed hash table and erasure codes
  • Data storage method based on distributed hash table and erasure codes
  • Data storage method based on distributed hash table and erasure codes

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0034] The specific implementation manners of the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0035] figure 1 It is the main flow chart of the data storage method based on distributed hash table and erasure code in the present invention.

[0036] Step S1 (node ​​creation step), establish the topological logic structure of each server node through a distributed hash table (such as Chord) (such as Figure 4 shown), and through the consistent hash algorithm, determine the location of the server where the block (or data block) corresponding to each key (key) is located. In the present invention, the entire system is built on the local file system storage, and the management of data blocks provides get (key) to return data blocks, put (key, block) to write data blocks...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a data storage method based on a distributed hash table and erasure codes, and mainly relates to the field of distributed storage. The data storage method includes the following steps that a storage system manages data blocks wholly through a decentralized distributed hash table structure; in order to guarantee reliability, redundancy storage is conducted on the data blocks through erasure code coding; data are regularly scanned in a background process to maintain uniformity and redundancy of the data. According to the data storage method based on the distributed hash table and the erasure codes, the storage system has the decentralized structure and is better than a traditional distributed storage system in expansibility, and storage cost caused by data redundancy is reduced through the erasure codes on the premise that reliability of the data is guaranteed.

Description

technical field [0001] The present invention relates to the field of distributed storage, in particular to a data storage method based on an erasure code and a distributed hash table, that is, a decentralized storage method. Background technique [0002] Due to its advantages in mass storage, distributed storage systems are gradually replacing traditional local file systems in the Internet field. Amazon's (Amazon) Dynamo distributed storage engine is another typical distributed storage system. It is designed to be completely decentralized, has very good scalability and availability, and provides external key-value-based key- value access interface. Dynamo internally uses the Chord protocol to route information, finds the node responsible for a key in a P2P environment, and uses consistent hashing in Chord, making the entire system more scalable. In Dynamo, in order to ensure that the data is available at any time as much as possible, the strategy of N copies is adopted. Ea...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): H04L29/08
Inventor 陈康高品周佳祥
Owner RESEARCH INSTITUTE OF TSINGHUA UNIVERSITY IN SHENZHEN
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com