A distributed storage system that stores metadata and data together

Abstract

The present invention provides a distributed storage system in which metadata and data are stored together. It adopts a four-layer layered architecture, including: a physical layer, an metadata block layer, a consensus layer, and a micro-meta data layer. The micro-metadata layer stores micro-elements. Data, the consensus layer is used to provide a consensus mechanism, and provide consensus services upwards for the call of the micro-metadata layer. After the micro-metadata passes the consensus service of the consensus layer, data writing and modification operations are performed. The parallel Byzantine fault-tolerant algorithm is used to modify multiple data at the same time. There are only one file and the consensus speed is fast. The distributed storage system has only multiple data nodes and no metadata server. All data in the three layers of the micro-metadata layer, the consensus layer and the metadata block layer are stored in a distributed storage method. In each data node; the physical layer includes all physical nodes of the distributed storage method implemented by the distributed storage system, that is, the server node. above the server node.

Description

technical field [0001] The invention belongs to the field of blockchain technology and distributed storage, and particularly relates to a management scheme for distributed storage and a related technology for co-storing metadata and data, that is, a distributed storage system for co-storing metadata and data. Background technique [0002] The distributed storage system can divide the complete data into multiple parts and store them on multiple physically dispersed storage nodes. The data storage and access can be completed by unified management of these nodes. Metadata is the key to a distributed storage system. Metadata mainly stores the location information of each file in the distributed storage. Only by accessing the metadata server can the location of the specific file be determined, so as to write and access the file. , if the metadata server fails or is attacked by humans, the entire distributed storage system will be paralyzed. The current distributed system metadata...

AI Technical Summary

Problems solved by technology

For this metadata management mode, the data synchronization of metadata will consume a lot of resources, and if there is a problem with the cluster system managing metadata nodes, the entire distributed storage system will collapse

[0007] 3. Separate management model without metadata

It splits the data and stores it on different nodes. The storage location of each part of the data is determined by the hash algorithm. Data access also needs to be addressed by the hash algorithm. Therefore, there will be more forms and caches in the system. This storage mode based on hash calculation is extremely complex, which increases the difficulty of system maintenance

Images