Distributed storage system for common storage of metadata and data

Abstract

The invention provides a distributed storage system for common storage of metadata and data, which adopts a four-layer hierarchical architecture and comprises a physical layer, an element block layer, a consensus layer and a micro-metadata layer, the micro-metadata layer stores micro-metadata, the consensus layer is used for providing a consensus mechanism and providing consensus service upwards for the micro-metadata layer to call, and the physical layer is used for storing the micro-metadata. The micro-metadata is subjected to data writing and modification operation after passing through a consensus layer consensus service, a parallel Byzantine fault-tolerant algorithm is adopted, a plurality of files can be modified at the same time, the consensus speed is high, a distributed storage system only has a plurality of data nodes, no metadata server exists, and the data storage efficiency is improved. All data of the infinitesimal data layer, the consensus layer and the element number block layer are stored in each data node by adopting a distributed storage method; the physical layer comprises all physical nodes, namely server nodes, of a distributed storage method implemented by the distributed storage system, and the data, the element block layer, the consensus layer and the micro-metadata layer are located above the physical layer and all run above the server nodes.

Description

technical field [0001] The invention belongs to the field of block chain technology and the field of distributed storage, and in particular relates to a distributed storage management scheme and related technologies for storing metadata and data together, that is, a distributed storage system for storing metadata and data together. Background technique [0002] The distributed storage system can divide the complete data into multiple parts and store them on multiple physically dispersed storage nodes, and the data storage and access can be completed through unified management of these nodes. Metadata is the key to a distributed storage system. Metadata mainly stores the location information of each file in the distributed storage. Only by accessing the metadata server can the location of the specific file be determined, so that the file can be written and accessed. , if the metadata server fails or is artificially compromised, the entire distributed storage system will be pa...

AI Technical Summary

Problems solved by technology

For this metadata management mode, the data synchronization of metadata will consume a lot of resources, and if there is a problem with the cluster system managing metadata nodes, the entire distributed storage system will collapse

[0007] 3. Separate management model without metadata

It splits the data and stores it on different nodes. The storage location of each part of the data is determined by the hash algorithm. Data access also needs to be addressed by the hash algorithm. Therefore, there will be more forms and caches in the system. This storage mode based on hash calculation is extremely complex, which increases the difficulty of system maintenance

Images