Method for distributing metadata of distributed type file system

Abstract

The invention discloses a method for distributing metadata of a distributed type file system. The method uses a catalogue of the file system as a basic unit of Hash, adopts an extensible Hash method to distribute the metadata to each metadata server to ensure that clients can position the position of the metadata; the metadata has high accessing efficiency, even distribution and balanced load; meanwhile, the method maintains storage locality of the catalogue, realizes conveniently prefetching, and improves the accessing efficiency. A unique and unchanged identifier is given to the catalogue to avoid metadata migration caused by the path name of Hash and improve the system performance; participation or exit of the metadata server cannot cause redistribution of mass metadata so as to furthest reduce the metadata migration and ensure high expandability.

Description

technical field [0001] The present invention relates to the field of distributed file systems, and more specifically, to a distributed file system metadata distribution method for clustered metadata servers (Metadata Server: MDS). Background technique [0002] With the emergence of an object storage device (Object Storage Device: OSD), it provides a foundation for constructing a large-scale distributed file system of PB (10 to the 15th power byte) level. In order to manage a large-scale distributed file system based on OSD more effectively, file metadata and data access are often separated, and all metadata of the file system is managed by an independent metadata server cluster, while the cluster composed of OSD is dedicated to Manage and store files. data figure 1 The structure of the distributed file system in this case is given. [0003] Metadata is data describing data, and the metadata of a file system includes basic attribute information and directory structure info...

AI Technical Summary

Problems solved by technology

The advantage of this method is that the location of the metadata can be directly calculated, and the metadata is evenly distributed to each MDS to obtain a better load balance, but it loses the storage locality of the file system directory, making path traversal consume a lot

Moreover, when a file (or directory) is renamed or the metadata server increases or decreases, its hash value changes accordingly, which will lead to a large amount of metadata migration, causing a sharp drop in system performance

[0007] Given that existing metadata allocation methods cannot meet the high-performance and high-scalability requirements of distributed file systems for metadata server clusters, we propose a directory-based scalable hashing method to allocate metadata

Images