System and method for prefetching file layout through readdir++ in cluster file system

Abstract

The invention discloses a system and a method for prefetching file layout by readdir++ in a cluster file system. The system includes: a client module (1), which is used to obtain or return a read directory authorization from a server module (2); After the directory reading is authorized, a directory reading request is sent to the server module (2); the page containing the file layout sent by the server module (2) is stored in the local cache, and the client module (1) reads the directory under the directory directly use the file layout stored in the local cache; the server module (2) is used to grant or recall the read directory authorization to the client module (1); when receiving the read directory request, also Encapsulate metadata information including file layout in a page, and send the page to the client module (1). As a result, the network interaction overhead of obtaining the file layout when reading files can be reduced, and the performance of reading and accessing a large number of small files can be greatly improved.

Description

technical field [0001] The invention relates to a metadata prefetching mechanism in a cluster file system, in particular to a system and method for prefetching file layout by readdir++ in a cluster file system. Background technique [0002] With the advent of the era of big data, the amount of global data information is increasing rapidly. There are more and more small-sized files in fields such as e-commerce, social networking, and scientific computing. Therefore, efficiently managing "massive small files" and providing low-latency small file access services are new issues facing the distributed file system. [0003] In recent years, the separation of metadata and data services has become the mainstream trend of distributed file systems. The advantage of this separation structure is that: the client directly accesses the storage device in an out-of-band manner, and can obtain high access performance for accessing large files. [0004] However, the case of small file acce...

AI Technical Summary

Problems solved by technology

The readdir+ technology used by pNFS obviously cannot effectively reduce the network interaction information overhead of file layout acquisition

Images