Adaptive Fast Incremental Read-ahead Method for Wide Area Network File System

Abstract

The present invention proposes an adaptive fast incremental read-ahead method for wide-area network file systems, which is characterized in that the client maintains a pre-read cache block for a specific number of files, and when the file read request is transferred from the kernel to the client, The client decides whether to add or replace the cache block size and the size of the prefetched content according to the saved cache context; when the file access service node receives the file request, it also saves the file prefetch in the main memory for fast Respond to prefetch requests for files. The invention is applicable to the CS architecture of the wide-area high-performance computing environment, runs on the file access service node and the client node, has good stability and scalability, and can adaptively and dynamically adjust the prefetch size according to the actual operation situation , and the number of cache blocks to improve the performance and availability of remote file data access.

Description

Technical field: [0001] The invention discloses an adaptive fast incremental pre-reading method oriented to a wide-area network file system, relates to challenges faced by wide-area high-performance computing, and belongs to the technical field of computers. Background technique: [0002] The network file system is a network abstraction above the file system, allowing remote clients to access remote files through the network in a similar manner to the local file system. Currently, the widely used network file systems include NFS, Lustre , Ceph, HDFS, etc. The realization of the network file system is realized through the client-server mode. The server stores files and data, and the client encapsulates local file system requests such as metadata requests and read and write requests into TCP or UDP packets. It is sent to the server through RPC or other network connection forms, and then the server executes the corresponding request action to return the result. The file reques...

AI Technical Summary

Problems solved by technology

[0007] Although the local file prefetching method can effectively increase the file request speed, there is no way to move it all to the network file system, because the information transmission speed on a single machine is much faster than the network speed, and strict heuristic pre-reading will be inefficient due to network delays. under

The pre-reading of ordinary network file systems either works on the file storage machine, which is equivalent to local file pre-reading, or works on the client, which does not take into account the two-way speed efficiency, and the pre-reading does not take into account the concurrent reading of multiple files. In the scenario, only consider the case of serial file reading

Images