System and method thereof for guaranteeing data effectiveness in parallel network file system

Abstract

The invention discloses a system and a method thereof for guaranteeing data effectiveness in a parallel network file system. The system comprises a client side and a metadata server, wherein the client side is used for sending a read-only directory authorization application, receiving endowed read-only directory authorization, marking a version number of a read-only directory and accessing a file under the read-only directory and a data page of the file; the metadata server is used for receiving the read-only directory authorization application sent from the client side and endowing the read-only directory authorization to the client side; and the client side returns the read-only directory authorization to the metadata server after access is finished, and clears the version number of the read-only directory.

Description

Technical field [0001] The invention relates to a cache validity mechanism after pre-reading data between files in a parallel network file system, and particularly relates to a system and method for ensuring data validity in a parallel network file system. Background technique [0002] With the advent of the big data era, the amount of global data information has grown rapidly. There are more and more small-sized files in the fields of e-commerce, social networking, and scientific computing. According to statistics from the Pacific Northwest National Laboratory in 2007, of the more than 12 million files stored internally, more than 80% of the files are smaller than 1MB, and more than 50% of the files are smaller than 32KB. In a sense, there is currently no purely large file application. Mankind has gradually entered into the "age of massive and small files". [0003] Parallel Network File System (pNFS) adopts a separate structure of metadata and data services. The client directl...

AI Technical Summary

Problems solved by technology

[0005] However, compared with metadata prefetching, the biggest difficulty in data prefetching is that even with good physical locality, it is impossible to accurately determine which file the prefetched data page belongs to, and it may even prefetch files in other directories. data pages, which creates a significant barrier to extending directory authority to "data validity assurance"

Images