Small file storage optimization system based on virtual file system in KUBERNETES user mode application

Abstract

The invention discloses a small file storage optimization system based on a virtual file system in KUBERNETES user mode application. The optimization system is applied to a target file system. The optimization system comprises a user mode application, a network file system and a virtual file system, wherein the virtual file system comprises a virtual file system client and a virtual file system server, the network file system comprises a network file system server and a network file system client, and the user mode application is mounted on a shared directory exposed by the network file systemserver through the network file system client; a file block distribution and formatting module in the virtual file system server creates a virtual file block for storing small files on object storagetarget equipment of one or more target file systems, and the virtual file block is mounted on a shared directory exposed by the network file system server; according to the method, the read-write performance of a large number of small files is greatly improved, and the advantages that distributed shared storage is convenient to manage and can be mounted anywhere based on a network are reserved.

Description

technical field [0001] The invention relates to the field of file system storage, and more specifically, to a small file storage optimization system based on a virtual file system in a KUBERNETES user mode application. Background technique [0002] In today's Internet, data is growing explosively. Various applications such as social networking, mobile communications, online video, and e-commerce can often generate a large number of small files of hundreds of millions or even billions or tens of billions. Due to huge challenges in metadata management, access performance, storage efficiency, etc., the problem of massive small files has become a recognized problem in the industry. [0003] In the field of computer storage, the storage performance of small files is an important aspect that affects the storage performance of the entire file system. Especially in the field of deep learning, the read and write performance of a large number of small files represented by small pictur...

AI Technical Summary

Problems solved by technology

[0007] The present invention aims to overcome at least one defect of the above-mentioned prior art, and provides a small file storage optimization system based on a virtual file system in a KUBERNETES user mode application, which is used to solve the problem of reduced efficiency of small file storage and reading

Images