Metadata snap storage and access method in distributed file system

Abstract

The invention provides a snap storage method in a file system. The file system comprises a directory tree with a snap mark, a snap counting file corresponding to the directory tree, and snap files which correspond to each snap timestamp of the directory tree; the snap storage method comprises the steps as follows: (1) recording indexes of each snap file arranged according to the order of snap timestamp with the snap counting file after creating snaps; (2) maintaining each snap file to enable the initiative effective time recorded in each snap file to be less than or equal to the timestamp of the snap file, and to be more than all metadata of last snap timestamp, and ordering the metadata recorded in same snap file according to the terminative effective time of the snap file. The invention further provides a snap access method in corresponding file system. Compared with traditional increment snap technology, the metadata snap storage and access method of the invention obviously reduces snap access expenditure and improves snap access efficiency under a premise of remaining the advantage that the expenditure of the storage space is relatively small.

Description

Technical field [0001] The present invention relates to the technical field of distributed file systems. Specifically, the present invention relates to a metadata snapshot storage method and a snapshot access method in a distributed file system. Background technique [0002] Since the 21st century, with the development of information globalization and the popularization of emerging network applications such as e-commerce and social networking, data explosions have occurred in various fields from social entertainment to medical transportation. The report shows that the total number of Internet users worldwide has reached 2.4 billion, Facebook processes 2.5 billion pieces of content every day, more than 500 TB of data, and Twitter generates 200 million pieces of information every day. The explosive growth of data makes the distributed file system gradually replace the local file system as the basic data storage method of enterprises, and it also puts forward higher requirements for...

AI Technical Summary

Problems solved by technology

[0008] It can be seen from the above typical examples that the traditional incremental snapshot scheme does not need to store all the data at the snapshot time in each snapshot, so it can reduce the storage space overhead. However, there is a data sharing relationship between snapshots. In order to obtain a single snapshot The corresponding data collection needs to traverse multiple snapshots to determine whether the traversed metadata file belongs to this snapshot, resulting in high access overhead

In particular, distributed systems often support longer running times than local systems, and the number of snapshots will be larger. The problem of high access overhead becomes more serious, which affects recovery efficiency when data is lost.

Images