Method and system for data cleaning suitable for mass storage

Abstract

The invention provides a method and system for data cleaning suitable for mass storage. The method includes the steps that step (101), a plurality of levels of catalogues are built below a mount point of cloud storage, and files are stored in the mounted catalogues, wherein the names of all the levels of catalogues are formed according to a plurality of bytes of file names; step (102), a distribution strategy is adopted for carrying out searching on one appointed level of catalogue, last access time of all the files below the catalogue is obtained, and the distribution strategy is that a plurality of processes are started simultaneously; step (103), according to the difference values among the last access time of all the files and current scanning time, which files needing to be deleted is judged, the concrete steps are that an initial threshold value is set, the files of which the different values are larger than the initial threshold value are searched and serve as the files to be deleted; if the files to be deleted are not searched, the initial threshold value is reduced, the files of which the different values are larger than the reduced initial threshold value are searched again and serve as the files to be deleted, and the operation is carried out until released storage space meets needs.

Description

Technical field [0001] The invention relates to the problem of mass storage cleaning, and specifically a method and system for data cleaning of mass storage media. Background technique [0002] With the rapid development of the Internet, network operators continue to build basic network facilities, and bandwidth is constantly improving, but at the same time, Internet applications based on high bandwidth are constantly introducing new ones. Netizens are more pursuing online, real-time, and high-definition Internet application experience. As a result, Internet applications generate a large number of data files, and the capacity of storage media has also grown from GB to TB, and then to the current PB level. [0003] Because the capacity of a single disk is very limited, if you want to build a storage of hundreds of terabytes or even several petabytes, the disk data that needs to be managed will be very large. Therefore, mass storage technology has emerged. Cloud storage is one of the...

AI Technical Summary

Problems solved by technology

Due to the huge disk storage space, the number of saved files is much larger than when no cloud storage is used. When the cloud storage space is about to be exhausted and files need to be deleted to free up storage space, the massive number of files will cause the data cleaning work to fail.

Images