Storage method for redundancy deletion block device based on location-sensitive hash

Abstract

The invention discloses a storage method for a redundancy deletion block device based on location-sensitive hash, and belongs to the data storage field. The method comprises the following steps: putting data blocks of redundant writing detection operation and a corresponding digital finger print into the current operating queue; D: judging whether the number of the data blocks in the queue exceeds threshold value or not, if so, taking threshold value of data blocks as a data section, and executing the step F, and otherwise, executing the step E; E: judging whether the data block at the front of the queue is overtime or not, if so, taking the data blocks as the data section, and executing the step F, and otherwise, executing the step D; F: judging whether the set of metadata of similar data sections exists or not, if so, executing the step G, and otherwise, establishing an empty set, and executing the step G; and G: orderly judging whether digital finger prints of data blocks exist in the set of metadata of the similar data sections or not, if so, modifying the memory addresses of the data blocks, and otherwise, generating the metadata of the data blocks. The method reduces the time of accessing the metadata in the redundant writing detection operation process.

Description

technical field [0001] The invention relates to the technical field of data storage, in particular to a storage method for a redundant block device based on position-sensitive hashing. Background technique [0002] With the explosive growth of digital information, the space occupied by data is getting bigger and bigger; in the past 10 years, the capacity of storage systems provided by many industries has grown from tens of GB to hundreds of TB, or even several PB, which has fully doubled. More than 10,000 times. With the exponential growth of data, enterprises are faced with more and more time points for quick backup and recovery, the cost of managing and saving data is getting higher and higher, and the space and power consumption of data centers are also increasing. The study found that up to 60% of the data stored in the application system is redundant, and it becomes more and more serious as time goes by. [0003] In order to alleviate the problem of space growth in th...

AI Technical Summary

Problems solved by technology

[0005] Block device is the most basic storage device, widely used in various storage systems such as SAN (Storage Area Network, storage area network), NAS (Network Attached Storage, network attached storage), but building a block device that supports embedded redundancy When developing a storage system, we face two major technical challenges: (1) How to notify the upper-layer system of the result of redundant deletion as the underlying block device, and how to be compatible with existing upper-layer storage systems that do not support redundant deletion

(2) The storage system has high requirements on the performance of block devices, and redundant deletion will generate a large amount of computing overhead, as well as increase a large amount of redundant metadata, and when writing data, it is necessary to check whether the written data is in the system, which will significantly increase the number of writes. Data Time Overhead

Images