A method and system for selectively rewriting self-referencing blocks in the process of data deduplication

Abstract

The invention discloses a method and a system for selectively rewriting a self-referencing block in a data deduplication process. The method comprise the following steps: obtaining a repeating block in the data deduplication process and a total number of containers backed up last time; 2, judging whether that contain ID of the repeating block is more than the total number of containers back up last time, if so, the repeating block being a self-referencing block, otherwise, the repeating block being not a self-referencing block; creating a buffer for simulating a recovery buffer, finding out whether a container ID of a self-reference block exists in the buffer, if so, the self-reference block being not an internal fragmentation block, otherwise, the self-reference block being an internal fragmentation block; depending on whether the utilization of the container in which the internal debris block is located is below a preset threshold, if so, overwriting the internal debris block to thecontainer, otherwise, the self-referencing block need not be overwritten to the container. The invention identifies the internal fragment block by simulating the recovery buffer, selectively rewritespart of the internal fragment block to limit the rewriting quantity, so that the utilization rate of the container is increased, and the recovery performance is remarkably improved.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more specifically relates to a method and system for selectively rewriting self-referential blocks in the process of deduplication of data. Background technique [0002] Deduplication technology is the most effective way to eliminate redundant data and save space in modern backup systems. The deduplication technology first divides the file into equal-length or variable-length data blocks, then uses the hash method to calculate the fingerprint of each data block, and determines whether the data block is repeated by comparing the fingerprint values. It is a lossless compression technology that can effectively save system storage space. [0003] Since the data blocks in a backup stream are scattered in different containers, two types of fragmentation are introduced: one is inter-version fragmentation, which is caused by the regular backup of similar data; the other is caused by the sam...

AI Technical Summary

Problems solved by technology

[0005] Aiming at the defects of the prior art, the purpose of the present invention is to solve the technical problem that the HAR rewriting method in the prior art cannot eliminate internal fragments caused by self-referential blocks, thus reducing the recovery performance

Images