A method and system for deduplication of image blocks based on image edge detection

Abstract

The invention discloses an image block deduplication method and system based on image edge detection, and belongs to the field of computer storage. The invention divides the picture into blocks in the state of two-dimensional pixel matrix, and uses the mean value hash algorithm to calculate the identification of small blocks, so as to avoid the influence of coding on deduplication. The mean hash algorithm of the picture is used to calculate the fingerprint of the small picture block, so as to determine whether the small picture block is repeated. The algorithm is simple, the calculation speed is fast, and it is sensitive to the content. For two pictures with high similarity and redundancy, the block operation divides their identical parts and the parts with differences, the identical parts are removed as redundant blocks, and the parts with differences are retained. This achieves the de-duplication of the image. The invention uses the image edge detection technology to determine the block mode, and under the same uniform block condition, the block effect realizes the block mode in which the main objects in the picture are separated, and finally achieves the highest deduplication rate.

Description

technical field [0001] The invention belongs to the technical field of computer storage, and more particularly relates to a method and system for deduplication of image blocks based on image edge detection. Background technique [0002] With the rapid development of the Internet and its applications, the amount of data in the information storage systems of background service providers is growing explosively, and the storage costs increase accordingly. Data deduplication technology is a lossless data compression technology. Due to its strong scalability and high compression throughput, it has been widely used in data storage and backup systems. Data deduplication technology is to divide the stored file into multiple data blocks in the data stream format, and use a specific algorithm to calculate the fingerprint digest of the data block, and the calculated fingerprint digest is used as the unique identifier of the data block , to match and determine duplicate data blocks. Da...

AI Technical Summary

Problems solved by technology

If the same picture adopts different encoding and compression methods, the final file stream will be completely different, and the use of file block deduplication technology will not achieve the purpose of deduplication

In addition, for picture data such as continuous shooting pictures, there is a lot of the same perceptual information between the two pictures, but they are also different in the encoding stream due to subtle differences in content. Use file blocks to deduplicate Technology also fails to achieve the purpose of deduplication

Images