Quick massive-picture deduplication method

Abstract

The invention discloses a quick massive-picture deduplication method. Each of to-be-deduplicated pictures is used to generate image fingerprint information through a perceptual hash algorithm; and multiple sets of random hash mapping are employed to construct an image hash feature dictionary, and thus duplicate pictures are removed. Compared with the prior art, the method generates image fingerprints through constructing low-dimensional image features, constructs the image hash feature dictionary, thus carries out quick deduplication of the massive images, completely removes image feature comparison consuming more time, alleviates the mapping space sparse problem caused by the low-dimensional image features and local-sensitive hashes through designing multiple reasonable hash mapping, canquickly extract the image features, and position the duplicate images, also reduces feature comparison frequency of the images to 0, and greatly improves deduplication efficiency of the massive imagesin a case of ensuring accuracy.

Description

technical field [0001] The invention relates to the technical field of picture deduplication processing, in particular to a fast method for deduplication of massive pictures. Background technique [0002] The existing duplicate image removal method first extracts its color, texture, shape and other features from each image in the image data set, and then uses the similarity of the features to measure the similarity between images, thereby achieving the purpose of removing duplicate images . But once the number of images increases to a certain scale, the time consumption required to perform pairwise feature comparison on the images is very huge and unacceptable. However, for a large amount of image data, most of the images in the image collection have no relationship, so the pairwise feature comparison not only does not contribute to the deduplication of the image, but also consumes a lot of computing time. This makes deduplication of massive images inefficient and time-con...

AI Technical Summary

Problems solved by technology

However, when the high-dimensional image features are mapped to low-dimensional hash buckets to form m-dimensional hash features, the high-dimensional image features are very sparse compared to the m-dimensional Hamming space. If only Sparse image low-dimensional features are used as the extracted image features and used as the features of the image itself, which will cause a large number of different images to appear in the same hash bucket, making it necessary to take additional measures for images in the same hash bucket. Multiple pairwise precise feature comparisons can finally search for duplicate images, and different images may have the same hash code, resulting in image deletion errors

Images