Image data set construction method and system and computer readable storage medium

Abstract

The invention discloses an image data set construction method and system and a computer readable storage medium. The method comprises the steps of dividing a first target hash value set which is obtained in advance to obtain corresponding hash value subsets having m hash values, with m being an integer larger than or equal to 1; extracting N hash values from any hash value subset respectively, andgenerating a first target hash value subset, wherein n is a positive integer smaller than or equal to m; calculating a union set of all the first target hash value subsets to obtain a second target hash value set, using a second target hash value set, and obtaining a corresponding image in the original image so as to construct a target image data set. The process of obtaining the first target hash value set includes obtaining an original image data set to obtain a corresponding original image, calculating a hash value of the original image, removing the repeated hash value according to the hash value obtained through calculation to obtain the first target hash value set. An image data set which is differentiated is constructed.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to an image data set construction method, system and computer-readable storage device. Background technique [0002] Data, algorithms, and computing power are the three pillars of machine learning. Data has a huge impact on the performance of machine learning models, and sufficient data is the basis for training machine learning models with good performance. The adequacy of data is not only reflected in the amount of data, but also in the diversity of data. Differentiated data is a more comprehensive description of the problem, and a large number of identical or similar data is just a repeated description of a certain aspect of the problem. For example, UAV inspections of power transmission lines usually collect a large number of images, and many factors lead to many identical and similar images: (1) For the inspection of multiple circuits on the same tower, the exact sa...

AI Technical Summary

Problems solved by technology

Therefore, even if a large amount of data is used to train the machine learning model, because it contains a large amount of the same and similar data, it not only wastes the cost of data labeling, but more importantly, it is difficult to train a machine learning model with good performance.

Images