Data screening method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses a data screening method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a sample identification of a datasample marked with a category, and obtaining an identification vector corresponding to the sample identification; then, taking the category number of the labeled categories as a clustering category number to carry out clustering processing on the identification vector; then, for each clustering category, obtaining the similarity between the clustering center identification vector and the non-clustering center identification vector; then, determining a target non-clustering center identification vector of which the similarity with the clustering center identification vector does not reach a preset similarity in each clustering category, and judging the labeled category of the data sample represented by the target non-clustering center identification vector as labeling noise; and finally, filtering out the data sample corresponding to the target non-clustering center identification vector in each clustering category, thereby achieving the purpose of improving the labeling quality of thedata sample, and providing a high-quality data sample for machine learning.

Description

technical field [0001] The present application relates to the technical field of machine learning, and specifically relates to a data screening method, device, storage medium and electronic equipment. Background technique [0002] At present, in the field of machine learning, data samples are usually labeled manually, so that machine learning is performed based on the data samples and corresponding identification data to obtain corresponding functional models. However, when manually labeling data samples, labeling noise will inevitably be generated, and these labeling noises will have an impact on machine learning. Contents of the invention [0003] The embodiments of the present application provide a data screening method, device, storage medium and electronic equipment, which can improve the quality of labeling of data samples. [0004] In the first aspect, the embodiment of the present application provides a data screening method, including: [0005] Obtain the sample...

AI Technical Summary

Problems solved by technology

However, when manually labeling data samples, labeling noise will inevitably be generated, and these labeling noises will have an impact on machine learning

Images