Data processing method and device, electronic equipment and computer readable medium

Abstract

The invention provides a data processing method comprising the steps that a data detection request is acquired, and the data detection request comprises multiple pieces of labeled data; for each pieceof labeled data, at least one pre-trained anomaly detection classifier is utilized to predict the labeled data to obtain a prediction result of each anomaly detection classifier for the labeled data,and the prediction result of the labeled data comprises information used for determining whether the labeled data is outlier data or not; and determining whether the labeled data is outlier data or not according to a prediction result of at least one anomaly detection classifier on the labeled data. The invention further provides a data processing device, electronic equipment and a computer readable medium.

Description

technical field [0001] The embodiments of the present disclosure relate to the technical field of data processing, and in particular, to a data processing method and device, electronic equipment, and a computer-readable medium. Background technique [0002] In the process of implementing artificial intelligence algorithms, data labeling and data cleaning are often the most energy-intensive links for labelers and algorithm engineers. At the same time, data quality has gradually become the upper limit of algorithm performance. Problems such as wrong labeling, missing labels, and ambiguous labeling, etc. It will cause great interference to the optimization process of the algorithm, making it difficult for the algorithm to clearly distinguish between positive and negative samples. [0003] At present, after the data labeling is completed, the marked data is usually reviewed and adjusted manually (experts), but the efficiency of manual review of the marked data is low, and the qu...

AI Technical Summary

Problems solved by technology

[0002] In the process of implementing artificial intelligence algorithms, data labeling and data cleaning are often the most energy-intensive links for labelers and algorithm engineers. At the same time, data quality has gradually become the upper limit of algorithm performance. Problems such as wrong labeling, missing labels, and ambiguous labeling, etc. It will cause great interference to the optimization process of the algorithm, making it difficult for the algorithm to clearly distinguish between positive and negative samples

[0003] At present, after the data labeling is completed, the marked data is usually reviewed and adjusted manually (experts), but the efficiency of manual review of the marked data is low, and the quality of the marked data is difficult to guarantee

Images