Tag noise correction based crowd-sourced tagging data quality improvement method

Abstract

The invention relates to a tag noise correction based crowd-sourced tagging data quality improvement method. The method comprises the following steps: running a tag integration algorithm in an initial crowd-sourced tagging data set to form a data set after tag integration, and estimating tagger quality and integrated tag quality information of samples in the process; performing multi-round K-fold cross validation by utilizing the data set after tag integration, and constructing a high-quality data set; determining a tag noise set in combination with the tagger quality and the tag quality of the samples by utilizing a prediction probability of a class tag of each sample in the multi-round K-fold cross validation process; and training a classification model by utilizing the high-quality data set generated in the multi-round K-fold cross validation process, and performing prediction and replacement on the class tag of each sample in the tag noise data set by using the model. With the tag noise correction method, the quantity of potential noise tag samples in the data set after original tag integration is reduced, so that the data quality is improved.

Description

technical field [0001] The invention belongs to the technical field of data labeling, and in particular relates to a method for improving the quality of crowdsourcing labeling data based on label noise correction. Background technique [0002] Obtaining high-quality labeled data is a fundamental task in the fields of information retrieval, machine learning, and data mining. Taking supervised learning in machine learning as an example, the whole learning process is to perform model training on a moderately scaled dataset with class labels, so as to obtain a learning model that can accurately predict unlabeled samples. Traditionally, the class labels in the training data are usually provided by experts in the application domain. The class labels provided by experts are highly accurate, which is conducive to building high-quality models. However, such expert annotation is itself costly. With the development of intelligent computing technology, more and more labeling requirem...

AI Technical Summary

Problems solved by technology

However, crowdsourcing annotation also has its inherent defects: the annotators are all ordinary users from the Internet, and compared with traditional expert annotation, the quality of their annotation is not guaranteed

[0004] One of the main reasons why the above label integration algorithm cannot further improve data quality is that the algorithm only uses label information from multiple uncertain labelers, while ignoring the characteristic information of the data itself.

Images