Text classification auxiliary labeling method based on collaborative training

Abstract

The invention discloses a text classification auxiliary labeling method based on collaborative training, and relates to the technical field of text classification. The text classification auxiliary labeling method comprises the following steps: labeling sample data; constructing a data set; training two classifiers; classifying and recording classification results; extracting correct features and error features of the classifiers; performing sample optimization; retraining the replacement data set until the accuracy reaches a confidence threshold; and meanwhile, performing classification by using the two classifiers, and outputting a result under the condition that classification results are the same. According to the text classification auxiliary labeling method, most of simple data can be automatically labeled with high quality, so that the labeling accuracy and efficiency are greatly improved.

Description

technical field [0001] The present invention relates to the technical field of text classification, in particular to a text classification auxiliary labeling method based on collaborative training. Background technique [0002] In recent years, with the advent of the era of big data and the rapid development of machine learning and artificial intelligence, practitioners' demand for data has become more and more urgent, and the limited workload of labelers has become increasingly unable to meet people's demand for massive data. , and manual labeling brings many disadvantages, the time cost and economic cost of manual labeling are high, and there will inevitably be inevitable errors in manual labeling, which makes the quality of the labeled corpus not high. [0003] Artificial intelligence, especially natural language processing and image processing, is facing a difficult situation: supervised learning methods need to obtain a large amount of labeled corpus, and to obtain thes...

AI Technical Summary

Problems solved by technology

[0005] Machine learning labeling methods, such as supervised machine learning methods such as logistic regression, support vector machine, and naive Bayesian, can achieve fast labeling of corpus, but the training of these models depends on high-quality data labeling, and when dealing with multi-category The corpus, the performance is not good

[0006] Crowdsourcing labeling is based on crowdsourcing theory, specifying a special labeling system and labeling tasks for each task, and then randomly sampling the samples, and then handing them over to specialized personnel for labeling, but this requires relatively high requirements for professionals , the dependence is relatively large, and a large amount of data review work is required

[0007] In the field of text classification in natural language processing, the problem of multi-classification of text is often encountered, and there may be hundreds of labels, which requires a very large number of labels, and ordinary labelers are better at For the binary classification of the corpus, it is very difficult and inefficient for them to mark a large number of labels at one time

Images