Data dynamic labeling method and device for coarse-grained text classification

Abstract

The invention provides a data dynamic labeling method and device for coarse-grained text classification. The method comprises the steps of balancing the labeling data according to the label category proportion; constructing a text TF-IDF word frequency matrix; performing feature screening by using chi-square distribution to obtain a training data set; training the training data set by using a machine learning algorithm to obtain an initial model; acquiring a test data set, and labeling data of a first preset data volume in the test data set by using the initial model to obtain predicted labeling data; obtaining data corresponding to each data annotation label obtained by respectively extracting a second preset number of pieces of predicted annotation data according to the label category for auditing; adding the data corresponding to each data labeling label into a training data set, and training the training data set by using a machine learning algorithm to obtain a correction model; judging whether the training data volume meets a second preset data volume or not, and if not, continuing to execute the process; wherein the storage correction model is a prediction model.

Description

technical field [0001] The present invention relates to the technical field of data labeling, in particular to a method and device for dynamic data labeling for coarse-grained text classification. Background technique [0002] Coarse-grained text classification is a classification method based on the sentence level. For common coarse-grained text classification projects, they all rely on supervised learning for processing. Therefore, having a good quality dataset is the basis for model building. However, the traditional data labeling method relies on manual labeling behaviors. This processing method is likely to cause the following errors: 1. Each data labeler has a different understanding of the text, which leads to deviations in text data labeling. 2. Due to the existence of prior knowledge, each labeler has a wrong understanding of the subject content of the text, so the category label is wrong. Contents of the invention [0003] The present invention aims to provide...

AI Technical Summary

Problems solved by technology

However, the traditional data labeling method is based on the behavior of manual labeling. This processing method is likely to cause the following mistakes: 1. Each data labeler has a different understanding of the text, which leads to deviations in text data labeling

Images