Text classification method and device, medium and electronic equipment

Abstract

The invention relates to the field of natural language processing, and discloses a text classification method and device, a medium and electronic equipment. The method comprises: obtaining target text data; inputting the target text data into a pre-trained text classification model; outputting compressed sentence representation information corresponding to the target text data and an expected value corresponding to the compressed sentence representation information through a variational information bottleneck processing layer; outputting classification prediction information through a classification module according to the compressed sentence representation information received from the variational information bottleneck processing layer; and generating and outputting a classification label corresponding to the target text data through a classification label generation layer according to the classification prediction information received from a classification module and an expected value corresponding to the compressed sentence representation information received from the variational information bottleneck processing layer. According to the method, the occurrence of an over-fitting phenomenon is reduced, and the popularization and application range of the pre-training model is expanded.

Description

technical field [0001] The present disclosure relates to the technical field of natural language processing, and in particular to a text classification method, device, medium and electronic equipment. Background technique [0002] Currently, pre-trained models are widely used in downstream tasks and have achieved good results. However, the size and parameters of the pre-training model are very large. Once the sample data of the target task is small, directly applying this type of pre-training model will often lead to the establishment of a wrong connection between a large amount of task-irrelevant information and the target label. The phenomenon of overfitting makes it difficult to learn effective information only through fine-tuning. At the same time, since manual labeling of data is very expensive and time-consuming in real-world application scenarios, the amount of sample data for many downstream tasks is very limited, which limits the popularization and application of p...

AI Technical Summary

Problems solved by technology

However, the size and parameters of the pre-training model are very large. Once the sample data of the target task is small, directly applying this type of pre-training model will often lead to the establishment of a wrong connection between a large amount of task-irrelevant information and the target label. Overfitting phenomenon makes it difficult to learn effective information only through fine-tuning

At the same time, due to the fact that manual labeling of data is very expensive and time-consuming in real application scenarios, the amount of sample data for many downstream tasks is very limited, which limits the popularization and application of pre-trained models.

Images