Text classification method and system

Abstract

The invention discloses a text classification method and system. According to the method, the limitation of Bert based on word coding is considered; internal relation information of characters in words is seriously lost. According to the invention, a word position coding mode is added on the basis of Bert; therefore, the obtained sentence vector is composed of a character vector, a sentence vector, a word vector and position information, sentence vector representation containing information of words is obtained, classification model training is conducted through the sentence vector training model, and finally the model is used for Chinese sentence classification.

Description

technical field [0001] The invention relates to a method for constructing a text classification model in the field of deep learning, in particular to an improved method of a Bert model and its application in text classification. Background technique [0002] The text classification problem is a classic problem in the NLP field, and it has accumulated many methods. Early text classification was mainly based on traditional machine learning methods, such as text classification based on TF-IDF. With the development of deep learning, many text classification models based on deep learning have emerged, such as Fast-Text, Text-CNN, etc. Until October 2018, Google officially released a transfer learning model Bert and achieved excellent results. It completely changed the relationship between pre-trained word vectors and downstream specific NLP tasks, and used transfer learning methods to solve problems in the NLP field. Questions have also become an important direction. [0003...

AI Technical Summary

Problems solved by technology

[0003] However, the Bert model is encoded by words without considering the relationship between words, which leads to the problem that it still has poor results in text classification

Images