Public opinion classification optimization method for long text

Abstract

The invention discloses a long text public opinion classification optimization method, which comprises the following steps of: a, performing public opinion judgment on an input text by using a traditional bert fine-tuning model, and judging whether the length of the text which is judged to be a neutral public opinion exceeds a set length threshold value or not; b, if not, maintaining an original public opinion judgment result, and if yes, performing more detailed public opinion analysis; and c, sending the current text to the pre-trained and fine-tuned bert models at the same time to obtain semantic vectors of each character in the current text before and after fine tuning. According to the method and the device, the character semantic change of the bert model before and after fine tuning is applied to the public opinion classification task for the long text; by identifying the text fragments with the public opinion tendency, the probability that the whole text fragments are judged to be neutral is reduced, and the detailed public opinion tendency of the user is better identified.

Description

technical field [0001] The invention relates to the technical field of text public opinion classification, in particular to a long text public opinion classification optimization method. Background technique [0002] When classifying public opinion on texts with many texts and long texts, the Bert model commonly used in the industry often gives a "neutral" judgment. This is because, on the one hand, in the longer content, most of the paragraphs are indeed neutral and objective statements, while a small number of text fragments expressing public opinion tendencies are only mixed in, and it is not easy to find even manual reading; on the other hand On the one hand, when the bert model classifies text public opinion, it only gives the judgment of public opinion tendency from the whole article, which can be considered as a weighted average of the public opinion tendency of the entire text, and the longer the article, the more likely it is to lower the judgment as Positive or ne...

AI Technical Summary

Problems solved by technology

This is because, on the one hand, in the longer content, most of the paragraphs are indeed neutral and objective statements, while a small number of text fragments expressing public opinion tendencies are only mixed in, and it is not easy to find even manual reading; on the other hand On the one hand, when the bert model classifies text public opinion, it only gives the judgment of public opinion tendency from the whole article, which can be considered as a weighted average of the public opinion tendency of the entire text, and the longer the article, the more likely it is to lower the judgment as Positive or Negative Likelihood

As a result, when classifying public opinion on long texts, the important public opinion fragments contained in it are ignored, and the overall public opinion judgment is given as neutral.

Images