A text classification method based on a local and global mutual attention mechanism

Abstract

The invention discloses a text classification method based on a local and global mutual attention mechanism, and the method comprises the following steps: obtaining text data, carrying out the preprocessing of the text data, and expressing text words through a pre-trained word vector; Capturing global long-term dependence of the text sequence by using a long-short-term memory network, and acquiring local semantic features of the text sequence by using a multi-scale convolutional neural network; Taking the global long-term dependency and the local semantic features as input of a local and global mutual attention mechanism to obtain weighted global long-term dependency and weighted local semantic features; Carrying out weighted pooling to obtain a final global representation vector and a final local representation vector; And inputting the vectors to a full connection layer to fuse the global representation vector and the local representation vector, and then inputting the vectors to a classification layer to classify. According to the method, global long-term dependency and local semantic features are captured in parallel, interaction between the two features is learned in an explicit mode, better text global and local feature representation is obtained, and the text classification precision is further improved.

Description

technical field [0001] The invention relates to the technical field of text classification, in particular to a text classification method based on a local and global mutual attention mechanism. Background technique [0002] Text classification is a fundamental problem in natural language processing that requires the assignment of one or more predetermined categories to a sequence of text. The core of text classification is to learn a sequence representation to deal with issues such as sentiment analysis, question classification and topic classification. [0003] At present, to learn a sequence representation, the common method is to model the long-term dependent representation of the sequence or the local semantic features of the sequence. The convolutional neural network can better extract the local semantic features of the text sequence through the convolution kernel. Y.Kim proposed a multi-channel convolutional neural network, using static word vectors from word2vec and...

AI Technical Summary

Problems solved by technology

C-LSTM can capture global long-term dependencies and local semantic features, but these two kinds of information are connected in a cascading manner, which makes the model deeper and easily leads to the problem of gradient disappearance

Also, this approach makes no attempt to learn the interaction between the two types of information

Images