End-to-end classification method of large-scale news text based on Bi-GRU and word vector

Abstract

The invention provides an end-to-end classification method of a large-scale news text based on Bi-GRU and a word vector. The end-to-end classification method comprises the following steps: S1. word Embedding word-level semantic feature representation is performed; S2. the attention weight Bi-GRU word level sentence feature coding model is constructed; S3. the Bi-GRU sentence level feature coding model based on the attention weight is established; S4. hierarchical Softmax is applied to realize end-to-end classification implementation. According to the method, the dimension of the vector can bereduced and the problem that the features are too sparse can be effectively prevented. The final output vector is optimized and the effectiveness of model feature coding is enhanced. The problem thatthe model is difficult to train because of the high dimension can be avoided and the additional semantic information can also be provided. The feature extraction model and various common classifiers can be flexibly combined so as to facilitate replacement and debugging of the classifiers. The computational complexity is reduced from | K | to log | K | in comparison with that of Softmax.

Description

technical field [0001] The present invention relates to a text classification technology for large-scale news corpus, in particular to an end-to-end classification method for large-scale news texts based on Bi-GRU (Bi-directional Gated Recurrent Unit) and word vectors, which integrates text Vector representation technology and deep learning model Bi-GRU principle. Aiming at the feature selection problem of text classification, a Bi-GRU model is designed to extract features, which improves the long-term dependence of the sequence neural network model in the training of semantic features in long texts. At the same time, through the method of attention mechanism, the semantic code containing the attention weight distribution of the input sequence nodes is obtained, and it is used as the input of the classifier, which reduces the information loss and information redundancy in the feature vector extraction process. The invention belongs to the field of natural language processing....

AI Technical Summary

Problems solved by technology

Among them, the cyclic neural network model is one of the main research methods for text classification because it can represent the sequence semantic and grammatical features of the text corpus. However, the classic RNN model has gradient explosion and disappearance problems in the solution process, and the effect on long texts is not good.

Images