Text classification method of gating loop unit based on residual jump connection

Abstract

The invention provides a text classification method of a gating loop unit based on residual jump connection. The method is characterized in that on the basis of an existing gating loop unit, residual information is introduced by utilizing jump connection and gating limitation of an expressway network is combined, and meanwhile, an unsaturated activation function and batch standardization are used to improve a definition formula of the gating loop unit, so that the gating loop unit can capture an ultra-long-term dependency relationship while coping with the gradient disappearance problem, and residual information is limited not to occupy a main position by utilizing gating of the expressway network. According to the method, the trained network model is utilized to automatically classify the Chinese texts and classify the positive / negative evaluation of the movie review, so that the labor cost is reduced. A plurality of groups of contrast experiments prove that the method has a certain applicability in a text classification task.

Description

technical field [0001] The invention relates to a text classification method of a gated recurrent unit based on a residual skip connection. Background technique [0002] In recent years, deep learning has been widely used in scientific and technological fields such as astronomy, geography, and industry. Time-series data is ubiquitous in our daily lives, from stock market trends, climate data in different cities, product sales over time, energy usage, and more. Among them, the text data on the network is also increasing day by day, and these data are mainly used in fields such as public opinion monitoring, information retrieval, and news text classification. If the data can be effectively classified, it will be more conducive to dig out valuable information, so the management and integration of text data is particularly important. Text classification is a basic task in natural language processing tasks. By using computer-related theoretical knowledge and skills, the purpose...

AI Technical Summary

Problems solved by technology

There are some imperfections in the existing text classification technology, such as: FastText method, its disadvantage is that word order information is not fully considered, TextCNN method, its disadvantage is that on the one hand, it cannot establish longer sequence information, on the other hand, its hyperparameters Adjustment is extra cumbersome

However, with the deepening of the network, the traditional RNN will have the problem of gradient disappearance

Although long-term short-term memory (LSTM) and gated recurrent unit (GRU) methods can capture long-term contextual information, the complexity of their gate structures makes training slow

Although the simple recurrent unit (SRU) method can reduce the training time of the neural network, its accuracy cannot meet the demand

Images