Text processing method and system based on parallel zero-redundancy long short-term memory network

Abstract

The invention belongs to the field of text information processing, and provides a text processing method and system based on a parallel zero-redundancy long short-term memory network, wherein the method comprises the steps of obtaining to-be-processed text data and converting the to-be-processed text data into a word embedding vector form; according to the number of words contained in the to-be-processed text data, adaptively calculating a context window coverage range of each word in the to-be-processed text data; in a parallel zero-redundancy long-short-term memory network, compressing all word embedding vectors within the coverage range of the context window to form a local attention vector matrix, and performing parallel calculation through Hadamard product matrix multiplication to obtain local context vectors corresponding to all word embedding vectors; and processing the local context vector corresponding to the to-be-processed text data through the classification network model to obtain a text classification or labeling result.

Description

technical field [0001] The invention belongs to the field of text information processing, and in particular relates to a text processing method and system based on a parallelized zero-redundancy long-short-term memory network. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] With the rapid development and continuous maturity of the new generation of artificial intelligence (AI) technology, the computing power of cloud computing, cluster computing and small servers continues to increase, and the widespread popularity of smart devices, various Internet applications have become part of the daily life of users. Indispensable content. At the same time, natural language applications such as news topic tracking, social computing, and public opinion analysis have also been included in the development plan of e-government and smart government. The...

AI Technical Summary

Problems solved by technology

[0005] The structural characteristics of LSTM (long-term short-term memory network) provide the ability to discover the associated information between words, but there are still the following problems: 1) Serial sequence characteristics: LSTM is a one-way model that can only read words in order, It may ignore the local semantic environment of natural language (such as flashbacks, emphasis, etc.), although bidirectional LSTM can provide additional reverse sequences, but its essence is still one-way propagation

Moreover, the design characteristics of LSTM determine that the model does not support parallel computing, and the next word must be processed after the current word is processed, which reduces the text processing speed

2) The number of cycles is not fixed: when the text data is long, the hidden state after multiple iterations may ignore long-span word associations, which brings difficulties to semantic analysis

Usually, shallower models only capture a small amount of contextual information, which is not conducive to the extraction of semantic information

[0006] To sum up, due to the existing LSTM structure, the text processing speed is slow, the amount of semantic information extraction is small, and the semantic analysis is difficult, which ultimately reduces the text processing efficiency.

Images