Generative abstract extraction method based on encoder-decoder

Abstract

The invention discloses a generative abstract extraction method based on an encoder-decoder, and the method is based on an encoder-decoder framework containing an attention mechanism. The idea of a variational self-coding network and replication is introduced. An encoder part of the method is the same as a conventional encoder. A basic unit used is a GRU. The decoder part has three layers in total, the first layer and the second layer are GRU decoding layers and are used for determining the output of the part, the third layer is a variational self-encoding layer and is used for outputting a potential structure variable part, and an attention mechanism is introduced into the second layer of GRU. In an output part of the network, the hidden layer states of a copying part and a generating part are combined into a whole and then mapped into the output of the network. Meanwhile, a historical dependency is added to the variational self-coding layer, so that the variational self-coding layercan adapt to a time sequence network. According to the method, the state information of the hidden layer of the encoder is fully utilized, the output accuracy is improved, and better performance is achieved.

Description

technical field [0001] The invention relates to the technical field of abstract extraction, in particular to an encoder-decoder-based generative abstract extraction method. Background technique [0002] Nowadays, the Internet is full of different kinds of information, how to extract the most critical and effective information from the massive information has become a very important research direction. Text summary extraction can automatically generate key summaries from single or multiple documents, helping people quickly obtain the information they need. At present, abstract extraction technology has been applied in various fields related to text, mainly to deal with the problem of information overload. For example, abstract extraction technology can automatically extract news headlines to help people understand the most valuable news content in the shortest time. [0003] The current mainstream text summarization algorithms can be roughly divided into two types: extracti...

AI Technical Summary

Problems solved by technology

[0005] Although the attention mechanism has been introduced in many existing models, they are not very sufficient. If the role of the attention mechanism can be emphasized more in the decoding part of the model, and the output state of the model itself is not discarded, but they are At the same time as part of the output state, it will be possible to obtain better model performance

Images