Text abstraction method based on hierarchical interaction attention

Abstract

The invention relates to a text abstraction method based on hierarchical interaction attention, and belongs to the technical field of natural language processing. According to the method, the generation of the abstract is guided by extracting the feature information of different levels of the encoder through the level interaction attention. Meanwhile, in order to avoid the problem of information redundancy caused by introduction of different levels of features, data noise is compressed by utilizing a variational information bottleneck. Aiming at the generative text abstract, under an attention-based encoding and decoding framework, the multi-layer context information of the encoder is extracted through an attention mechanism to guide the decoding process, and meanwhile, the variational information bottleneck is introduced to constrain the information, so that the quality of the generative text abstract is improved. Experimental results show that the method can significantly improve theperformance of the encoding and decoding framework in a generative abstract task.

Description

Technical field [0001] The invention relates to a text summarization method based on hierarchical interactive attention, and belongs to the technical field of natural language processing. Background technique [0002] With the development of deep learning technology, generative text summarization methods have become a hot research topic. Traditional encoding and decoding models based on the attention mechanism usually only consider the high-level semantic information of the encoder as the semantic representation of the context, and ignore the detailed features such as the word-level structure obtained by the low-level neural network. The present invention proposes a multi-layer feature extraction and fusion method based on a hierarchical interactive attention mechanism to obtain features of different levels of the encoder, and at the same time introduce a variational information bottleneck at the decoding end to compress and denoise the fused information, thereby generating highe...

AI Technical Summary

Problems solved by technology

Traditional attention mechanism-based encoding and decoding models usually only consider the high-level semantic information of the encoder as the semantic representation of the context, while ignoring the detailed features such as the word-level structure obtained by the low-level neural network.

Images