Machine reading understanding method based on a multi-head attention mechanism and dynamic iteration

Abstract

The invention provides a machine reading understanding method based on a multi-head attention mechanism and dynamic iteration, and belongs to the field of natural language processing. The constructionmethod of the machine reading understanding model comprises the following steps: constructing an article and problem coding layer; constructing a recurrent neural network based on bidirectional attention flow; a self-attention layer is constructed and answer output is predicted based on a dynamic iterative decoder. According to the method, answer prediction can be carried out on questions in a machine reading understanding task text; according to the invention, a new end-to-end neural network model is established, and a new idea is provided for exploring a machine reading understanding task.

Description

technical field [0001] The invention belongs to the field of machine reading comprehension, and relates to a method for encoding articles and questions, and then using a bidirectional attention flow, a self-attention layer and a dynamic iterative decoder to predict answer output. Specifically, it refers to constructing an article question encoding layer, constructing a recurrent neural network based on a bidirectional attention flow, constructing a self-attention layer, and predicting answer output based on a dynamic iterative decoder. Background technique [0002] The main form of the question-answering reading comprehension task is to give a short passage with a certain vocabulary and questions based on the short passage, and limit the answer to a text fragment in the original text. It needs to pass a certain form of logic on the basis of fully understanding the original text. Infer answers to prediction questions. At present, the mainstream models in this field mainly in...

AI Technical Summary

Problems solved by technology

As an early model tested on the SQuAD dataset, Match-LSTM provides ideas for the design of subsequent excellent models, but there are also many problems: the model only calculates the one-way attention of the article word vector to the question, and loses a lot of semantics information, does not perform well when the answer is long, and the exact match results are only about 30%

BiDAF has achieved excellent results in the SQuAD 1.1 dataset evaluation, but there are still problems: there is no similar self-matching process, and the context dependencies cannot be obtained well

R-Net has achieved a performance close to that of humans in the SQuAD 1.1 dataset evaluation, but there are still some problems: (1) Both the BiDAF model and the R-Net model use the pointer network to predict the answer position at one time, and may not be able to obtain the global optimum untie

(2) R-Net does not have a mechanism similar to two-way attention flow, and cannot fully integrate the semantic information of articles and questions

Images