Semantic-based candidate answer screening method and system for machine reading understanding

Abstract

The invention discloses a semantic-based candidate answer screening method and system for machine reading understanding. The method comprises the following steps of screening out the candidate answerfragments corresponding to a document from the document according to answers; screening out the optimal candidate answer segment from the candidate answer segments according to the semantic similarityand the answers; and obtaining a candidate answer document from the document according to the semantic similarity and the answers, and calculating to obtain the position information of the optimal candidate answer segment in the candidate answer documents, wherein the candidate answer document and the position information are the input values at a machine reading understanding model training stage. According to the present invention, the answers are screened according to the semantic similarity, screening is more comprehensive, the correct answers are not easy to miss, and the higher accuracyis achieved. The semantic-based candidate answer screening method and system for machine reading understanding can be widely applied to the field of natural language processing.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a candidate answer screening method and system for machine reading comprehension based on semantics. Background technique [0002] In recent years, machine reading comprehension tasks have received extensive attention in the field of natural language processing. Major scientific research institutions have also launched many related data sets and competitions for machine reading comprehension. Traditional cloze and multiple-choice tasks have achieved very high accuracy. In the traditional reading comprehension model, the data set for model training usually uses a data set of a document and a question, but with the progress of the times, the data set that everyone is most concerned about is the data generated based on real search engine data Compared with the traditional dataset of one document and one question, such datasets usually contain multiple relevant documents ...

AI Technical Summary

Problems solved by technology

Although this type of method has achieved good results, this type of method simply analyzes from the statistical level, but ignores the polysemy characteristics of the language; in the multi-document multi-answer reading comprehension task, the literal expression between different candidate answers is often The forms are different, but there is a certain correlation at the semantic level, and the traditional method based on statistical analysis is often difficult to capture the semantic correlation of different answers, so it is difficult to make full use of the information of multiple candidate answers

Therefore, the prior art has the problems of low screening accuracy and high probability of missing answers

Images