Method and device for obtaining question-answer-related paragraphs based on semantic change manifold analysis

Abstract

The present invention provides a method, device, and computer storage medium for acquiring question-and-answer-related paragraphs based on semantic change manifold analysis, which can quickly and accurately extract paragraphs that may contain answers, and improve the efficiency of open-domain question-and-answer, including steps: according to the question, in Search in various public search engines on the Internet, extract paragraphs from them as corpus for matching degree calculation, perform word segmentation, and splice the word segmentation results into question word sequences and paragraph word sequences, perform sliding window slicing to obtain paragraph subword sequences, and then perform embedding representation. Then through the mapping model conversion, two mapping vectors of the same dimension are obtained and the similarity is calculated, and the similarity is obtained to form a semantic change trend curve. The manifold feature is extracted by the method of manifold learning, and the high-dimensional mapping of the manifold feature is constructed. Dimensional mapping inputs the trained weight model to obtain the matching score between the question and each paragraph in the matching calculation corpus, and the k paragraphs with the highest scores are taken as the most relevant paragraphs for question answering.

Description

technical field [0001] The invention belongs to the technical field of open domain question answering and deep learning, and relates to a method and a device for obtaining relevant paragraphs of question answering based on semantic change manifold analysis. Background technique [0002] Open-domin QA (Open-domin QA) does not directly provide a document or a given paragraph when a given question is given, but needs to find answers in a large document collection or the entire Internet. Generally speaking, open-domain Q&A needs to retrieve relevant documents according to a given question until a paragraph is searched, and then give an answer through reading comprehension. This process usually requires scoring and sorting the paragraphs to complete. In addition, for compound questions that may exist, it may also be necessary to search for multiple paragraphs to support multi-step reasoning and find the final answer based on bridging information. [0003] Machine reading compreh...

AI Technical Summary

Problems solved by technology

[0003] Machine reading comprehension extracts or comprehends answers from a given paragraph to answer questions. The process often requires complex mathematical probability models and calculation steps to achieve, and its direct application in large-scale document collections consumes huge resources.

Images