Question and answer corpus generation method and device based on text generation model

Abstract

The invention relates to the field of artificial intelligence, and provides a question and answer corpus generation method and device based on a text generation model, computer equipment and a storagemedium. The method comprises the steps of obtaining historical questions and a standard document, extracting keywords in the standard document and paraphrasing sentences corresponding to the keywords, performing word segmentation processing on the historical questions, identifying and discarding entity nouns in the historical questions to obtain syntactic feature words of the historical questions, combining the syntactic feature words with the keywords, and inputting the combined data into a pre-trained text generation model to obtain a target question corresponding to the keyword, wherein the text generation model by training based on a training sample marked with the keyword and syntax feature words are obtained, and according to the target question corresponding to the keyword and a paraphrasing statement corresponding to the keyword, a question-answer pair comprising the target question sentence and the paraphrasing sentence is constructed so as to improve the quality of the target question sentence and the question-answer pair.

Description

technical field [0001] The present application relates to the technical field of artificial intelligence, in particular to a question-answer corpus generation method, device, computer equipment and storage medium based on a text generation model. Background technique [0002] With the development of artificial intelligence technology, artificial intelligence has been applied in more and more scenarios. Among them, the question answering system is one of the important fields of artificial intelligence, especially for many businesses that currently need a customer service system to solve some of the user's questions, and most of the user's questions are concentrated on some high-frequency problems in the head. It is the motivation of Frequently Asked Questions (FAQ, frequently asked questions). [0003] The quantity and quality of the FAQ corpus are the basis of the entire system, but there is currently no way to provide a general and fully covered FAQ corpus, so each vertica...

AI Technical Summary

Problems solved by technology

[0003] The quantity and quality of the FAQ corpus is the basis of the entire system, but there is currently no way to give a general and fully covered FAQ corpus, so each vertical field needs to start building the FAQ corpus from scratch

Reconstructing the corpus usually uses the method of entering historical data to create FAQs. However, this data entry method will result in insufficient matching between some of the entered questions and answers.

Images