Multi-round dialogue device and method

Abstract

The invention discloses a multi-round dialogue device and method.The multi-round dialogue device comprises a data processing module, a characterization module, a feature extraction module, a question and answer feature similarity module and an objective function module. The data processing module is used for analyzing multi-round dialogue data of historical chats to obtain input data; the representation module is used for mapping input data to obtain a sentence vector set; the feature extraction module is used for analyzing the sentence vector set; the question and answer feature similarity module is used for processing the sentence vector set to obtain a score matrix; the objective function module is used for setting an objective function suitable for the multi-round dialogue device according to the scoring matrix. Under the condition that the sample size is not large, the multi-round dialogue device can learn good context features, so that the questions of the user can be predicted more accurately and answers can be provided, the network structure is simple, and a lightweight memory and an energy-saving model are realized.

Description

technical field [0001] The invention relates to the field of natural language processing, in particular to a multi-round dialogue device and method. Background technique [0002] The general pre-training language model, such as the bert model, is a multi-layer bidirectional transformer network structure, which performs self-supervised learning on the basis of massive corpus. The feature representation obtained through the bert model greatly improves the accuracy of natural language processing tasks. , but each layer of the bert model is self-supervised, resulting in an overall complexity of O(n2), which requires a lot of machine resources. [0003] In a multi-round and context-dependent chat system, the bert model is not ideal. In addition to the large amount of calculation, slow speed, and high training cost, its biggest defect is that the bert model is a model based on general corpus training, because the general corpus The learned semantic features lack strongly relevant...

AI Technical Summary

Problems solved by technology

[0002] The general pre-training language model, such as the bert model, is a multi-layer bidirectional transformer network structure, which performs self-supervised learning on the basis of massive corpus. The feature representation obtained through the bert model greatly improves the accuracy of natural language processing tasks. , but each layer of the bert model is self-supervised, resulting in an overall complexity of O(n2), which requires a lot of machine resources

[0003] In a multi-round and context-dependent chat system, the bert model is not ideal. In addition to the large amount of calculation, slow speed, and high training cost, its biggest defect is that the bert model is a model based on general corpus training, because the general corpus The learned semantic features lack strongly relevant contextual dialogue information, and most of the corpus is based on documents, lacking dialogue data. Therefore, using the bert model for multiple rounds of dialogue cannot improve the robot's natural language understanding ability and accuracy of intention judgment.

Especially in the colloquial, specific scene or professional industry knowledge field, in the multi-round chat system with semantic correlation between the upper and lower sentences, the expressive power is limited and the accuracy rate is not high

Images