Statement similarity detection method and device, electronic equipment and storage medium

Abstract

The invention relates to an artificial intelligence technology, and discloses a statement similarity detection method, which comprises the following steps: constructing a basic language model, and replacing a coding layer with a pre-trained Chinese pre-training language model to obtain a multi-task language model; performing multi-task model training on the multi-task language model by utilizing a training data set; obtaining a text statement input by a user, and obtaining a related statement set according to the text statement; calculating the similarity between the text statement and the related statement set by using a double-sentence similarity detection network in the trained multi-task language model to obtain a similarity value; and sorting and displaying the related statement set according to the similarity value. In addition, the invention also relates to a block chain technology, and the related statement set can be stored in nodes of a block chain. The invention further provides a statement similarity detection device, electronic equipment and a computer readable storage medium. The problem that the accuracy of statement similarity detection is low can be solved.

Description

technical field [0001] The invention relates to the technical field of artificial intelligence, in particular to a sentence similarity detection method, device, electronic equipment and computer-readable storage medium. Background technique [0002] With the advent of the era of big data, the scale of the amount of information on the network continues to expand, how to obtain the desired information from massive amounts of information, that is, information retrieval, is becoming more and more important. Information retrieval often obtains data information related to the input text by detecting the similarity of sentences. [0003] The existing method is to use the language model to identify and detect the similarity of the sentence, but the existing language model is based on the training of a specific field and a small amount of labeled data. Due to the small amount of labeled data, in The generalization ability in actual use is poor, and the accuracy of the results is low...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a sentence similarity detection method, device and computer-readable storage medium, the main purpose of which is to solve the problem of low accuracy of sentence similarity detection

Images