Semantic text similarity calculation method based on attention

Abstract

The invention discloses a semantic text similarity calculation method based on attention. The method comprises the steps: 1, carrying out preprocessing on each text pair in a data set so as to obtaincorresponding text data samples; 2, dividing all the samples into a training set and a verification set, establishing a neural network, and training the neural network by using the samples in the training set to obtain a network model for semantic text similarity calculation; and 3, preprocessing a text pair to be calculated, and inputting the preprocessed text pair into the network model to obtain a semantic similarity calculation result of the text pair. The neural network designed by the invention can more effectively extract semantic information of texts and extract finer-grained interaction information between two texts; and the neural network uses a plurality of attention mechanisms to enhance important information in interaction information between two texts and improve semantic text similarity calculation accuracy.

Description

technical field [0001] The invention belongs to the technical field of natural language processing, and in particular relates to an attention-based semantic text similarity calculation method. Background technique [0002] Semantic text similarity calculation is an important research topic in the field of natural language processing. Semantic text similarity aims to calculate the degree of semantic equivalence between two sentences / texts. It can be applied to many tasks, such as machine translation, paraphrase problems, Automatic question answering, text classification, information retrieval, etc. [0003] At present, there have been many research results in the calculation of semantic text similarity, which can be summarized into the following three categories: [0004] (1) The semantic text similarity calculation method based on literal matching, the typical semantic similarity calculation method based on TF-IDF, the semantic text similarity calculation method based on TF...

AI Technical Summary

Problems solved by technology

[0004] (1) The semantic text similarity calculation method based on literal matching, the typical semantic similarity calculation method based on TF-IDF, the semantic text similarity calculation method based on TF-IDF is to model the text into a word frequency vector, using Cosine similarity is used to measure the similarity between texts; the advantage of this method is fast calculation speed and small workload, but the disadvantage is that it ignores the semantic information of words and needs to manually set the stop vocabulary

[0005] (2) The probabilistic topic semantic text similarity calculation method based on latent semantic analysis, the typical semantic text similarity calculation model based on LDA (Latent Dirichlet Allocation), the main idea is to use the common information in the words to construct the topic of the text model, dig out the potential semantic information in the text, and thus calculate the semantic similarity between the texts; the advantage of this type of method is that it takes into account the deep semantic information of words, and the accuracy rate is higher than that of the (1) type method. The disadvantage is that there is no Considering the positional relationship between words and words, it is limited by the type of samples

[0006] (3) The semantic text similarity calculation method based on deep learning. Two types of deep learning frameworks are proposed for the semantic text similarity calculation. The first framework is based on the Siamese network, and the second framework is called matching aggregation. The idea of ​​the framework is to calculate the semantic similarity from multiple perspectives by capturing the interaction features between the text representations of the input sentences. The difference between the two is mainly reflected in the capture of the interaction features; the advantage of the Siamese network framework is that the shared parameters make The model is easier to train, the disadvantage is that during the encoding process, there is no clear interaction between the two sentences, and some important information may be lost

Images