Deep learning-based text similarity detection method for financial industry

Abstract

The invention provides a deep learning-based text similarity detection method for a financial industry, and the method comprises the steps: S1, building a special noun lexicon, obtaining a conditionalprobability model based on a conditional random field, and carrying out the probability calculation through the conditional probability model; S2, using a Bi-LSTM-RNN model to take out each word in the sentence according to the sequence, extracting the information of the word, and embedding the information into a semantic vector, thereby obtaining the semantic representation of the sentence; S3,analyzing a logic structure of the sentence according to the semantic information extracted by the neural network, organizing the sentence into a tree structure, and finally expressing the paragraph according to a vector tree mode; and S4, matching the vector tree extracted from the text with a historical data document in a database, and comparing similarities from two angles respectively, one being the similarity between the vector trees, and the other being the similarity between every two nodes, so as to finally obtain a result.

Description

technical field [0001] The invention belongs to the field of natural language processing, and specifically designs a text similarity detection method based on deep learning for the financial industry. Background technique [0002] With the development of information technology and artificial intelligence, more and more data are sorted, how to make good use of these large amounts of historical data has become very important. For many technology companies, historical data generated by users is an extremely precious asset. At this stage, many numerical and structured storage data have been well utilized, and various data mining methods emerge in endlessly, but for natural language data, they have not yet been well structured and mature. mining technology. At present, there are documents written in natural language in many fields, and these historical documents can also guide new problems. Due to the characteristics of the language in the Chinese field, natural language proce...

AI Technical Summary

Problems solved by technology

[0004] But simply considering the coincidence of words is biased in some fields

At the same time, for the method of constructing the syntax tree, if the structure of the sentence is too long, the syntax tree will be too complicated, so that the comparison process will fall into the partial tree structure

Images