Text information representation method and system, computer equipment and storage medium

Abstract

The invention belongs to the field of artificial intelligence, the invention relates to a text information characterization method and a system, computer equipment and a storage medium, and the methodcomprises the steps: obtaining a to-be-analyzed corpus, carrying out the word segmentation preprocessing of the to-be-analyzed corpus, generating corresponding word vectors based on the obtained segmented words, enabling the to-be-analyzed corpus to be text information, and enabling the text information to comprise at least one statement; obtaining word vectors of segmented words contained in each statement in the to-be-analyzed corpus to obtain a word vector group of each statement, and sequentially inputting the word vectors in the word vector group into the initial sentence vector algorithm model according to a sequence to generate initial sentence vectors of the corresponding statements; and inputting the initial sentence vector into a pre-trained sentence vector model to obtain a final sentence vector of each sentence, the final sentence vector being used for representing text information, and the pre-trained sentence vector model being generated based on a context relationship of the sentences. According to the method provided by the invention, the influence caused by different semantics of words in different sentences can be avoided, and the representation of the text information is more accurate.

Description

technical field [0001] The embodiments of the present invention belong to the technical field of artificial intelligence, and in particular relate to a text information representation method, system, computer equipment, and storage medium. Background technique [0002] In the field of natural language processing, text information representation is the basis for solving text processing problems. In the prior art, word vector summation based on Word2Vec is generally used as the text information representation method, but the semantics of the same word in different sentences and different contexts are different, so the representation of text information based on word vectors is inaccurate, and it is not suitable for the representation of text information such as article information in the field of information flow recommendation. Contents of the invention [0003] In view of this, the embodiments of the present invention provide a text information characterization method, sys...

AI Technical Summary

Problems solved by technology

[0003] In view of this, the embodiments of the present invention provide a text information characterization method, system, computer equipment, and storage medium to solve the problem that the text information characterization based on word vectors in the prior art is not accurate enough and is not suitable for article information in the field of information flow recommendation. The problem of representation of textual information of the class

Images