Short text classification method fusing knowledge graph and topic model

Abstract

The invention discloses a short text classification method fusing a knowledge graph and a topic model, which belongs to the field of natural language processing, and can effectively expand the feature content of a short text without changing the semantics of the original text by fusing the knowledge graph and the topic model technology and utilizing the knowledge graph to obtain external knowledge and perform feature expansion. And training a topic model by using the expanded text data, mining semantic association between the texts and taking the semantic association as an expansion feature, relieving inaccuracy of a text classification task caused by a synonym phenomenon, and finally performing classification prediction of the short texts by using a support vector machine.

Description

technical field [0001] The invention belongs to the field of natural language processing, and in particular relates to a short text classification method based on a knowledge map and a topic model. Background technique [0002] With the rapid development of the Internet, a large amount of short text data has been generated in the fields of online news, social media, instant messaging, etc. How to effectively mine valuable information from short text data is a current key research topic. [0003] Text classification has a wide range of applications, including information recommendation, automatic question answering, search engines, mail filtering, etc. In the past few decades, researchers at home and abroad have proposed and improved some machine learning and deep learning algorithms, and applied them in the field of text classification. These solutions include: using vector space models to represent text features, and then Use a classifier for text classification; or use de...

AI Technical Summary

Problems solved by technology

However, for short text classification problems, these methods have not achieved satisfactory results.

Compared with long texts, short texts have the characteristics of short length and weak topic clarity. After preprocessing such as removing stop words, there are usually only a few or a dozen words with actual meaning left, and it is difficult to construct high-quality texts. features for classification

Directly applying the method for long text classification to short text classification problems will result in sparse features, making it difficult to achieve accurate classification

In addition, the polysemy phenomenon in the text often brings a lot of noise to the text classification

Images