Online theme modeling method on basis of theme heredity

Abstract

The invention discloses an online theme modeling method on the basis of theme heredity. The online theme modeling method includes the steps of: capturing text data of a current time slice, performing theme modeling according to an LDA (latent dirichlet allocation) model, computing theme strength, ranking a theme, computing a gene of the theme, capturing text data of the next time slice, converting distribution vectors of theme-vocabulary, computing prior parameters of Dirichlet distribution of the next time slice, adopting a Gibbs sampling method and the like. The online theme modeling method has the advantages that 1, an online theme model is suitable for processing of time-sequential text streams and can be applied to a public opinion monitoring system greatly; 2, alignment features of the theme in an OLDA (online latent dirichlet allocation) model are reserved, different genes are set for the themes according to the theme strength, and the defects that the themes are mixed and new themes are not detected timely are overcome; 3, scores of broad themes can be effectively lowered by the aid of a theme strength computing method.

Description

technical field [0001] The invention relates to a discovery and evolution analysis method of network hot topics, in particular to an online topic modeling method based on topic inheritance. Background technique [0002] Topic discovery and evolution analysis for emerging media such as forums and microblogs is a current research hotspot. It can provide support for the discovery and prediction of public opinion, which is conducive to the timely disposal of public opinion and the maintenance of social stability. In recent years, topic models represented by LDA (Latent Dirichlet Allocation) have been intensively studied. Due to the topic modeling ability of LDA itself, it has inherent advantages in the field of topic evolution. Some extended models based on LDA were subsequently proposed, such as the TOT (Topic Over Time) model that reflects the temporal intensity changes of topics, the DTM (Dynamic Topic Model) model that uses state space to record topic content and intensity ...

AI Technical Summary

Problems solved by technology

But the weight vector ω of OLDA δ It is a fixed value and cannot be adjusted according to the dynamic changes of the theme

Moreover, each topic in the same time slice shares the same weight, and it is difficult to set this value

If the weight setting is too small, the front and back topics cannot be aligned; if the weight setting is too large, the influence of historical data will be too high, resulting in some topics that are not the same event being forced to align together due to the appearance of co-words

Especially when a new topic appears in the t time slice, the topic is easily mixed with an old topic and aligned with the related topic in t-1, making it difficult to detect the new topic

In addition, OLDA maintains an incrementally updated vocabulary, and new words in each time slice are added to the vocabulary, which will eventually cause memory overflow due to the large vocabulary, and the increase in processing dimensions will increase the running time

Images