LDA (latent dirichlet allocation) and VSM (vector space model) based similar Chinese herb literature recommendation method

Abstract

The invention discloses an LDA (latent dirichlet allocation) and VSM (vector space model) based similar Chinese herb literature recommendation method. The method includes: adopting an IKAnalyzer to perform word segmentation on topics and summary information of literature on the basis of a terminological dictionary for Chinese herbs, constructing a vector space, performing dimensionality reduction on the vector space, constructing a semantic dictionary, numbering all lexical items in the dictionary in sequence, performing vectorization through each document on the basis of the semantic dictionary, constructing term vectors of each document, utilizing LDA and a Gibbs sampling algorithm to perform training to obtain probability distribution of each document on themes, then computing a value of similarity between every two documents by the aid of KL divergence, computing cosine similarity of the term vectors of each document on the basis of term frequency, performing joint weighting on the two kinds of similarities prior to performing similarity sorting, and then making recommendation. By the method, the literature, similar both in content and theme, in the Chinese herb literature can be recommended to users, and recommendation results are closer to user requirements.

Description

technical field [0001] The invention relates to the technical field of computer similar literature recommendation, in particular to a method for recommending Chinese herbal medicine similar literature based on LDA (Latent Dirichlet Allocation) and VSM (Vector Space Model, vector space model). Background technique [0002] When users search for documents and view detailed information, they are often not satisfied with the information provided by one document, but also hope to view other documents with similar content. At this point, it is necessary to recommend documents similar in content to the current document to the user. [0003] Most of the traditional document similarity recommendation methods are based on the calculation of literal text content similarity. For example, the similarity calculation method based on TF-IDF is a very common method, but this type of algorithm has some defects, such as only being able to perceive the superficial meaning of the text, and unab...

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to overcome the defect that the above-mentioned existing methods can only perceive the superficial meaning of the text and cannot further mine the similarity at the hidden semantic level of the text, and provide a method for recommending similar documents of Chinese herbal medicine based on LDA and VSM

Images