Information retrieval method and system based on interactive document clustering

Abstract

The invention provides an information search method and system based on interactive document clustering. The method comprises the following steps that a document set is horizontally partitioned and preprocessed; word frequency statistics is conducted, and high-frequency words constitute a characteristic word set; vector space representation of documents is generated, the distances between the documents are calculated, and a similarity matrix is generated; a Laplacian matrix is generated, the number of clusters and a representation matrix are determined according to intervals between proper values of the Laplacian matrix, secondary clustering is conducted, and initial distance results are obtained; users conduct interactive operation on the initial distance results, new characteristic words are mined through chi-square statistics, a vector space is reconstructed, and the clustering process is repeated; finally, clustering results are shown to the users, and therefore the users obtain different categories of search results. According to the information search method and system, a semi-supervised learning approach in which the users intervene is adopted, the documents are clustered and analyzed, and the users obtain the different categories of search results.

Description

technical field [0001] The invention belongs to the field of information technology, and relates to the technology of using computer-aided mining and organizing text information, in particular to an information retrieval method and system based on interactive document clustering, which uses a semi-supervised learning method with user intervention to cluster documents Category analysis for users to obtain different categories of search results. Background technique [0002] With the rapid development of social networks and the influx of information into massive computing environments, modern people are bombarded by excessive information fatigue. Most people are overwhelmed just receiving "new information", let alone have enough time to digest and absorb each information. Text information is a typical type of mass information. Mining and organizing text information with the help of computer is of practical significance. [0003] The document clustering system can provide an ...

AI Technical Summary

Problems solved by technology

Once there is a problem in the preprocessing stage, several documents under a small category should actually belong to two major categories, then there is no corresponding operation to handle such exceptions

[0011] In addition to the clustering method described above, there are other methods, such as topic model [Latent DirichletAllocation, Journal of Machine Learning Research, David M. Blei etc., 2003, page993-1022] to mine potential semantic rules in document sets, but it The disadvantages are also obvious: the number of topics needs to be specified, the document does not explicitly belong to a topic, and the direct readability is relatively weak

Images