A weighted association rule mining method for Chinese inter-word matrix based on item frequency and weight

Abstract

The invention discloses a Chinese inter-word matrix weighted association rule mining method based on item frequency and weight. First, the Chinese documents to be mined are preprocessed by removing Chinese stop words, extracting feature words, and calculating feature word weights, respectively. Construct the Chinese feature word library and Chinese document index library; use the matrix weighted support calculation method based on item frequency and weight to mine the weighted frequent itemsets of the Chinese feature word matrix, and obtain the weighted frequent item set of the Chinese feature word matrix; use the confidence - Interest degree evaluation framework mines the weighted association rule mode of the Chinese feature word matrix from the weighted frequent item set of the Chinese feature word matrix. The method of the present invention fully considers the occurrence frequency and weight value of the feature words in the document, and can excavate a more practical, more reasonable and Chinese inter-word matrix weighted association rule pattern that can better reflect various association relationships between the feature words. Patterns applied to the field of information retrieval query expansion can improve the performance of information retrieval.

Description

technical field [0001] The invention belongs to the field of Chinese text mining, in particular to a Chinese inter-word matrix weighted association rule mining method based on item frequency and weight. Background technique [0002] In the study of association pattern mining, the core problem is the calculation of the support degree of association patterns. In the current research, there are mainly three types of association model support calculation methods as follows: the first type is the unweighted association model support calculation method (see literature R. Agrawal, T. Imielinski, A. Swami. Mining association rules between sets of items in large database[C].In Proceeding of 1993ACM SIGMOD International Conference on Management of Data,Washington D.C.,1993,(5):207-216.), this is an early classic support calculation method, which combines the association mode in the transaction The probability of occurrence in is taken as the support of the association mode. This met...

AI Technical Summary

Problems solved by technology

This method only considers the frequency of the association mode, and does not consider the importance of the association mode in the transaction database (ie, the weight of the association mode)

The second type is the calculation method of association mode support with fixed item weights. This method uses the product of the sum of item set weights and unweighted association mode support as the weighted item set support (C.H.Cai, A.da, W.C.Fu, etal.Mining Association Rules with Weighted Items[C] / / Proceedings of IEEEInternational database Engineering and Application Symposiums,1998:68-77.), this method overcomes the defects of the first type of method and considers the item weight, but the item weight The value is fixed during the mining process, which cannot solve the situation that the item weight changes with different transaction records

Images