Method and system for mining fully weighted positive and negative association patterns between words in text

Abstract

The invention discloses a method of item-all-weighted positive or negative association model mining between text terms and a mining system applied to the method. The method comprises the following steps of preprocessing by using a Chinese text preprocessing module to establish a text database and a feature word item library; mining item-all-weighted feature word candidate item sets from the text database by utilizing a feature word frequent item set and negative item set mining implementation module, calculating a weight dimension ratio, and cutting out uninteresting item sets by adopting a multi-interestingness threshold value pruning strategy to obtain an interesting item-all-weighted feature work frequent item set and negative item set model; mining an effective item-all-weighted positive or negative association rule model from frequent item sets and negative item sets by utilizing an item-all-weighted positive or negative association rule mining implementation module between terms, and outputting the mined positive or negative association rule model to a user by utilizing an item-all-weighted association model result display module between terms. By applying the method and the system, unnecessary frequent item sets, negative item sets and association rule models can be greatly reduced, Chinese feature word association rule mining efficiency is improved and a high-quality association model between Chinese terms is obtained.

Description

technical field [0001] The invention belongs to the field of data mining, specifically a method for mining fully weighted positive and negative association patterns between text words based on the ratio of weight dimensions and a mining system thereof, which is applicable to the discovery of association patterns of characteristic words in text mining and the expansion of text information retrieval queries, etc. field. Background technique [0002] In the past 20 years, the research on association pattern mining technology has made remarkable achievements, and has gone through three research stages: item unweighted mining technology, item weighted mining technology and item fully weighted mining technology. [0003] Phase 1: Research on Mining Unweighted Positive and Negative Association Patterns [0004] The main feature of item unweighted positive and negative association pattern mining is that the probability of the item set appearing in the database is the support degree...

AI Technical Summary

Problems solved by technology

The disadvantage of the traditional item unweighted association mode mining is that it does not consider the existence of item weights, which often leads to a large number of redundant, uninteresting and invalid association modes during mining.

The defect of weighted positive and negative association rule mining is that although it pays attention to the different importance between items, it ignores the fact that the item weight has different weights in each transaction record in the database.

The traditional item unweighted mining method mines these fully weighted data, because it does not consider the inherent characteristics of fully weighted data, but only considers the frequency of items, often resulting in a large number of redundant, invalid and false association models. The item weighted mining method cannot be applied to fully weighted data mining

Images