K-means text clustering method and device with built-in constraint rules

Abstract

The embodiment of the present application discloses a k-means text clustering method and device with built-in constraint rules, the method includes: using the first constraint rule to preprocess the text set to be clustered to obtain the first preprocessing corresponding to the first constraint rule Set; obtain k texts in the text set to be clustered as cluster centers, k<N, N is the total number of texts in the text set to be clustered; if the cluster centers are included in the first preprocessing set, then the first preprocessing The rest of the text in the processing set except the cluster center is added to the cluster corresponding to the cluster center, and the text that has been added to the cluster in the text set to be clustered is cleared; the remaining text in the text set to be clustered is added to the corresponding cluster. In the cluster corresponding to the nearest cluster center; recalculate the new cluster center of each cluster, and output all the clusters if the new cluster center meets the preset stop condition. Using the above clustering method can reduce the computational complexity of text clustering, and at the same time improve the clustering accuracy.

Description

technical field [0001] The invention relates to the technical field of information processing and text mining, in particular to a k-means text clustering method and device with built-in constraint rules. Background technique [0002] Cluster analysis is one of the important tools in the field of data mining, and its basic work is to cluster data. Clustering is a process of classifying data into different clusters. Data in the same cluster have great similarity, while data between different clusters have great dissimilarity. Commonly used clustering algorithms include hierarchical clustering, grid clustering, partition clustering, etc. Among them, the k-means clustering algorithm in partition clustering is relatively more commonly used and easy to implement. [0003] A typical k-means clustering algorithm includes the following steps: [0004] 1. Randomly select k texts from a data set including N texts as cluster centers, k<N; [0005] 2. For each text except the clust...

AI Technical Summary

Problems solved by technology

[0008] After analysis, the inventor believes that for each iteration of the typical k-means clustering algorithm, all the data in the data set need to participate in a calculation to calculate the distance from each text except the cluster center to each cluster center, especially when the data When the amount of text in the set is large and / or the number of selected cluster centers is large, the computational complexity is high

In addition, the typical k-means clustering method extracts features based on text content to calculate the distance between texts. The clustering effect is not ideal, and it is easy to group texts with similar features but different topics / irrelevant texts, or Texts with dissimilar features but same / related topics are classified into wrong clusters

Images