K-means text clustering method and device with built-in constraint rules

Abstract

Embodiments of the invention disclose a k-means text clustering method and device with built-in constraint rules. The method comprises the following steps of utilizing a first constraint rule to preprocess a to-be-clustered text set to obtain a first preprocessing set corresponding to the first constraint rule; acquiring k texts in the to-be-clustered text set as cluster centers, wherein k is smaller than N, and N is the total number of the texts in the to-be-clustered text; if each cluster center is contained in the first preprocessing set, adding the rest of the texts except the cluster center in the first preprocessing set to a class cluster corresponding to the cluster center, and removing the texts added to the class cluster in the to-be-clustered text set; respectively adding the remaining texts in the to-be-clustered text set to the class clusters corresponding to the cluster centers closest to the texts; and computing a new cluster center of each class cluster again, and if thenew cluster center meets a preset stopping condition, outputting all class clusters. By the adoption of the clustering method, the computation complexity of text clustering can be reduced, and meanwhile, the clustering precision can be also improved.

Description

technical field [0001] The invention relates to the technical fields of information processing and text mining, in particular to a k-means text clustering method and device with built-in constraint rules. Background technique [0002] Cluster analysis is one of the important tools in the field of data mining, and its basic work is to cluster data. Clustering is a process of classifying data into different clusters. The data in the same cluster have great similarity, while the data between different clusters have great dissimilarity. Commonly used clustering algorithms include hierarchical clustering, grid clustering, partition clustering, etc. Among them, the k-means clustering algorithm in partition clustering is relatively more commonly used and easy to implement. [0003] A typical k-means clustering algorithm includes the following steps: [0004] 1. Randomly select k texts from a data set including N texts as cluster centers, k<N; [0005] 2. For each text except ...

AI Technical Summary

Problems solved by technology

[0008] After analysis, the inventor believes that for each iteration of the typical k-means clustering algorithm, all the data in the data set need to participate in a calculation to calculate the distance from each text except the cluster center to each cluster center, especially when the data When the amount of text in the set is large and / or the number of selected cluster centers is large, the computational complexity is high

In addition, the typical k-means clustering method extracts features based on text content to calculate the distance between texts. The clustering effect is not ideal, and it is easy to group texts with similar features but different topics / irrelevant texts, or Texts with dissimilar features but same / related topics are classified into wrong clusters

Images