Chinese text sorting method based on correlation study between sorts

Abstract

The invention discloses a Chinese text sorting method based on correlation study between sorts. The method comprises the following steps of: firstly, dividing words of a document and performing rough selection on characteristics by computing word frequencies; secondly, further determining representative word items according to discrimination indexes between the word items and sorts so as to form characteristic word items which are finely selected; thirdly, training the document to be expressed by a tfidf weight and a discrimination index weight according to an index of the characteristic word items; fourthly, establishing a group of two-sort sorters corresponding to different projection vectors and training to obtain a code array expressing the correlation between two-sort sorters; and finally, projecting a multi-vector expression of a new document to all the two-sort sorters, introducing the code array, computing the similarity between each sort and the document, and outputting the maximum of the similarity as a sort judging result of the new document. The new document is sorted based on a correlation studying result between the sorts, and the running efficiency of an algorithm is improved on the premise of ensuring the sorting performance.

Description

technical field [0001] The invention belongs to the research field of Chinese text classification algorithms, and in particular relates to a Chinese text classification method which adopts the discrimination index between words and categories to select features and learns based on the correlation between categories. Background technique [0002] With the rapid development of China's publishing industry, the number of Chinese documents in electronic format continues to rise. The work of document classification is becoming more and more cumbersome, so it is necessary to use advanced machine learning and pattern classification methods to assist traditional manual classification. [0003] The Chinese text classification method mainly consists of two parts: feature selection and classification algorithm. The characteristics of the document set are generally expressed in the form of Bag-of-Words model (Bag-of-Words) and document vector model (Vector Space Model). The probability...

AI Technical Summary

Problems solved by technology

[0008] All the above algorithms need to use methods such as SVM to train and construct classifiers. The algorithm runs a lot and takes a long time to run. There are many limitations in practical applications.

Images