Document classification method based on variance

Abstract

The invention discloses a document classification method based on variances. The document classification method based on the variances comprises the steps of (a) gathering enough classification-specific scientific and technical literatures, novels and essays as training samples; (b) performing word segmentation through an existing word segmentation method and calculating word frequencies; (c) performing normalization processing on the word frequencies; (d) calculating the variance of the word frequencies in each document; (e) concluding intervals of the variances of the word frequencies; (f) processing the documents to be classified; (g) judging drop-in intervals of the variances; (h) acquiring document classifications according to the drop-in intervals. By means of the document classification method based on the variances, the documents to be classified can be classified automatically, reasonably and scientifically according to different variances of the word frequencies caused by wording features of scientific and technical literatures, novels and essays, and high efficiency is achieved while the classification accuracy rate is ensured. Therefore, the theoretical foundation is laid for classifying existing literatures into scientific and technical literatures, novels and essays, and a complete, scientific theory method is provided.

Description

technical field [0001] The present invention relates to a document classification method based on variance, and more specifically, to a document classification method based on variance to distinguish scientific and technological documents, novels and prose according to the difference in word frequency variance caused by the characteristics of words used in scientific literature, novels and prose . Background technique [0002] With the development and progress of Internet technology, the document resources in the network are constantly enriched, including literary works such as novels and essays that enrich people's spiritual life, and scientific and technological documents that provide people with knowledge and lay the foundation for scientific research. The crystallization of wisdom and technology is the precious wealth of human civilization. However, with the advent of the era of big data, the exponential growth of massive resources poses challenges for the effective org...

AI Technical Summary

Problems solved by technology

However, without the dependence of the topic, these classification algorithms cannot macroscopically establish an effective classification model for documents without topic restrictions before the domain is determined.

The above problems exist in the distinction between literary works and scientific and technological documents, that is, it is impossible to effectively classify whether a document belongs to a scientific and technological document, a novel or an essay

Because the fields and themes involved in literary works and scientific literature are infinite, and when the fields and themes involved in different types of documents are still intersecting, it is impossible to provide accurate category definitions during the training phase, and it is also impossible to extract features that can represent categories. value set, so it is difficult to build a classification model for this classification problem

Images