Method for realizing text feature extraction based on improved small-world network model

Abstract

The invention discloses a method for realizing text feature extraction based on an improved small-world network model. According to the method, a semantic relevancy function is determined according to a Chinese word segmentation preprocessing process and determined vocabulary position weights and word class weights in combination with a (HowNet) two-vocabulary relevancy algorithm and a vocabulary-to-text importance method, wherein the function is subjected to normalization processing, and calculation conditions of values are more standard; and two parameters, namely a density parameter and a weight parameter are set for a lexical semantic network model graph, the two parameters are effectively fused, and an appropriate threshold value is set to extract text feature vocabularies. The method has higher accuracy and overcomes the defect that a traditional method is only suitable for extracting text features of one category; the method has higher application value, contribution degrees of different vocabularies to text thought can be precisely calculated, data processing is more standard, the result error rate is lowered, the constructed lexical semantic network model graph better conforms to the actual condition, and meanwhile a good theoretical basis is provided for subsequent text clustering.

Description

technical field [0001] The invention relates to the technical field of semantic network, in particular to a text feature extraction method based on an improved small-world network model. Background technique [0002] At present, the traditional keyword extraction algorithm only pays attention to the surface statistical characteristics of the document (such as word frequency, word position, word length, etc.), and ignores the semantic information and structural information of the document, resulting in the lack of keyword semantic and structural information. However, the existing keyword extraction algorithms based on the word network use the structural information of the document to a certain extent, but the utilization of semantic information is still insufficient, and the construction process of the network depends too much on the granularity of word segmentation. The information gain method is only suitable for extracting text features of one category, but cannot be used ...

AI Technical Summary

Problems solved by technology

However, the commonly used text feature extraction methods do not consider the semantic status of the feature vocabulary and its contribution to the expression of the text, and more importantly, the processing of data normalization is lacking.

Images