Text keyword automatic extraction method and device based on co-occurrence language network

Abstract

The invention discloses a text keyword automatic extraction method and device based on a co-occurrence language network. The defect that a large amount of manual annotation data is needed in supervised machine learning is overcome, the text keyword automatic extraction method and device overcome the defect of weak generalization ability of a language analysis method, avoid the problem that a statistical method is easy to ignore low-frequency and very important keywords. The method comprises the steps of preprocessing a webpage, constructing a language network graph model, jointly extracting candidate keyword features, comprehensively sorting the candidate keyword features and outputting the keywords. According to the method, web text preprocessing, co-occurrence language network model construction, keyword feature joint extraction and candidate keyword sorting optimization are carried out, so that the extracted keywords have good readability, coherence and correlation, and can be widely applied to the fields of natural language processing, information retrieval, text mining, sentiment analysis, multi-mode human-computer interaction and the like.

Description

technical field [0001] The invention relates to the field of keyword extraction, in particular to a text keyword automatic extraction method and medium based on a co-occurrence language network. Background technique [0002] With the rapid development of network technology and the advent of the era of big data, a large amount of network text data has been generated in cyberspace. In the analysis of text big data, keyword extraction is a basic work and has important practical significance. Keyword is the smallest semantic unit. Keyword extraction refers to extracting words or phrases related to the subject of the text from a single or multiple texts, also known as keyword tagging. In the early days, because full-text search was not supported, in order to be able to use keywords to search papers, authors were required to manually set keywords in papers. The traditional way of manually labeling keywords has been unable to effectively deal with today's big text data. Therefore,...

AI Technical Summary

Problems solved by technology

[0011] This method effectively solves the shortcomings of supervised machine learning that require a large amount of manually labeled data, overcomes the weak generalization ability of language analysis methods, and avoids the problem that statistical methods tend to ignore low-frequency but very important keywords. The method of presenting the language network model realizes the automatic extraction of network text keywords without relying on dictionaries and training samples

Images