Topic model based document keyword extraction method and system

Abstract

The invention discloses a topic model based document keyword extraction method and system. The document keyword extraction method comprises the following steps of document information preprocessing, document structure graph construction, document topic distribution extraction, word weight extraction and keyword generation. The document keyword extraction system comprises the following modules: a document information preprocessing module, a document structure graph construction module, a document topic distribution extraction module, a word weight extraction module and a keyword generation module. According to the method and system, extracted keywords are more reasonable and related to a topic of a document more closely; and partial deficiencies in the keyword extraction field at present are overcome, a better document summarization effect is achieved, and a user can conveniently and quickly know an abstract of the document.

Description

technical field [0001] The invention relates to a data mining technology, in particular to a method and system for extracting document keywords based on a topic model. Background technique [0002] Keywords are a summary of the main content of the document, and it is an important way to quickly understand the subject of the document. Keywords can be seen in various places. For example, we can see the tags of each news article on news websites, and we can see the keywords discussed in the paper when we browse scientific and technological papers. It reduces the difficulty for people to search for information in massive amounts of information. Current keywords have been applied in various fields. In the field of information retrieval, keywords are widely used. Search engine companies such as Baidu and Google search based on keywords in webpage text, and the results based on keywords in documents are often what users want. In the field of social networks, many current functio...

AI Technical Summary

Problems solved by technology

The method of labeling by users spontaneously is applicable to few scenarios. Users only label specific objects of interest. Currently, it is impossible to provide an effective model to encourage users to label other content

Due to the current rapid development of information technology, the amount of information on the Internet is also increasing explosively, and new content is generated all the time. It is too expensive to ask experts to manually mark relevant documents, and the marked documents can only be used for research, and it is difficult to for commercial use

Images