Keyword Corpus Labeling Training Extraction System

Abstract

The invention discloses a keyword corpus labeling training extraction tool, which aims to provide a labeling training tool that can reduce the complexity of manual labeling process and can improve the labeling efficiency and accuracy of massive keyword corpus. The present invention is realized through the following technical solutions: the keyword corpus labeling preparation module distinguishes massive corpus data from different sources, the semi-automatic corpus keyword labeling module creates a keyword labeling task, independently selects an adaptation algorithm and carries out algorithm model-based Automatic labeling, by integrating at least one keyword extraction algorithm from CHI, LDA, TEXTRANK, and TFIDF, pre-labeling the text corpus data to be labeled, and merging the labeling results of various algorithms. When the labeling task is completed, the feedback key The keyword tagging model learning and training module trains the keyword tagging algorithm model; the keyword tagging model effect evaluation module automatically evaluates the quantitative tagging effect of the model indicators.

Description

technical field [0001] The invention relates to the technical field of text mining, in particular to a semi-automatic labeling training and extraction system for keyword corpus. Background technique [0002] In the field of natural language processing, the key to processing massive text files is to extract the issues that users care about most. Whether it is long text or short text, it is often possible to spy on the theme of the entire text through a few keywords. At the same time, whether it is text-based recommendation or text-based search, there is also a great dependence on text keywords, and the accuracy of keyword extraction is directly related to the final effect of the recommendation system or search system. Therefore, keyword extraction is an important part in the field of text mining. The rapid development of the Internet has provided people with easy access to information, and the number of electronic documents such as web pages, emails, and e-books is increasi...

AI Technical Summary

Problems solved by technology

Both unsupervised and supervised methods have their own advantages and disadvantages: the unsupervised method does not need to manually label the training set, so it is faster, but because it cannot comprehensively use multiple information to rank candidate words, the effect may not be as good as the supervised method; The supervised method can adjust the influence of various information on judging keywords through training and learning, so the effect is better. However, in today's data age, labeling training sets is very time-consuming and labor-intensive.

The disadvantage of the supervised text keyword extraction algorithm is that it requires high labor costs

The third category is to achieve the effect of recognizing words by letting the computer simulate the human understanding of the sentence. Due to the complexity of Chinese semantics, it is difficult to organize various language information into a form that can be recognized by the machine. Due to the need to mark a large number of training corpora, the use of Manual methods are time-consuming and labor-intensive, and this word segmentation system is still in the experimental stage

At present, the keyword corpus in the field is relatively scarce, and the keyword corpus labeling work is currently mainly done through manual labeling. There are widespread problems such as poor quality of corpus labeling, cumbersome labeling process, low labeling efficiency, and high human resource costs.

At the same time, the existing keyword corpus tagging system has disadvantages such as a single tagging method, and it is difficult to automatically update the tagging method model. Therefore, there is an urgent need for a semi-automatic keyword tagging and training platform that can assist manual tagging of corpus to solve the above problems

Images