Keyword extraction method and device based on information entropy, equipment and medium

Abstract

The invention relates to the field of artificial intelligence, and provides an information entropy-based keyword extraction method, apparatus and device, and a medium, which can preprocess a received tag text, reduce the redundancy of characters while standardizing the text, improve the speed and effect of data processing, and adopt an inverse information entropy vector to improve the accuracy of keyword extraction. On one hand, the effect of the TFIDF can be copied under the condition that the data quality is high, on the other hand, noise-doped data can be effectively processed, keywords with the category distinguishing capacity are mined, the problem that the category distinguishing capacity of a traditional TFIDF fails is effectively solved, then misjudgment is avoided, and the method has high interpretability and is suitable for large-scale popularization and application. And extracting a target keyword according to the word weight matrix to realize automatic and accurate keyword extraction. In addition, the invention also relates to a blockchain technology, and the target keyword can be stored in the blockchain node.

Description

technical field [0001] The present invention relates to the technical field of artificial intelligence, in particular to a keyword extraction method, device, equipment and medium based on information entropy. Background technique [0002] In the field of artificial intelligence, keyword extraction plays an important role. For example, in the process of enterprise employee training, more and more links have gradually shifted from offline to online. Taking the professional training of life insurance agents as an example, if the training is to be practical and effective, so that agents have the opportunity to apply what they have learned to the actual life insurance sales, customer retention and other scenarios, the training and production departments must differentiate between the crowd and the training. In-depth research on chemistry. The traditional method mainly revolves around conducting regular communication with the business department to collect the appeals and points ...

AI Technical Summary

Problems solved by technology

[0003] However, because the traditional keyword extraction technology only considers the number of documents in which a certain word appears, it is very easy to fail in a scene with a similar text language environment, and it is difficult to play the role of discovering differentiated keywords. Words only appear once in other categories of people, and because the word appears in every document, the word will be treated as a "common word", thereby reducing the weight of the word in the vocabulary of the category to which it belongs, resulting in recognition error

Images