Theme identification method, system and equipment based on theme co-occurrence network and external knowledge

Abstract

The invention discloses a theme identification method based on a theme co-occurrence network and external knowledge, and the method specifically comprises the steps: constructing a theme co-occurrence network based on annotation data: detecting theme sub-words in an existing domain knowledge text with theme annotation, and constructing the theme co-occurrence network according to the theme sub-words; constructing a switch module fusing external knowledge and a topic co-occurrence network: performing information richness sorting on the domain knowledge text with the topic annotation by using the co-occurrence network, and combining the sorting with the external knowledge to form the switch module; improving self-training of a domain knowledge text theme recognition model by introducing a switch module: training the domain knowledge text theme recognition model by using a self-training method, using information of the domain knowledge text without theme annotation as far as possible, and preventing generalization performance reduction caused by non-selective learning of the domain knowledge text without theme annotation by self-training; limited corpus information is fully and efficiently utilized, and the performance of a domain knowledge text theme recognition model is improved.

Description

technical field [0001] The invention belongs to the technical field of text recognition, and in particular relates to a topic recognition method, system and equipment based on a topic co-occurrence network and external knowledge. Background technique [0002] The explosive growth of the total amount of knowledge in the Internet age has made the problem of knowledge fragmentation increasingly prominent. Due to the lack of a unified form of knowledge organization and management, users usually need to spend more time looking for valuable domain knowledge texts in the massive fragmented knowledge for learning. , the efficiency of information acquisition is low. Classifying and storing domain knowledge texts according to the knowledge topics they describe can improve the efficiency of retrieval and learning, and at the same time provide technical support for downstream applications related to other domain knowledge. [0003] Domain knowledge texts usually contain a lot of domain...

AI Technical Summary

Problems solved by technology

[0007] 3. A control method and device for extracting Chinese entity relations based on word co-occurrence, patent number: CN201110001355.9; patent 1 counts the number of subject keywords contained in each text, and finds the subject keywords that contain the most as the text Subject: Patent 2 achieves semi-supervised classification prediction by finding category centers and random sampling methods to reduce data redundancy. Neither Patent 1 nor Patent 2 further mines information such as data co-occurrence networks.

The word co-occurrence relationship used in patent 3 is whole word matching, without considering the subword characteristics in domain knowledge

Images