Method and system for automatic synonym discovery based on topic model

Abstract

The invention discloses a method for automatically discovering synonyms based on a subject model, which at least comprises the following steps: importing data of synonyms to be discovered; word segmentation being performed on the imported data according to the information of the database; constructing theme model and clustering theme model; performing minimum correlation clustering for topic clustering; outputing synonym. The invention does not need prior knowledge and manual labeling, realizes automatic clustering of synonyms, and improves the efficiency of synonym discovery. To a certain extent, the problem of semantic similarity is solved, and manual intervention is not needed in the implementation process except the final screening, which greatly improves the efficiency of synonym automatic discovery.

Description

technical field [0001] The invention relates to the technical field of natural language processing, in particular to a topic model-based automatic synonym discovery method and system thereof. Background technique [0002] With the development of the information age, the scale of network text data is getting bigger and bigger, so the processing of natural language has gradually become more and more important. The importance of discovery technology is also increasingly reflected. The existing mainstream automatic synonym discovery algorithm needs prior knowledge to construct the reference text pattern for synonym discovery, which limits the efficiency of synonym discovery; while another reference text pattern matching method requires manual analysis of the part-of-speech and semantics of the known vocabulary in advance. Annotate, build reference text schema. [0003] refer to figure 1 It can be seen that the discovery of synonyms in the existing systems needs to be suppleme...

AI Technical Summary

Problems solved by technology

The existing mainstream automatic synonym discovery algorithm needs prior knowledge to construct the reference text pattern for synonym discovery, which limits the efficiency of synonym discovery; while another reference text pattern matching method requires manual analysis of the part-of-speech and semantics of the known vocabulary in advance. Annotate, build reference text schema

[0003] refer to figure 1 It can be seen that the discovery of synonyms in the existing system needs to be supplemented by manual screening. Since the method of automatic discovery of synonyms has a certain error rate, the current methods of finding synonyms have the problem of low efficiency.

[0004] At present, in the application with the patent application number CN201410156107.5, a synonym determination, search method and server are claimed, but according to the understanding of the technical solution in the application document, the solution given in the comparison document cannot improve the efficiency of synonym discovery efficiency

Images