Semi-supervised Chinese and English implicit discourse relation identification method and system

Abstract

The invention provides a semi-supervised Chinese and English implicit discourse relation identification method and system. The method comprises the following steps: step 1, preparing a data set; step 2, model construction: on the basis of the implicit discourse relation identification model based on the attention mechanism, respectively constructing a Chinese implicit discourse relation identification model and an English implicit discourse relation identification model; step 3, performing unsupervised model training; step 4, supervised model training; and 5, outputting a prediction result. According to the semi-supervised Chinese and English implicit chapter relationship recognition method provided by the invention, a Chinese and English implicit chapter relationship recognition model is jointly trained based on a large number of unlabeled chapter relationship data sets and a small number of manually labeled chapter relationship data sets in two languages; the unlabeled data set and the labeled data set in the two languages can be fully and effectively utilized, so that the Chinese and English implicit discourse relation recognition performance is improved at the same time.

Description

technical field [0001] The invention relates to the technical field of computer language processing, in particular to a semi-supervised Chinese-English implicit discourse relationship recognition method and system. Background technique [0002] Implicit discourse relation recognition aims to automatically infer semantic relations between two arguments (sentences or clauses) that lack discourse connectives, e.g., transition and causality. In different languages, there are usually a small number of human-annotated textual relationship datasets, for example, the Chinese CDTB dataset and the English PDTB dataset. Since it is very difficult to manually label chapter relational datasets, the scale of both the CDTB dataset and the PDTB dataset is relatively small. Among them, there are about 5,500 instances of implicit discourse relations marked in the CDTB dataset, while the current largest PDTB dataset only contains about 16,000 instances. Although these manually annotated corp...

AI Technical Summary

Problems solved by technology

[0005] In view of the above situation, it is necessary to solve the lack of a method for identifying implicit discourse relations in the prior art using manually annotated discourse relationship datasets in both Chinese and English, which limits the identification of Chinese and English implicit discourse relations to a certain extent. The problem of performance improvement

Images