Chinese text error detection method and system based on word order and semantic conjoint analysis

Abstract

The invention discloses a Chinese text error detection method and system based on word order and semantic conjoint analysis. In order to solve the problem that an existing Chinese text error detection method cannot deeply understand Chinese text semantics and cannot automatically distribute weights, a Chinese text error prediction model which regards a text as a one-dimensional picture and uses a bidirectional recurrent neural network to fit the text and a self-attention mechanism to distribute weights is designed. A semantic understanding module (FR) composed of a full convolutional neural network (FCN) and a residual network (ResNet) is adopted, and the method has the following two advantages that firstly, the full convolutional neural network (FCN) is used for regarding one-dimensional text data as a one-dimensional picture, text semantics are understood, and the problem that semantic processing means are lacked in the prior art is solved; and secondly, a residual network (ResNet) is used to deepen the number of layers of the network, improve the number of features and deepen the understanding degree of text semantics.

Description

technical field [0001] The invention belongs to the fields of Chinese text processing, text cleaning and text error detection, and relates to a Chinese text error detection method and system based on joint analysis of word order and semantics. Background technique [0002] With the development of science and technology, the popularization of 4G and 5G, the informationization of the whole society is increasing day by day. Online office and remote office are no longer a fantasy, and the paperless era has come. With the advent of paperless, more and more information is stored in storage devices in the form of electronic information. Because of the particularity of the text, only slight differences may bring about completely different meanings. It may be that one word increases and the meaning of the whole sentence becomes different. These problems have brought huge troubles and losses to people. Like official documents, academic papers, legal documents, and case documents, th...

AI Technical Summary

Problems solved by technology

However, the current mainstream technology is difficult to dig out the semantic problems of words, so it cannot perform error detection well.

Moreover, the relationship between different words is different, and different weights need to be assigned to represent their relevance. The existing methods are not ideal for the distribution of weights.

Images