Text error correction method and device, equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a text error correction method and device, equipment and a storage medium, and the method comprises the steps: obtaining a to-be-corrected text sequence; inputting the to-be-corrected text sequence into the Bert model, identifying wrongly written characters in the to-be-corrected text sequence to obtain a wrongly written character set, and correcting the wrongly written character set based on a preset candidate character set to obtain a corrected target text sequence; sequentially extracting first target characters after error correction from the target text sequence, obtaining second target characters before error correction corresponding to the first target characters, and calculating font similarity and character similarity between the first target characters and the second target characters; and calculating an error correction judgment factor according to the font similarity and the character similarity, comparing the error correction judgment factor with a preset threshold value, and determining an error correction result of the text sequence to be subjected to error correction according to a comparison result. By means of the mode, the problem that class overfitting is caused due to the fact that a Bert model is too rough in word similarity judgment can be solved.

Description

technical field [0001] The invention relates to the technical field of natural language of artificial intelligence, in particular to a text error correction method, device, equipment and storage medium. Background technique [0002] Text error correction is a technology in the field of natural language processing to detect whether there are typos in a text and to correct the typos. Text error correction is generally used in the text preprocessing stage, and it is also widely used in the problem of inaccurate speech recognition. At present, the common problems of text error correction in the industry are as follows: ①more correction, ②less correction, and ③miscorrection. The causes of these problems include glyph splitting, distance calculation, etc. In terms of language statistical models, the language statistical models including ngram model and Bert model are widely used at present. Among them, the judgment result of Bert model for word similarity is 0, 1. This judgment m...

AI Technical Summary

Problems solved by technology

[0003] The present invention provides a text error correction method, device, equipment and storage medium, which can solve the problem that the Bert model is too rough for word similarity judgment and leads to class overfitting

Images