Text normalizing method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a text regularization method and device, electronic equipment and a storage medium. The method comprises the steps that a to-be-regularized text is determined;inputting the to-be-structured text into the text structured model to obtain a structured text corresponding to the to-be-structured text output by the text structured model; wherein the text regularization model is obtained by training based on a to-be-regularized sample text, a regularized sample text and a sample editing type of each segmented word in the to-be-regularized sample text; whereinthe text regularization model is used for determining an editing type of each segmented word in the to-be-regularized text, determining a regularization mode of the to-be-regularized text based on whether the to-be-regularized text contains insertion segmented words of which the editing types are insertion types or not, and regularizing the to-be-regularized text based on the regularization mode.According to the method, the device, the electronic equipment and the storage medium provided by the embodiment of the invention, the text normalization accuracy and the text normalization efficiencyare improved.

Description

technical field [0001] The present invention relates to the technical field of natural language processing, in particular to a text regularization method, device, electronic equipment and storage medium. Background technique [0002] Text regularization refers to deleting substantive words in the original text or repetitive descriptions with similar semantics, words irrelevant to the topic or colloquial words in the original text while keeping the semantics of the text basically unchanged, as well as adjusting the word order in the original text to make it regular The latter text is more written, neat and concise. [0003] Current text regularization methods are usually based on an encoder-decoder model (Encoder-Decoder) to convert text into regular text. However, the above encoder-decoder model has the problem of difficult model learning during the training process, resulting in poor regularization effect, and the model has a large amount of calculation and low efficiency....

AI Technical Summary

Problems solved by technology

[0004] Embodiments of the present invention provide a text regularization method, device, electronic equipment, and storage medium to solve the defects of poor accuracy and low efficiency of text regularization methods in the prior art

Images