Chinese lip language recognition method based on deep learning

Abstract

The invention discloses a Chinese lip language recognition method based on deep learning. The method comprises the following steps: S1, acquiring a lip sequence image; S2, extracting features of the lip sequence image from the obtained lip sequence image; S3, inputting the extracted features of the lip sequence image into a bidirectional long-short-term memory network to perform time-space featuresequence learning, training the features of the learned lip sequence image, and training the features of the learned lip sequence image to a lip language recognition model; and S4, decoding and recognizing the extracted features of the lip sequence image according to the trained recognition model from the features of the learned lip sequence image to the lip language, and recognizing a lip language result. The mouth positioning accuracy is high, the calculation method is simplified, and the problem of information decline is better solved by adopting the bidirectional LSTM network.

Description

technical field [0001] The invention belongs to the field of lip language recognition, in particular to a Chinese lip language recognition method based on deep learning. Background technique [0002] With the development of artificial intelligence technology, audio-visual input in complex scenes, and simple spelling text input are a thing of the past. The proportion of speech recognition is gradually increasing, and it is becoming the mainstream natural interaction method. However, pure voice interaction is easily affected by the environment and prone to noise interference, such as noisy outdoor roads, human voice disputes in conference rooms, engine or air-conditioning noise in car scenes, etc., will be greatly reduced There is a clear gap in the accuracy of speech recognition and user experience. In order to improve the problem of inaccurate speech recognition, lip recognition technology has emerged. Lip language recognition technology refers to the scheme of identifying...

AI Technical Summary

Problems solved by technology

Most of the traditional lip language recognition schemes include mouth detection, mouth segmentation, mouth normalization, feature extraction and construction of lip language classifiers. However, the performance of traditional lip language recognition schemes is not satisfactory, and the accuracy of lip language interpretation The rate is only 20%-60%, the accuracy of lip language recognition results is low, and it is difficult to improve

Images