Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Mandarin lip language recognition method based on deep learning

A technology of deep learning and recognition methods, applied in the field of computer vision and deep learning, can solve the problem of low accuracy of audio recognition

Active Publication Date: 2019-03-26
TIANJIN UNIV
View PDF6 Cites 18 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In addition, lip language recognition technology can also solve the problem of low audio recognition accuracy in noisy environments

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Mandarin lip language recognition method based on deep learning
  • Mandarin lip language recognition method based on deep learning
  • Mandarin lip language recognition method based on deep learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0029] In order to further understand the invention content, characteristics and effects of the present invention, the following examples are given, and detailed descriptions are as follows in conjunction with the accompanying drawings:

[0030] as attached figure 1 As shown, a deep learning-based lip recognition method for Mandarin Chinese, considering the characteristics of the Chinese language structure, adopts an encoder-decoder algorithm architecture. In order to have versatility and scalability, a convolutional neural network is used to extract video Features, the sub-units of the encoder and decoder use a recurrent volume neural network, and the Mandarin Chinese label uses word embedding encoding. For the accuracy of lip language algorithm recognition, an attention mechanism is added to the output part of the encoder-decoder. Include the following steps:

[0031] Step 1. Create a lip-reading dataset based on the original data:

[0032] Step 1-1, each frame of the raw ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a mandarin lip language recognition method based on deep learning. The characteristics of the Chinese language structure are considered, an algorithm framework of an encoder and a decoder is utilized, a convolution neural network is used for extracting video features for universality and expansibility, sub-units of the encoder and the decoder adopt a circular convolution neural network, mandarin labels adopt a word embedding coding mode, and an attention mechanism is added to an encoder-decoder output part for accuracy of lip language algorithm identification. Accordingto the method, mandarin sentence-level lip language recognition is recognized as a research object, restriction factors influencing lip language recognition are analyzed, a targeted solution is addedto a built lip language recognition model, a lip language recognition technology which can be practically applied is obtained, and a theoretical and technical basis is provided for the lip language solution with higher accuracy and higher expansibility.

Description

technical field [0001] The invention relates to the fields of computer vision and deep learning, in particular to a method for recognizing lips in Mandarin Chinese based on deep learning. Background technique [0002] Lip language recognition is a technology that translates and understands speaker information through the visual features of lips, face and tongue movements without the help of speech information. It also relies on the information provided by the context, language knowledge . Lip language is also known as visual language, or pattern recognition based on lip movement when speaking. [0003] Traditional lip language recognition methods mostly use artificial extraction of low-level or high-level features of lips, and then send the obtained features to feature classifiers, such as SVM, Adaboost, etc., for word-level classification. Although such a method has a small amount of calculation, it is subject to the very skillful process of manual feature extraction, and...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G10L15/25G06K9/00G06N3/04G06F17/27
CPCG10L15/25G06F40/289G06V40/161G06N3/044G06N3/045
Inventor 赵美蓉吴大江邢广鑫郑叶龙
Owner TIANJIN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com