Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Overall emotion recognition method combining image and speech

An emotion recognition and speech technology, applied in speech analysis, character and pattern recognition, acquisition/recognition of facial features, etc., can solve the problems of large amount of emotion recognition calculation, emotion recognition error, difficult to improve accuracy, etc., and achieve reliable emotion classification. High performance, robust overall performance, and strong adaptability

Pending Publication Date: 2017-10-17
NANJING UNIV OF POSTS & TELECOMM
View PDF6 Cites 26 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0005] At present, the existing human emotion recognition is only a single face image emotion recognition or a single voice emotion recognition. Using image emotion recognition under the background of dim light and blurred images, or using speech recognition under the background of environmental noise will make human Emotion recognition has a large error, and even misjudged it as other emotion categories
A single aspect of human emotion recognition technology has encountered a bottleneck, and the accuracy rate is difficult to improve
In addition, in human-computer interaction, the existing emotion recognition calculation is still huge, resulting in delay, which will cause data errors, which is very unfriendly in human-computer interaction

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Overall emotion recognition method combining image and speech
  • Overall emotion recognition method combining image and speech

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0041] The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

[0042] A comprehensive emotion recognition system that combines images and voices to study the reasonable configuration of the two main methods of human emotion recognition (voice emotion recognition and face image emotion recognition) to increase the timeliness and accuracy of human emotion recognition. Firstly, speech emotion samples and expression emotion samples are collected from Berlin emotional speech database and facial expression recognition image library. Then, the weak classifier PNN algorithm and LDC method are used to train the speech samples, and the weak classifier HMM model and Gabor transformation are used to train the expression samples. By setting the confidence level and integrated learning, the posterior probability of reliable speech and expression emotion classification results is obtained. And use this as the ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present invention discloses an overall emotion recognition method and system combining an image and speech. The process of recognition comprises: after acquiring a corresponding speech and video signal from an input video, an information acquisition apparatus transmits the corresponding speech and video signal to corresponding emotion classification modules respectively, and after classification, an integrated learning trainer allocates weights, and after weighting, a recognition result is output to complete a recognition process. The system comprises an information acquisition apparatus, an emotion classifier and an integrated processor. The information acquisition apparatus comprises a video acquisition device and an audio acquisition device; the emotion classifier comprises an expression emotion classification module for performing emotion classification on acquired video information and a speech emotion classification module for performing emotion classification on acquired audio information; and the integrated processor comprises a weighting module and an integrated learning trainer. The method and system provided by the present invention have the advantages of higher emotion classification reliability, flexible adjustment on confidence parameters and high precision; and through bi-directional recognition of expression and speech, the human emotion recognition process is simulated to a large extent.

Description

technical field [0001] The invention belongs to the interdisciplinary technical fields of computer technology, information technology and data mining, and relates to a comprehensive emotion recognition method and system combining image and voice. In human-computer interaction, two aspects of the face image and voice of the same person are mainly used Emotion recognition with weighted assignment. Background technique [0002] Facial emotion recognition refers to the use of computers to extract and analyze human facial expression information, classify and understand according to the current public understanding and way of thinking, and integrate the prior knowledge of human emotional information to make the computer independently associate, think and reason. Finally, analyze human emotions from face information. Since facial expression recognition has broad application prospects, it has gradually become one of the research hotspots in the fields of human-computer interaction,...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G06K9/62G10L25/63
CPCG10L25/63G06V40/175G06V40/172G06F18/2411G06F18/254G06F18/2415
Inventor 殷越铭樊小萌胡海峰
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products