Emotion recognition method and device based on LSTM audio and video fusion and storage medium

Abstract

The invention discloses an emotion recognition method and device based on LSTM (Long Short Term Memory) audio and video fusion and a storage medium. An LSTM model is adopted, a more detailed frame-level feature is used for training the model, and the obtained emotion recognition is accurate. Meanwhile, a method of combining decision fusion with later fusion is adopted, the recognition results of the two modes can be more effectively fused for the features of speech emotion recognition and the features of facial expression recognition, and a more accurate emotion recognition result is obtainedthrough calculation. According to the method provided by the invention, the emotional state of the prediction object can be obtained more accurately, and the accuracy and robustness of emotion recognition are improved.

Description

technical field [0001] The present application relates to the technical field of speech recognition, in particular to an emotion recognition method, system, device and storage medium based on LSTM audio-video fusion. Background technique [0002] Emotion is important information in people's communication process, which is usually expressed in facial expressions, speech, text, body movements, etc. With the rapid development of information technology, people's demand for smart devices is becoming more and more vigorous, and intelligent technologies such as human-computer interaction are becoming more and more important. Emotion recognition technology has a wide range of applications in human-computer interaction, car and aircraft driving, and medical care. applications and prospects. [0003] The modalities of emotional expression include facial expressions, speech, text, physiological signals, postures, etc. The current mainstream affective computing methods are mainly divi...

AI Technical Summary

Problems solved by technology

[0005] Zhao Xiaoming and Zhang Shiqing proposed a robust speech emotion recognition method based on compressed sensing, providing a robust speech emotion recognition method in the background of noise; fully considering the effectiveness of different types of feature parameters, the extraction of feature parameters From the two aspects of prosodic features and sound quality features, the Mel frequency cepstral coefficient MFCC is extended to further improve the anti-noise effect of the feature parameters, but the emotional recognition when the acoustic channel cannot obtain signals has not yet been solved.

Images