Multi-modal emotion recognition method and system fusing attention mechanism and DMCCA

Abstract

The invention discloses a multi-modal emotion recognition method and system fusing an attention mechanism and DMCCA. The method comprises the following steps of: respectively extracting electroencephalogram signal features, peripheral physiological signal features and expression features from preprocessed electroencephalogram signals, peripheral physiological signals and facial expression videos; extracting electroencephalogram emotion features, peripheral physiological emotion features and expression emotion features with discriminability by using the attention mechanism; using the DMCCA method for the electroencephalogram emotion features, the peripheral physiological emotion features and the expression emotion features to obtain electroencephalogram-peripheral physiological-expression multi-modal emotion features; and classifying and identifying the multi-modal emotion features by using a classifier. According to the method, the attention mechanism is adopted to selectively focus on the features with higher emotion discriminability in each mode, and the DMCCA is used to fully utilize the correlation and complementarity between the emotion features of different modes, so that the accuracy and robustness of emotion recognition can be effectively improved.

Description

technical field [0001] The invention relates to the technical fields of emotion recognition and artificial intelligence, in particular to a multimodal emotion recognition method and system that integrates attention mechanism and discriminative multiple set canonical correlation analysis (DMCCA). Background technique [0002] Human emotion is a psychological and physiological state that accompanies the process of human consciousness and plays an important role in interpersonal communication. With the continuous advancement of technologies such as artificial intelligence, people are paying more and more attention to obtaining more intelligent and humanized human-computer interaction (Human–Computer Interactions, HCIs) experience. People's requirements for machine intelligence are getting higher and higher, and they expect machines to have the ability to perceive, understand and even express emotions, so as to realize humanized human-computer interaction and better serve human ...

AI Technical Summary

Problems solved by technology

[0005] Purpose of the invention: for the shortcomings of low single-modal emotion recognition accuracy, poor robustness and existing multi-modal emotional feature fusion methods, the purpose of the invention is to provide a fusion attention mechanism and identify multiple sets of canonical correlation analysis ( DMCCA) multi-modal emotion recognition method and system, by introducing an attention mechanism to selectively focus on the discriminating emotional features in each modality, and combined with DMCCA to make full use of the correlation and complementarity between different modal emotional features can effectively improve the accuracy and robustness of multi-modal emotion recognition

Images