Dynamic expression recognition method and system based on attention mechanism between space-time streams

Abstract

The invention discloses a dynamic expression recognition method and system based on an attention mechanism between space-time streams. The method comprises the steps of firstly collecting facial expression video clips, and establishing a facial expression video library containing expression category labels; then constructing a double-flow convolutional neural network model embedded with an attention mechanism module between space-time flows, wherein the model comprises a data processing layer, a space flow branch, a time flow branch, the attention mechanism module between the space-time flows, a feature fusion layer, a full connection layer and a classification layer; training the model by using video samples in a facial expression video library; and finally, performing facial expression recognition on a newly input video by using the trained model. According to the method, the attention mechanism module between the space-time streams is embedded in the double-stream convolutional neural network, so that the information interaction between the space domain features and the time domain features can be realized, the dynamic association information between the space domain features and the time domain features is captured, the features with high discrimination capability are obtained, and the accuracy and robustness of facial expression recognition are improved.

Description

technical field [0001] The invention belongs to the field of machine learning and pattern recognition, and relates to a dynamic expression recognition method and system, in particular to a dynamic expression recognition method and system based on a space-time flow attention mechanism. Background technique [0002] With the rapid development of computer technology and artificial intelligence, the way of human-computer interaction is constantly changing, which makes people more and more inclined to communicate directly with computers. In the process of human communication, it is necessary to understand the emotional state of the other party, and in the process of human emotional exposure, facial expressions account for about 55% of the information. Therefore, recognizing facial expressions by computer has become a very hot topic. [0003] Facial expression recognition is a process of extracting facial expression features from images or videos, and judging the expression categ...

AI Technical Summary

Problems solved by technology

The problem with this method is that it can only recognize a static face image, which ignores the time domain information of facial expressions, and it is difficult to achieve the best recognition effect

The problem with this method is that using the DMF module to extract the spatial features of facial expressions and using the LTCNN module to extract the temporal features of facial expressions are two independent processes, and there is not enough information interaction, which may affect the final recognition effect.

Images