Video-based micro-expression recognition method and device, equipment and storage medium

Abstract

The invention relates to the technical field of micro-expression recognition, and discloses a video-based micro-expression recognition method, device and equipment and a storage medium. The method comprises the steps of: obtaining a to-be-recognized video in real time; performing face time sequence extraction on a to-be-recognized video to obtain time sequence fragments; performing image time sequence feature extraction on the time sequence segments through a three-dimensional convolutional neural network to obtain a feature vector matrix group; obtaining a region incidence matrix group trained by a graph neural network model, and fusing the region incidence matrix group and the feature vector matrix group to obtain a to-be-classified vector; and performing micro-expression recognition on the to-be-classified vector to obtain a micro-expression result corresponding to the to-be-recognized video. Therefore, the micro-expression type of a person is automatically recognized from the space dimension and the time dimension through the three-dimensional convolutional neural network and through the area incidence matrix group reflecting the incidence relation between key areas in a face, and the micro-expression recognition accuracy and quality are improved.

Description

technical field [0001] The present invention relates to the technical field of micro-expression recognition, in particular to a video-based micro-expression recognition method, device, equipment and storage medium. Background technique [0002] Micro-expression is a short and unconscious facial expression. Its start and end are not subject to subjective control. Short-term micro-expression can reflect people's inner psychological state unconsciously. If it can be extracted from the customer service industry Micro-expressions, through the recognition of micro-expressions, changes in customer satisfaction can be obtained, and changes in customer satisfaction can be quickly identified, which can help companies improve service quality and customer satisfaction. It can be seen that micro-expression recognition is currently more active. And the forward-looking field is also a cross field combining psychology and machine vision technology. In the existing technology, the recognitio...

AI Technical Summary

Problems solved by technology

[0002] Micro-expression is a short and unconscious facial expression. Its start and end are not subject to subjective control. Short-term micro-expression can reflect people's inner psychological state unconsciously. If it can be extracted from the customer service industry Micro-expressions, through the recognition of micro-expressions, changes in customer satisfaction can be obtained, and changes in customer satisfaction can be quickly identified, which can help companies improve service quality and customer satisfaction. It can be seen that micro-expression recognition is currently more active. And the forward-looking field is also an interdisciplinary field combining psychology and machine vision technology. In the existing technology, the recognition of micro-expressions can be roughly divided into two categories. One is based on the traditional method of facial local area changes. It is the change of local areas in the facial image, such as lips, etc. to make a classification. This type of scheme often does not perform well in large data sets, because the habits of the crowd are quite different, which makes it difficult to take into account a large number of people; the other type It is an image recognition technology based on deep learning and artificial intelligence. This kind of scheme often recognizes the entire image, but the object of this scheme is a static image, and only static features can be extracted for recognition. Therefore, the above two types of schemes exist The accuracy of micro-expression recognition is not high, and it is prone to misjudgment, which will lead to misinterpretation of the customer's mood and easy loss of customers

Images