Virtual learning environment micro-expression recognition and interaction method based on double-flow convolutional neural network

Abstract

The invention relates to a virtual learning environment micro-expression recognition and interaction method based on a double-flow convolutional neural network, and the method comprises the followingsteps: S1, carrying out the preprocessing of micro-expression data: carrying out the Euler video amplification of a micro-expression video, extracting an image sequence, carrying out the face positioning of the image sequence, and carrying out the cutting of the image sequence, and obtaining the RGB data of a micro-expression; extracting optical flow information from the data amplified by the Euler video to obtain an optical flow image of the micro-expression; s2, dividing the preprocessed data into a training set and a test set, and constructing a double-flow convolutional neural network by using a transfer learning method so as to learn space and time domain information of micro expressions; s3, carrying out maximum value fusion on the output of the double-flow convolutional neural network to enhance the recognition accuracy and obtain a final micro-expression recognition model; and S4, creating a virtual learning environment interaction system by using the micro-expression recognition model, and obtaining a user face image sequence through Kinect to carry out a micro-expression recognition task.

Description

technical field [0001] The invention belongs to the field of artificial intelligence and relates to a micro-expression recognition and interaction method in a virtual learning environment based on a double-stream convolutional neural network. Background technique [0002] Emotion plays a very important role in daily life. It can reflect a person's emotional state and can be expressed through facial expressions, voice, body language, etc. Among them, facial expression is the most important way of emotional expression, and it can also assist other expressions, so it has received extensive attention. However, in many cases, facial expressions are often easily masked or suppressed, resulting in short-duration micro-expressions with small motion ranges. This is a fast facial expression with a duration of only 0.5s and small and asymmetric movements. The recognition of micro-expressions can effectively help people capture the true emotions of human beings more accurately. Howeve...

AI Technical Summary

Problems solved by technology

However, due to the small range and short duration of micro-expressions, the conventional network structure cannot learn the temporal features of micro-expressions well.

Images