Facial expression recognition method and system, storage medium, computer program and terminal

Abstract

The invention belongs to the technical field of computer vision. The invention discloses a facial expression recognition method and system, a storage medium, a computer program and a terminal, and themethod comprises the steps: pre-training an image generation model according to the combination of a given depth image and an RGB image, and enabling the trained image generation model to convert aninput depth image into an RGB image according to an RGB image style used for training; generating eyebrows, eyes and mouths of expressions in the RGB images, training a convolutional neural network considering the eyebrows, the eyes and the mouths, and achieving expression recognition through the convolutional neural network. According to the invention, the feature information of eyes, eyebrows and mouths is enhanced, and the recognition accuracy is higher; the effect of the image generation model is good, important information about expressions is reserved through the image generation model,and the forms of RGB images used for expression recognition are unified; the accuracy of expression recognition is higher; when only one channel of the depth map is used for identification, the effectobtained by the method is better.

Description

technical field [0001] The invention belongs to the technical field of computer vision, and in particular relates to a facial expression recognition method, system, storage medium, computer program and terminal. Background technique [0002] At present, research on facial expression recognition from two-dimensional RGB color images has made great progress. Putting the RGB expression images collected in the laboratory into the convolutional neural network, deeply mining and extracting the characteristics of the expressions in the image, and training the network to learn the classification of expressions are the current mainstream research directions. But it is worth mentioning that there are still many shortcomings in using this method for expression classification. Two-dimensional RGB expression pictures are a kind of color pictures. Using such pictures to classify expressions is easily affected by factors that have nothing to do with expressions, such as light, angle, and ...

AI Technical Summary

Problems solved by technology

But it is worth mentioning that there are still many shortcomings in using this method for expression classification

Two-dimensional RGB expression pictures are a kind of color pictures. Using such pictures to classify expressions is easily affected by factors that have nothing to do with expressions, such as light, angle, and skin color. The extraction of the same features of the same expression by the neural network is likely to be different due to these factors. different, but these factors are actually irrelevant to a person's expression; this type of method introduces a lot of information that has nothing to do with expression, which greatly affects the accuracy and efficiency of expression recognition, especially in the face of different environments. The problem is even more serious when collecting pictures

[0004] To sum up, at present, the expression recognition method based on two-dimensional RGB color images is difficult to achieve a general expression recognition method for different scenarios. The reason is as mentioned above. When faced with pictures collected in different environments, The accuracy of recognition has been seriously affected

However, when using the scanned 3D face data, due to the lack of texture information, the recognition rate of its expression still needs to be improved; this is what we hope to solve in the design scheme

[0005] Through the above analysis, the existing problems and defects of the existing technology are: how to design a scheme that not only preserves the unaffected geometric information advantages of the depth map in cross-scenes, but also combines the expression texture information of the RGB image, and achieves a higher facial expression recognition accuracy

Images