A Real-time Driver Emotion Recognition Method Fused with Facial Expression and Speech

Abstract

The invention discloses a method for real-time recognition of driver's emotion by combining facial expression and voice. Firstly, the face is tracked in real time through kinect SDK to obtain the driver's facial image and voice signal, and then the driver's facial image, The acoustic signal is preprocessed, and the feature extraction model based on unsupervised feature learning and sparse coding is trained according to the given objective function. After the model is obtained, the preprocessed information is input into the model to obtain emotional features based on facial images and sound signals ; and extract words according to the spoken content, create a dictionary through the frequent words obtained by the Apriori algorithm, and obtain text-based emotional features through the dictionary, and finally connect the emotional features based on facial images and sound signals with the text-based emotional features. Get the feature vector, input the feature vector to the support vector machine SVM, train the SVM classifier, and get the SVM model. The final SVM model is used to identify the driver's emotion, which has high robustness.

Description

technical field [0001] The invention relates to a method for real-time recognition of driver's emotion, in particular to a real-time driver's emotion recognition method which integrates facial expressions and voice. Background technique [0002] In recent years, with the rapid increase of the number of private cars, the number of annual traffic accidents is also rising sharply, causing huge losses to people's lives and property. There are many reasons for traffic accidents, mainly including two factors: active factors and passive factors. Passive factors mainly refer to the abnormality of the car's own parts and some uncontrollable external factors; active factors refer to the driver's fatigue, abnormal behavior, and emotional abnormalities. At present, there are already detection equipment for the abnormality of the car itself, and many scholars have conducted research on driver fatigue driving and abnormal driver behavior, and great progress has been made. None reported....

AI Technical Summary

Problems solved by technology

In order to solve the problem of high-precision and real-time recognition of driver's emotion, the present invention introduces Kinect, a high-speed 3D camera device, to extract RGB image information, Depth image information and voice information, and proposes a set of feasible driver's emotion recognition for these features method, greatly improving the recognition accuracy and speed

Images