Driver road rage identification method based on deep fusion of facial expressions and voice

Abstract

The invention discloses a driver road rage identification method based on deep fusion of facial expressions and voice. The method comprises the following steps: extracting facial image information and voice information of a driver from facial video image information of the driver; preprocessing the facial image frame information and inputting the facial image frame information into a multilayer convolutional neural network to obtain facial expression features; firstly, extracting Mel-frequency cepstral coefficients and first-order and second-order coefficient values of voice information for initial feature extraction, and splicing initial features of two sections of voice segments and inputting the splicing initial features into a full connection layer network to obtain discriminative voice frame features corresponding to facial expression frames; performing low-rank bilinear pooling fusion on the obtained facial frame expression features and the voice frame features to obtain fusion features; and performing decision fusion on the facial expression features, the voice features and the fusion features to obtain a final road rage recognition result. In a complex driving environment, the driver anger identification result can still be output in a high-precision mode, and then safe driving early warning is effectively carried out.

Description

technical field [0001] The invention relates to the field of computer-aided driving, in particular to a driver's road rage recognition method that deeply integrates facial expressions and voice. Background technique [0002] With the progress of society and the improvement of people's living standards, car transportation has become a normal state, and cars will become the main way people choose to travel now and for a long time to come. [0003] In recent years, with the acceleration of people's travel rhythm, time has become an important factor for everyone to consider. As a result, the rapid travel mode of using cars instead of travel has brought a lot of traffic to the current traffic. On the one hand, the construction speed of the existing infrastructure can no longer meet the needs of a large number of driving trips, which often leads to traffic congestion; on the other hand, drivers often violate traffic rules in order to compress travel time, such as Abnormal behavio...

AI Technical Summary

Problems solved by technology

However, this invention requires the help of a third-party platform to identify voice content for auxiliary sentiment analysis, which is relatively complicated to implement

In addition, this method will have a better recognition effect when the quality of visual features and voice data is high, but when the facial image and voice signals are of low quality, such as blurred video, there is more noise in the voice, etc., the performance of emotion recognition will be affected. Accuracy has some influence

Therefore, the road rage intelligent identification method for drivers still needs to be improved

Images