Multi-modal fusion method and device for psychological stress detection

Abstract

The embodiments of the invention provide a multi-modal fusion method and device for psychological stress detection. The multi-modal fusion method is based on an attention-enhanced feature matrix of physiological data->text, physiological data->pictures, text->physiological data, text->pictures, pictures->physiological data and pictures->text, based on a feedforward fully-connected neural network,a fusion feature matrix of the text, the pictures and the physiological data is obtained; then based on importance weights of the text, the pictures, and the physiological data and the fusion featurematrix of the text, the pictures and the physiological data, a fusion representation matrix of three modes is obtained; and finally, based on the fusion representation matrix of the three modes and the feedforward fully-connected network, a pressure classification vector reflecting the psychological stress problem is obtained. By merging text picture data and physiological related data, not only the deficiencies caused by the subjectivity of text and picture data are compensated, but also certain inherent problems of physiological related data are solved.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a multimodal fusion method and device for psychological stress detection. Background technique [0002] With the increasing pressure of social competition, the psychological pressure of teenagers has gradually become a more and more serious problem. Excessive psychological stress can lead to many physical and psychological problems, which makes psychological stress detection more and more important. [0003] Existing psychological stress detection work focused on social media only focuses on text and picture content. However, text and picture content are subjective and sometimes cannot express the true psychological state. [0004] Some existing works related to physiological signals have proved the effectiveness of physiological signals in detecting psychological stress, such as heart rate variability, electrocardiogram, current skin response, electroencephalogram, blood pres...

AI Technical Summary

Problems solved by technology

However, there are also some inherent problems in the data related to physiological signals. For example, the physiological data related to the state of extreme excitement and extreme stress are very similar. Therefore, the data related to physiological signals sometimes cannot fully express the real psychological state.

[0005] According to the above description, it can be seen that there is currently a lack of an effective psychological stress detection method and device.

Images