A sentiment classification method and system based on multimodal contextual semantic features

Abstract

The invention discloses an emotion classification method and system based on multimodal context semantic features. The method includes: dividing a short video into the same number of semantic units by taking utterances as a unit, generating corresponding video, speech and text samples, and extracting three representational features of expression features, spectrograms and sentence vectors; The representation features are respectively input into the expression, speech, and text emotion feature encoders, and the corresponding emotional semantic features are extracted; the contextual relationship between the expression, speech, and text emotion semantic features is used to construct the corresponding adjacency matrix; The semantic features and the corresponding adjacency matrix are input to the corresponding graph convolutional neural network, the corresponding contextual emotional semantic features are extracted, and the multimodal emotional features are obtained by fusion, which is used for the classification and recognition of emotions. The present invention makes better use of the contextual relationship of emotional semantic features through the graph convolutional neural network, and can effectively improve the accuracy of emotional classification.

Description

technical field [0001] The invention belongs to the field of emotion computing, and in particular relates to an emotion classification method and system based on multimodal context semantic features. Background technique [0002] In people's daily communication, emotion is an important bridge for mutual understanding between people. The perception and understanding of emotion can help people understand each other's various behaviors and psychological states. Facial expressions and speech are important ways for people to express their emotions. The emotional research on these single modalities has become increasingly mature and has been applied in people's lives. However, with the deepening of the research, the researchers found that since the emotional information expressed by a single modality is incomplete, there are certain limitations in single-modal sentiment analysis. Therefore, more and more researchers turn their attention to emotion classification research based on...

AI Technical Summary

Problems solved by technology

Existing methods try to learn hidden associations between multiple modalities at different stages, or make emotional predictions based on the information of different modalities before performing vote fusion, which solves related problems to a certain extent and improves multiple modalities. Modal sentiment classification performance, but most of them ignore the context dependence in each modality information, and do not consider the context information of each utterance in the video, there are still places to be improved

Images