Sentiment classification method and system based on multi-modal context semantic features

Abstract

The invention discloses a sentiment classification method and system based on multi-modal context semantic features. The method comprises the following steps: segmenting a short video into semantic units with the same number by taking an utterance as a unit, generating corresponding video, voice and text samples, and extracting three characterization features, namely an expression feature, a spectrogram and a sentence vector; respectively inputting the three extracted characterization features into expression, voice and text emotion feature encoders, and extracting corresponding emotion semantic features; constructing corresponding adjacent matrixes by using context relationships of emotion semantic features of expressions, voices and texts; and inputting the expression emotion semantic features, the voice emotion semantic features, the text emotion semantic features and the corresponding adjacent matrixes into corresponding graph convolutional neural networks, extracting corresponding context emotion semantic features, and fusing the context emotion semantic features to obtain multi-modal emotion features for emotion classification and recognition. According to the method, the context relationship of the emotion semantic features is better utilized through the graph convolutional neural network, and the accuracy of emotion classification can be effectively improved.

Description

technical field [0001] The invention belongs to the field of emotion computing, and in particular relates to an emotion classification method and system based on multi-modal context semantic features. Background technique [0002] In people's daily communication, emotion is an important bridge for mutual understanding between people. The perception and understanding of emotion can help people understand each other's various behaviors and psychological states. Facial expressions and speech are important ways for people to express their emotions. The research on these single-modal emotions has become increasingly mature and has been applied in people's lives. However, with the deepening of research, researchers found that because the emotional information expressed by a single modality is incomplete, there are certain limitations in single-modal sentiment analysis. Therefore, more and more researchers are turning their attention to the research of emotion classification based...

AI Technical Summary

Problems solved by technology

Existing methods try to learn hidden associations between multiple modalities at different stages, or make emotional predictions based on the information of different modalities before performing vote fusion, which solves related problems to a certain extent and improves multiple modalities. Modal sentiment classification performance, but most of them ignore the context dependence in each modality information, and do not consider the context information of each utterance in the video, there are still places to be improved

Images