Decision-making layer fusion method for multi-modal sentiment classification

Abstract

The invention discloses a decision-making layer fusion method for multi-modal emotion classification. The method comprises the following steps: dividing samples in a multi-modal emotion data set into a training set and a test set; respectively constructing sentiment classification models of various modalities, and respectively training the sentiment classification models of various modalities by using samples of corresponding modalities in the training set; using the trained sentiment classification models of various modes to perform sentiment classification on samples of corresponding modes in the test set, and counting classification results to obtain sentiment classification confusion matrixes of various modes; performing sentiment classification on the corresponding modes of the tested sample by using the trained sentiment classification models of various modes; and performing decision-making layer fusion on sentiment classification results of various modes of the tested sample by using the classification confusion matrix to obtain a sentiment category of the tested sample. According to the method, the prior knowledge of information differences of different modes and the complementarity between the modes are fully utilized, and the accuracy and robustness of multi-mode sentiment classification can be effectively improved.

Description

technical field [0001] The invention relates to the technical field of pattern recognition and emotion computing, in particular to a decision-making layer fusion method for multi-modal emotion classification. Background technique [0002] Social media is a huge source of opinions on various products and user services. When a user comments on a product on the Internet, he will inadvertently show his satisfaction with the product, that is, the emotion expressed during the review process. For merchants, it is hoped to understand the attitude of the user group to the product as quickly as possible, so as to adjust the marketing strategy or improve the quality of the product in time, so as to improve the user's satisfaction with the product. [0003] Most of the previous research on emotion classification focused on identifying human emotional states through single-modal information, such as emotion classification based on speech, emotion classification based on facial expression...

AI Technical Summary

Problems solved by technology

How to determine the weights of the emotional classification results of different modalities to reflect the differences of different modalities in emotional classification is still an open subject facing challenges

Images