GIF short video emotion recognition method and system fusing text information

Abstract

The invention discloses a GIF short video emotion recognition method and a GIF short video emotion recognition system fusing text information. The method firstly extracts the sequence features in theGIF short video by using a 3D convolutional neural network, and simultaneously extracts the image visual features in the sequence by using the convolutional neural network. Then the high-level semantic features are decoded by using convolution long-short memory recurrent neural network technology, and the probability distribution matrix of emotion classification is calculated, and the emotion scores of the video part are obtained by interval mapping. Then, the vocabulary which contains emotional information is selected from the words in the annotated text, and the emotional score is calculatedby the emotional scoring tool. Finally, the video emotion score and the text emotion score are given different weights to add and do the validity judgment, and the GIF short video emotion classification. The invention can effectively pay attention to the emotional information of images in the GIF video, simultaneously taking into account the temporal characteristics of the video stream, and fusesthe text information and the video information, thereby improving the accuracy and robustness of the emotional classification of the GIF video.

Description

technical field [0001] The invention relates to a video emotion recognition method and system, in particular to a GIF short video emotion recognition method and system fused with text information, belonging to the technical field of computer vision video emotion recognition. Background technique [0002] Emotion recognition in video sequences is a research topic involving computer vision, pattern recognition and artificial intelligence. With the rapid development of Internet technology and the maturity of social media, more and more people use short videos to express their opinions and emotions. However, due to the existence of semantic gap and the difficulty of learning emotional information from video sequences in short video emotion recognition, it is very challenging to propose a method that can not only learn video sequence information but also accurately understand emotional semantics. [0003] Traditional emotion recognition work mainly focuses on text content that c...

AI Technical Summary

Problems solved by technology

However, the CNN used in the current relevant literature cannot describe the information in the time domain of GIF short videos. At the same time, the early fusion of video and text will increase the complexity of the model, and the fusion of video information and text information at the feature level cannot be judged. Importance of two kinds of information in final sentiment classification

Images