A multi-modal non-contact sentiment analysis recording system

Abstract

The invention discloses a non-contact emotion analysis and recording system based on multimodality, which is characterized in that the composition includes: a sound receiving module for receiving sound from the external environment, and sound feature extraction for obtaining audio emotion labeling information of voice and processing module, a speech recognition module for converting speech content to text content, a text feature extraction and processing module for obtaining speech text emotion annotation information, and a comprehensive scheduling for completing all data processing, storage, and scheduling tasks module, a display module for completing the display of the detected voice emotional state, and a clock module for completing the functions of time recording and providing time labels. The invention can recognize voice emotion by combining two modalities of text and audio, thereby improving the accuracy of recognition.

Description

technical field [0001] The invention relates to the field of human-computer emotion interaction, in particular to a multimodal non-contact emotion analysis and recording system. Background technique [0002] Language is the most important tool for communication between people. Human speech includes text symbol information and also contains people's emotions. Artificial processing of emotional information features from speech is of great significance in the field of artificial intelligence. Humans communicate through language, and human emotions are expressed through multiple channels and modes, such as expressing emotions through language content, audio, facial expressions, and body movements. Speech emotion recognition is to identify the speaker's emotional information from voice signals. [0003] The current research mainly extracts emotional feature information from speech prosody. The speech emotion recognition system mainly relies on the low-level acoustic features of s...

AI Technical Summary

Problems solved by technology

[0003] The current research mainly extracts emotional feature information from speech prosody. The speech emotion recognition system mainly relies on the low-level acoustic features of speech for recognition. The representative features are pitch frequency, formant, short-term average zero-crossing rate and pronunciation duration. Time, etc., this method tends to lead to a higher feature dimension. Pattern recognition research shows that the accuracy rate is not proportional to the dimension of the feature space, and the generalization ability will be weakened in high-dimensional situations, and even lead to a higher dimensionality. disaster

[0004] There is also a linguistics approach that considers the emotional analysis of speech signals, considers the semantic components of the speech text, and uses the semantics and grammar of sentences to provide emotional clues to the speaker. History, word frequency, etc.; the disadvantage of this method is that it requires a lot of knowledge, which first brings difficulties to speech recognition, and semantic analysis requires relevant language knowledge, which increases the difficulty of sentiment analysis. The method is complex and difficult to implement at this stage

[0005] In the field of speech emotion information processing, almost all pattern recognition methods are used, such as artificial neural network (ANN), hidden Markov model (HMM), mixed Gaussian model (GMM), support vector machine (SVM), etc., but if the Comparing all these results together, it can be found that the means of feature extraction are extremely limited. Almost all studies use prosodic features or the linear combination and transformation of these prosodic features as the research object, and most of them are only in the audio mode. Analysis, so that the speech emotional features are always limited to a smaller category, not comprehensive enough

Images