Music emotion classification method based on multi-modal learning

Abstract

The invention discloses a music emotion classification method based on multi-modal learning, and the method comprises the following steps: data preprocessing: carrying out the preprocessing of the audio, lyrics and comments of music according to the needed modal information, so as to obtain the effective input of a model; representation learning: mapping each modal to a respective representation space by using different modeling modes; feature extraction: extracting feature vectors of different modals after model mapping, and reducing dimensions to the same dimension; multi-modal fusion:, carrying out cascade early fusion on the features of the three different modals, and establishing more comprehensive feature representation; and emotion classification decision making: performing supervised emotion classification on the music by using the fused features. According to the music sentiment classification method, a method based on multi-modal joint learning is provided, the defect that noise or data loss exists in a current mainstream single-modal model method can be effectively reduced, and the accuracy and stability of music sentiment classification are improved.

Description

technical field [0001] The invention relates to the fields of emotion recognition and multimodal learning, in particular to a music emotion classification method based on multimodal learning. Background technique [0002] The emotional features widely used in the field of emotion recognition include sentence-based global statistical features and speech-based temporal features, but these two types of emotional features based on different durations cannot effectively express emotional issues. At present, the most research is to use acoustic features as auxiliary semantic information for speech emotion recognition, extract the emotional information contained in speech and identify its category. Commonly used speech feature extraction methods include: fundamental frequency feature extraction, formant feature extraction, Mel frequency cepstral coefficient (MFCC) extraction, derivative-based impersonal speech emotion feature extraction, and teager energy operator-based nonlinear f...

AI Technical Summary

Problems solved by technology

[0004] With the development of Internet technology and the advancement of data storage technology, music resources have grown geometrically, and the problem of information overload has emerged. However, traditional music labels (such as genres, singers, years, etc.) lack openness and ignore music. Its own characteristics, there is a lot of room for improvement

Images