The invention discloses a personality detection method based on multi-
modal alignment and multi-vector representation. The personality detection method comprises the following steps:
resampling voiceand video
modal data according to each epoch; inputting the plurality of samples and the text
modal data thereof into an intra-modal representation module for independent coding to obtain a voice sequence, a
video sequence and a text sequence; inputting the voice sequence, the
video sequence and the text sequence into an inter-modal alignment characterization module, and splicing after pairwise alignment and interaction to obtain enhanced voice characterization, video characterization and text characterization; respectively splicing all the voice representations, all the video representationsand all the text representations to obtain a voice vector, a video vector and a text vector, and inputting the voice vector, the video vector and the text vector into a
convolutional neural network tobe converted into at least two types of personality vectors; and respectively linearizing the at least two types of personality vectors, and mapping through a
sigmoid function to obtain prediction probabilities of the at least two types of personality characteristics.
Modal representation is enhanced through pairwise interaction of the three
modal data, the discrimination capability of the modelis improved, and a more accurate prediction result is obtained.