The invention discloses a multi-
modal data fusion method based on a
discriminant multi-
modal deep confidence network. The multi-
modal data fusion method based on the
discriminant multi-modal deep confidence network comprises the steps that the
discriminant multi-modal deep confidence network is established; for the deep confidence network corresponding to multi-
modal data, the weight of the network after the deep confidence network is optimized is obtained by means of limited Boltzmann machines; objective functions of the multi-modal Boltzmann machines are minimized by means of the alternative optimization strategy, the weights of the optimized Boltzmann machines are obtained, and a final discriminant multi-modal deep confidence
network model is obtained; the multi-
modal data to be fused are input into the deep confidence
network model, and then a fusion result is obtained. The invention further discloses a multi-
modal data fusion system based on the discriminant multi-modal deep confidence network. According to the multi-modal data fusion method and
system based on the discriminant multi-modal deep confidence network, monitored
label information is introduced into a traditional multi-modal deep confidence network, the relations between the data with different modals are mined in a discriminant mode, and thus the high accuracy rate can be guaranteed during a large-scale multi-modal data classifying and searching task.