Cross-modal subject correlation modeling method based on deep learning

Abstract

The invention belongs to the technical field of cross-media correlation learning, and particularly relates to a cross-modal subject correction modeling method based on deep learning.The method includes two main algorithms of multi-modal file expression based on deep vocabularies and correlation subject model modeling fusing cross-modal subjection correction learning.A deep learning technology is utilized for constructing deep semantic vocabularies and deep vision vocabularies to describe a semantic description part and an image part in a multi-modal file.Based on multi-modal file expression, a cross-modal correlation subject model is constructed to model a whole multi-modal file set, so that the generation process of the multi-modal file and the correlation between different modals are described.The accuracy is high, and adaptability is high.The cross-modal subject correction modeling method has important meaning for efficient cross-media information retrieval in consideration of multi-modal semantic information on the basis of the large-scale multi-modal file (a text and an image), can improve retrieval correlation and promote user experience, and has great application value in the field of cross-media information retrieval.

Description

technical field [0001] The invention belongs to the technical field of cross-media correlation learning, and in particular relates to a cross-modal image-text topic correlation learning method based on deep learning. Background technique [0002] With the development of Internet technology and the maturity of Web2.0, a large number of multimodal documents have been accumulated on the Internet. How to analyze and process the complex structure of these multimodal documents to provide theoretical support for practical applications such as cross-media retrieval has changed. become a very important research hotspot. Generally speaking, a multimodal document usually exists in the form of multiple modal co-occurrences. For example, many web images are accompanied by many user-defined image descriptions or annotations, and some web documents contain some illustrations. . However, although these multi-modal data are often related to each other, due to the problem of semantic gap, t...

AI Technical Summary

Problems solved by technology

However, such a direct matching method does not well consider the heterogeneity of images and texts, so learning their correlations by mapping deep topic features to a common space can well mine their correlations to solve the above question raised

Images