A Cross-modal Retrieval Method Based on Recurrent Generative Adversarial Networks

Abstract

The invention discloses a cross-modal retrieval method based on a cyclic generative confrontation network. The method designs a novel dual-channel cyclic generative confrontational neural network, and establishes the semantics of cross-modal data by training the neural network. Correlation. Given that different modal data can flow bidirectionally in the network, each modal data generates another modal data through a set of generative adversarial networks, and the generated data is used as the input of the next set of generative adversarial networks, so as to realize the bidirectional flow of data Cyclic generation, the network continuously learns the semantic relationship between cross-modal data. In order to improve the retrieval efficiency, this method also uses the threshold function and approximation function to approximate the result of the middle layer of the generator to the corresponding binary hash code, and designs a variety of constraints to ensure the similarity and The difference of cross-modal and inter-class data further improves the accuracy and stability of retrieval.

Description

technical field [0001] The invention belongs to the technical field of multimedia information retrieval, and in particular relates to a cross-modal retrieval method based on a cyclic generative confrontation network. [0002] technical background [0003] With the advent of the Internet era, people can access massive amounts of information in various modes including pictures, videos, texts, and audios anytime and anywhere. How to obtain the content they need from these massive amounts of information has become the focus of Internet users. Often rely on the precise search services provided by search engines such as Google, Baidu, and Bing. However, most of the traditional Internet retrieval services are still at the level of single-modal retrieval, and there are few retrieval applications for cross-modal data. The efficiency, accuracy, and stability of retrieval need to be improved, and most of them rely on existing The data label of the data cannot achieve cross-modal retrie...

AI Technical Summary

Problems solved by technology

However, most of the traditional Internet retrieval services are still at the level of single-modal retrieval, and there are few retrieval applications for cross-modal data. The efficiency, accuracy, and stability of retrieval need to be improved, and most of them rely on existing data labels, it is impossible to achieve cross-modal retrieval of unlabeled data

Images