Cross-modal retrieval method based on adversarial learning and asymmetric hashing

Abstract

The invention discloses a cross-modal retrieval method based on adversarial learning and asymmetric hashing. The cross-modal retrieval method comprises the following steps: 1) selecting a cross-modaldata set and dividing the cross-modal data set into a training set and a test set; training a model, wherein the model comprises a loss module, a feature extraction unit and a Hash code learning unit,wherein the hash code learning unit comprises a hash code learning network and a shared multi-label binary matrix, and the loss module comprises the confrontation loss of the feature extraction unit,the ternary margin loss of the hash code learning unit, the cosine quantization loss and the asymmetric hash loss; 2) generating a binary hash code of each modal data in the cross-modal data set by using the trained optimal model; and 3) for a given query data, firstly generating a binary hash code, and then calculating a Hamming distance between the binary hash code of the query data and the binary hash code of the modal data in the cross-modal data set, which is different from the modal of the query data, so as to obtain a cross-modal data instance meeting the condition.

Description

technical field [0001] The invention relates to a cross-modal retrieval method based on adversarial learning and asymmetric hashing, and belongs to the technical field of computer software. Background technique [0002] With the massive increase of multimedia data in online social media and search engines, there is an increasing demand for mutual retrieval between different modal data (such as video, image, text, etc.). For example, when a user inputs a text query, it needs to obtain picture information or audio and video information related to the query text. In order to solve the retrieval problem in large-scale cross-modal data, an efficient and fast method is the hash method. The cross-modal hash method mainly has two stages, one is the extraction of the characteristics of each modal data, and the other is It is the similarity mining between various modal data. Currently existing cross-modal hashing methods can be roughly divided into two types: traditional cross-modal...

AI Technical Summary

Problems solved by technology

However, the existing cross-modal methods based on deep learning cannot fully extract the characteristics of each modal data in the feature extraction stage, and the hash codes generated in the hash code generation stage are not optimal.

Images