Cross-modal retrieval method and device, network training method and device, equipment and medium

Abstract

The invention relates to a cross-modal retrieval method and device, a network training method and device, equipment and a medium. The cross-modal retrieval method comprises the steps of obtaining to-be-retrieved data and candidate data; the data to be retrieved and the candidate data correspond to different modes; extracting a first feature of the to-be-retrieved data and a second feature of the candidate data based on a cross-modal retrieval network; and retrieving data matched with the to-be-retrieved data from the candidate data according to the matching degree of the first feature and the second feature. By adopting the cross-modal retrieval network disclosed by the invention, the input local information of the to-be-retrieved data and the candidate data can be accurately captured, more effective features are output, and the more effective features are more distinctive in the same mode and are more identifiable in different modes, so that the fine-grained cross-modal retrieval performance is improved.

Description

technical field [0001] The present disclosure relates to the field of computers, and in particular, to a cross-modal retrieval method, a network training method, an apparatus, a device and a medium. Background technique [0002] Cross-modal retrieval refers to a retrieval method in which the modalities of the retrieval results and the modalities of the query data are different. For example, use images to retrieve text, video, audio, etc. [0003] In the related art, cross-modal retrieval usually performs similarity calculation on the features of different modalities output by the retrieval network, obtains a correlation score, and performs retrieval according to the correlation score. When training the retrieval network, the two modal data are usually mapped into the high-dimensional representation space of the same dimension, and then the features of the two modalities are obtained and then the comparison loss function is directly used for training. The features output by...

AI Technical Summary

Problems solved by technology

[0004] The present disclosure provides a cross-modal retrieval method, network training method, device, equipment, and medium to at least solve the problem that the features output by the retrieval network in the related art are not fine enough, and only coarse-grained judgments about whether the features are related are available. The problem of poor retrieval effect of fine-grained cross-modal retrieval

Images