Cross-modal retrieval method based on deep correlation network

Abstract

The invention discloses a cross-modal retrieval method based on a deep correlation network. The method comprises the following steps that a primary vector of first modal data and a primary vector of second modal data are obtained through an initial feature extraction method; secondly, a deep correlation network model is established and trained, and high-level representation vectors corresponding to a retrieval target and retrieval library members are obtained through the deep correlation network model; thirdly, similarity matching is conducted on the retrieval target and each retrieval memberin a retrieval library through the high-level representation vectors, that is, the Euclidean distance is calculated; fourthly, calculation results of the Euclidean distance are successfully distributed from small to large, and a cross-modal retrieval result list of the retrieval target is obtained. According to the method, multiple layers of corresponding incidence relations are built between dataof different modals, and meanwhile various neural networks are fused, so that a deep model has a better representation effect; the precision of the cross-modal retrieval is higher, and the stabilityis better.

Description

technical field [0001] The invention relates to a multimedia data retrieval technology, in particular to a cross-modal retrieval method based on a deep association network. Background technique [0002] In the era of explosive growth of multimedia information, people are more inclined to search for diversified results, not just for single-modal retrieval. For example, if you see a landscape painting, if you submit this picture to the retrieval system, the retrieval system will not only retrieve similar landscape paintings, but also retrieve information such as audio or text related to this picture at the same time, which will improve the retrieval effect. will be more influential. This process of using data from one modality to retrieve data from another modality is called cross-modal retrieval. [0003] Traditional cross-modal retrieval, such as text retrieval of images, is often based on matching the text annotation information of the image with the retrieved text, so it...

AI Technical Summary

Problems solved by technology

Although the second type of method has been proved to be more suitable for cross-modal retrieval tasks in practice; however, the existing second type of algorithms still have unstable retrieval effects, or the types of components that make up the deep network are too single, resulting in low retrieval accuracy The problem

Images