Online cross-modal retrieval method and system using three-step strategy

Abstract

The invention provides an online cross-modal retrieval method and system using a three-step strategy. The online cross-modal retrieval method comprises the following steps: acquiring analog stream data formed by different modals; for analog stream data, generating the representation of each class label by introducing a hadamard matrix, the representation of each class label is used as global information for learning a hash code, and the representation of each class label also keeps local similarity information, using the correlation between newly arrived data in the analog stream data and existing data to learn a Hash code with more discriminative power; updating a hash function by using the learned hash code; and calculating a hash code of the to-be-retrieved sample by using the updated hash function, and calculating a Hamming distance of a binary sample based on the hash code, thereby returning a sample of another modal similar to the to-be-retrieved sample according to the Hamming distance. According to the method, the THOR can reserve more semantic information and learn more accurate hash codes.

Description

technical field [0001] The invention belongs to the technical field of cross-modal hash retrieval, and in particular relates to an online cross-modal retrieval method and system using a three-step strategy. Background technique [0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art. [0003] With the explosive growth of data composed of many different modalities, it has become a major challenge to find similar information across modalities from massive data under the condition of a given query statement. In many cases, traditional nearest neighbor search methods do not lead to optimal performance due to high complexity in terms of time and storage space. In recent years, Approximate Nearest Neighbor Search (ANN), especially hash learning, has attracted extensive attention and gradually replaced traditional nearest neighbor search methods. Cross-modal hashing methods aim to m...

AI Technical Summary

Problems solved by technology

Most existing cross-modal online methods only update hash functions based on newly arrived data or the correlation between new data and existing data, ignoring the global information

[0006] 2) How to enhance the adaptability of the model to adapt to the variable-length label space is a problem that needs to be solved. So far, most existing online methods implicitly assume that the label space is fixed, that is, all class labels are Should be given in the first data block

In practice, this assumption may be too restrictive, and these methods may not be effective in updating the hash function when new labels appear in newly arrived data blocks.

Images