Cross-modal retrieval method and system based on unmarked data migration

Abstract

The invention provides a cross-modal retrieval method and system based on unmarked data migration. The clustered single-mode image and text data without annotation information are used as a migrationsource domain, a cross-mode data set with annotation information is used as a target domain, the source domain is migrated to the cross-mode data set of the target domain through migration learning, the training data scale is expanded, the semantic information of the cross-mode data is increased, and a better common space is learned. According to the method and system, the problem that the cross-modal data set is small in data scale is well solved, and the condition that actual user query is not within a predefined category range is better met; and meanwhile, the upper-layer semantic information of different modal data can be better extracted, the isomerism difference between modals is overcome, the similarity between the modals is increased, and the cross-modal retrieval accuracy is improved.

Description

technical field [0001] The invention relates to the technical field of cross-modal data retrieval, in particular to a cross-modal retrieval method and system based on unmarked data migration. Background technique [0002] Data of different modalities, such as images and texts, widely exist on the Internet and show a trend of mutual integration. The cross-modal retrieval task attempts to break the boundaries between different modal data, and achieve information retrieval across different modal data, that is, try to use a certain modal sample to retrieve samples of other modals with similar semantics. It is widely used in data management. Existing cross-modal retrieval methods try to map the feature representations of different modal data into a common space to learn a unified representation, and measure the similarity by calculating the distance between their corresponding unified representations. However, due to the heterogeneity of different modal data, data distribution ...

AI Technical Summary

Problems solved by technology

[0005] In order to overcome the problems of data heterogeneity in different modalities, excessive unlabeled data, insufficient training data and non-extensibility, the present invention proposes a cross-modal retrieval method and system based on unlabeled data migration. The single-modal image and text data with labeled information are used as the migration source domain, and the cross-modal data set with labeled information is used as the target domain. Through transfer learning, the source domain is migrated to the cross-modal data set of the target domain to expand the training data scale. Increase semantic information across modal data to learn a better common space

Images