Cross-modal hash retrieval method and system based on supervised semantic coupling consistency

Abstract

The invention discloses a cross-modal hash retrieval method and system based on supervised semantic coupling consistency, and the method comprises the steps: extracting the features of modal sample data in each modal database, and constructing a sample library set; solving a hash code of each modal sample in the sample library set; extracting features of the to-be-retrieved modal data, and solvinghash codes of the to-be-retrieved modal data according to the features; and comparing the hash code of the to-be-retrieved modal data with the hash code of each modal sample, wherein the modal sampleof which the comparison result meets the preset condition is used as the retrieval result of the to-be-retrieved modal data. According to the method, the high-level semantic relationship is considered, and the inline coupling between the modals is also considered, so that the modals not only can supervise and learn the hash codes of the modals, but also can keep the semantic consistency between the modals. According to the method, the discrimination strength and compactness robustness of the hash code can be effectively improved, the retrieval rate between modes is increased, and the accuracyof cross-modal retrieval is improved.

Description

Technical field [0001] The invention belongs to the technical fields of e-commerce, multimedia intelligence and data mining, and particularly relates to a cross-modal hash retrieval method and system based on supervised semantic coupling consistency. Background technique [0002] With the rapid development of network intelligent media technology, information retrieval also presents a diversified form, and the demand for diversified retrieval not only involves content retrieval in the traditional single-modal context, but also involves the interrelated content among multiple modalities. Retrieval, through multi-modal information retrieval, the data and information contained in the retrieval content can be displayed more comprehensively. For example, the retrieved text information can be used to generate various modal data information such as image information, video information, and voice information corresponding to semantic relevance. The information relevance between different...

AI Technical Summary

Problems solved by technology

For example: in the paper "RasiwasiaN, Pereira J C, Coviello E, et al.A New Approach to Cross-Modal Multimedia Retrieval." A correlation analysis CCA method is proposed, which maximizes the measurement of the characteristic information of each modal data by constructing a projection matrix. similarity; in the paper "Putthividhya D, Attias H T, Nagarajan S S. Topicregression multi-modal Latent Dirichlet Allocation for image annotation." By referring to the Dirichlet model, a cross-modal retrieval method based on the topic regression model was constructed. Learning the potential subject information of each modality independently for different modalities, and combining the regression model to establish the potential subject relationship between each modality can better describe the semantic relevance between different modalities, but such methods usually The main body distribution of the mode requires a strong assumption, so it has certain limitations in the actual application process

In view of the fact that the types of different modalities are inconsistent and distributed in various spaces, the current mainstream method is to learn the common space in the middle of the characteristics of different modal types, and measure the similarity of each modal feature in the common space. Cross-modal retrieval; however, most algorithms ignore the semantic relevance of modal feature representations, that is, they do not fully consider the semantic information between modalities and the inline information of each modal itself, resulting in the identification of modal feature categories. The accuracy is not strong, reducing the accuracy and robustness of the query

Images