Cross-media retrieval method based on uniform sparse representation

Abstract

The invention relates to a cross-media retrieval method based on uniform sparse representation. The method comprises the following steps that a cross-media database comprising a plurality of media types is established and a feature vector of each kind of media data is extracted; a feature mapping matrix for cross-media uniform sparse representation is studied for each media type, the incidence relations between all the different media types are considered, spatial sparsity of the cross-media data after mapping is conducted is also considered, and the feature spatial sample distribution after mapping and the feature spatial sample distribution before mapping are kept approximate; the probability that every two media data belong to the same type serves as the similarity between different media; the similarity between media data in query samples and a query target set is calculated, and the most similar cross-media retrieval result comprising all the media types is output. According to the cross-media retrieval method based on uniform sparse representation, the incidence relations, the sparsity and the sample distribution between the cross-media data are fully considered, the effectiveness of uniform feature representation can be improved, and therefore the accuracy of cross-media retrieval is improved.

Description

technical field [0001] The invention relates to the technical field of multimedia retrieval, in particular to a cross-media retrieval method based on unified sparse representation. Background technique [0002] With the advent of the era of big data, multimedia data on the Internet has grown rapidly, including text, images, video, audio and other media data. However, existing search engines such as Google and Baidu still rely on keyword-based retrieval. On the one hand, this retrieval method ignores the information of multimedia data such as images, videos, and audios. On the other hand, when there is no text around the multimedia data , the search cannot be performed. Although some research work focuses on content-based single-media retrieval, such as image search, etc., it cannot support content-based cross-media retrieval, such as using an image sample to retrieve all relevant media data, including not only related Images, but also text, video, audio, 3D, etc. This ret...

AI Technical Summary

Problems solved by technology

However, this method can only consider the relationship between the two media types during the learning process, and does not consider the use of sparsity to reduce noise in cross-media data, and the two learning steps of the method are carried out independently, without It is possible to consider association information and semantic category information at the same time, so the error accumulation propagates to the subsequent feature learning stage, which reduces the accuracy of the final cross-media retrieval

Images