Cross-media sequencing method based on multi-depth network structure

Abstract

The invention relates to a cross-media sequencing method based on a multi-depth network structure. The method comprises the following steps of 1, building a cross-media data set including a plurality of media types, and extracting feature vectors of all media data; 2, training the multi-depth network structure by using the cross-media data set, and using the trained multi-depth network structure for unified expression of study of different media data; 3, using the trained multi-depth network structure to obtain the unified expression of different media data so as to calculate the similarity of different media type data; and 4, taking each datum of each media type to be used as an inquiry sample, retrieving data in another media, calculating the similarity of the inquiry sample and the inquiry sample, performing sequencing according to the sequence from high similarity to low similarity, and obtaining a result sequencing table of target media data. The method provided by the invention has the advantages that various network structures are used in a combined way; associated information between the media and inside the media can realize modeling at the same time; further, the unified expression study is performed by using two stages of networks; and the accuracy rate of the cross-media sequencing is improved.

Description

technical field [0001] The invention relates to the field of multimedia retrieval, in particular to a cross-media sorting method based on a multi-deep network structure. Background technique [0002] In recent years, with the rapid development of Internet and multimedia technology, multimedia data has become the main content of big data, including images, texts, videos, audios, etc. As the total amount of multimedia data continues to grow, how to effectively retrieve this information has become a key issue in the use and management of big data. The commonly used retrieval method is text-based keyword retrieval, that is, the user enters the query text, and the system matches the user query with the text label of the data to obtain the retrieval results. However, this retrieval method requires a large amount of manual annotation of media data. In order to overcome this deficiency, researchers have proposed content-based media retrieval, that is, users upload media data as qu...

AI Technical Summary

Problems solved by technology

However, in the first stage, they only perform semantic abstraction on single media and ignore media associations, which may cause the loss of associated information

In the second stage, shallow network structures are mostly used, which cannot fully model cross-media correlation information, thus limiting the ranking effect of unified representation

Images