Cross-media sorting method based on deep neural network

Abstract

The invention discloses a cross-media sorting method based on a deep neural network. The method comprises the following steps that (1) a sorting sample of using a text to retrieve an image or a sorting sample of using the image to retrieve the text is constructed as a training sample; (2) cross-media sorting study based on the deep neural network is conducted to the constructed training sample, and multimedia semantic space and a cross-media sorting model are obtained; (3) the studied cross-media sorting model is adopted to conduct cross-media retrieval. Due to the use of the deep neural network driven by cross-media sorting data, the semantic comprehensive ability of the obtained retrieval model is stronger, and using the text to retrieve the image or using the image to retrieve the text to obtain a superficial model method with a traditional performance is better.

Description

technical field [0001] The invention relates to cross-media retrieval, in particular to a deep neural network-based cross-media sorting method. Background technique [0002] Images have rich semantics. Generally speaking, an image is composed of individual pixels, and the computer cannot directly understand the semantic information contained in the image. With the development of multimedia technology and network technology, more and more images emerge. Retrieval technology can help users quickly find the content they are interested in from massive data, and has become an increasingly important field in computer application technology. The traditional retrieval technology, whether it is keyword-based retrieval or content-based retrieval, cannot well meet the needs of users who want to retrieve images with text or retrieve text with images. In a keyword-based retrieval system, images need to be annotated in advance. However, the huge number of existing images makes the lab...

AI Technical Summary

Problems solved by technology

However, the number of existing images is huge, making the labeling process a huge amount of work.

Moreover, since the annotation content will inevitably be affected by the subjective factors of the annotator, for the same image, different annotators may annotate different keywords, so the keywords often cannot objectively reflect all the semantics contained in the image.

The content-based image retrieval technology does not need to label the image, and image retrieval is realized based on comparing the similarity between the retrieval samples submitted by users and the retrieved images, but the traditional content-based image retrieval technology has two weaknesses: one One is that the user can only retrieve media objects of the same type as the query example, for example, images can only be retrieved through images; the other is that there is a semantic gap between the underlying features of the image and the high-level semantics, that is, the low-level features cannot directly reflect the high-level semantics, so the retrieval performance is limited

Images