Sketch image retrieval method based on joint space attention and metric learning

Abstract

The invention discloses a sketch image retrieval method based on joint spatial attention and metric learning. The method comprises the following implementation steps: extracting an edge image from a natural image; forming a training set in a triple form by the sketch, the natural image and the edge image according to category correlation; preprocessing the training set image; constructing a retrieval network with a joint space attention module and an auxiliary classifier; training a retrieval network by adopting cross entropy classification loss, triple loss and intra-class distance constraint loss; after training is completed, discrete codes of the image library images are calculated through the retrieval network, retrieval results are obtained according to inverted sorting of Hamming distances between the discrete codes and query sketch codes, and natural images related to the retrieval intention of the user can be retrieved through the simple freehand sketch.

Description

technical field [0001] The invention relates to the fields of image processing technology and deep learning technology, in particular to a sketch image retrieval method based on spatial attention and metric learning. Background technique [0002] Content-based image retrieval methods require users to provide accurate relevant natural images for retrieval, which limits their application scenarios. Hand-drawn sketch is an intuitive form of retrieval output, and it is easy to obtain now that touch-screen devices are popular, and can express users' specific retrieval intentions without restriction. Therefore, the sketch image retrieval algorithm has a wide range of application values. [0003] Yu et al. [1] Inspired by AlexNet, a deep learning model Skech-a-Net suitable for sketch recognition is designed. A larger convolution kernel is used to enable the network to learn the features of sketches. A multi-scale network fusion method is used to fuse sketch features of various sca...

AI Technical Summary

Problems solved by technology

There are two problems to be improved in the above deep learning-based sketch image retrieval methods: First, most methods only use the edge map as an intermediate modality to reduce the domain difference between sketches and natural images, and do not further explore the edge map. Intrinsic relationship with natural images, such as the two have consistent key regions in space; second, image retrieval problems require compact intra-class coding, and the above-mentioned triplet-based methods do not solve the intra-class distribution caused by invalid triplets. scattered problems

Images