Hand-drawn image real-time retrieval method based on multi-granularity associative learning

Abstract

The invention belongs to the field of image retrieval, and particularly relates to a hand-drawn image real-time retrieval method based on multi-granularity associative learning, which comprises the following steps of: training an improved deep neural network model by adopting a triplet loss function and a multi-granularity associative learning method, extracting an embedded vector of a sketch branch by the trained deep neural network model, sending the sketch branch to a discriminator to judge the grade of the sketch branch, sending the sketch branch to a dimension reduction layer corresponding to the grade, calculating the Euclidean distance between the sketch branch and the image, and returning the retrieved top-k images according to the Euclidean distance; according to the method, a multi-stage model is designed, diversity confusion of incomplete sketches is avoided, and a progressive multi-granularity association learning method for the incomplete sketches is provided, so that the embedding space of each incomplete sketch approaches the embedding space of a subsequent sketch and a corresponding target picture, and the target picture is retrieved with the fewest sketch strokes as much as possible.

Description

technical field [0001] The invention belongs to the field of dynamic sketch retrieval, and in particular relates to a real-time retrieval method for hand-painted images based on multi-granularity associative learning. Background technique [0002] Image retrieval is divided into sample-based image retrieval (EBIR) and sketch-based image retrieval (SBIR) according to the type of image retrieved. SBIR is a method that uses hand-drawn sketches lacking color information and texture information as input, and then the retrieval system returns image library images similar to the hand-drawn sketches. The hand-drawn sketches involved in this method are an abstract expression of what humans see. Different from text and labels, hand-drawn sketches can convey image information that is difficult to express in words in a more intuitive and vivid way, effectively preventing Alienation of information in the process of transmission. For example, when a user wants to inquire about a product...

AI Technical Summary

Problems solved by technology

In the case of reference pictures, the sketches drawn by different painters for the same object have different degrees of abstraction, which leads to different shapes of sketches; in the absence of reference pictures, different drawers can only rely on Conceive and draw based on your own subjective impression, which greatly increases the diversity of sketch forms

Secondly, everyone's drawing level and painting style are different, which further increases the difference in style of the drawn sketches, resulting in differences in the semantic association of sketch data, which increases the difficulty of semantic understanding of sketches

Although state-of-the-art vision systems are good at identifying poorly drawn sketches, the time required to draw a complete sketch depends on the drawing ability of the drafter. If the result can only be retrieved after a complete sketch is drawn, this waiting time too long

Images