Hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks

Abstract

The invention belongs to the technical field of image data processing, and discloses a hand-drawn sketch generation method based on variational self-encoding and generative adversarial networks. The method comprises the following steps: obtaining a vector format sketch, performing format conversion, and constructing a new data set; obtaining global and local feature structures of stroke vectors inthe new data set; utilizing a decoder to obtain stroke normal distribution parameter data and stroke state classification distribution probability data of the new data set; randomly sampling from thenormal distribution and calculating a stroke state to obtain prediction output of each time step, and obtaining a sketch generated by prediction; obtaining true and false information of the input data by using a discriminator to obtain a sketch generation model fusing VAE and GAN; and inputting the data into a generation model to obtain a prediction generation sketch. According to the sketch generation method, the generated sketch is high in quality, and generation of multiple types of sketches is supported.

Description

technical field [0001] The invention belongs to the technical field of image data processing, and in particular relates to a hand-drawn sketch generation method based on variational self-encoding and generative confrontation network. Background technique [0002] As early as ancient times, sketches have played an important role in assisting human communication and creative design. At the same time, vector graphics require only the least space for storage and transmission during transmission. This situation promotes the development of sketches in artificial intelligence technology. develop. The research on sketches is mainly on recognition, and has achieved good results to a certain extent, but there is only a small amount of research on sketch generation. In 2018, a seq2seq variational self-encoder (VAE) model called Sketch-RNN proposed by Google, which can generate sketches based on manual input, was introduced as the first artificial intelligence technology and is known a...

AI Technical Summary

Problems solved by technology

The research on sketches is mainly in recognition, and achieved good results to a certain extent, but there is only a small amount of research on sketch generation.

In 2018, a seq2seq variational self-encoder (VAE) model called Sketch-RNN proposed by Google, which can generate sketches based on manual input, was introduced as the first artificial intelligence technology and is known as the most advanced technology ; but the model also has its own limitations, including: it can have a better output when learning a single category sketch, but when multiple categories are input into the model, the quality of the generated sketch will decrease; when a sketch is composed of multiple components , Sketch-RNN cannot capture the relative positional relationship between components very well, resulting in low quality of generated sketches

However, the quality of the sketch generated by this model needs to be improved. The encoder part of the Sketch-pix2seq model has a small receptive field, which can only learn the stroke relationship in a local range, but cannot capture the long-distance dependency between strokes, resulting in captured The effective information is limited, the sketch quality is reduced, and the Sketch-RNN and Sketch-pix2seq models are both based on the VAE generation framework

[0003] To sum up, the problems existing in the existing technology are: (1) The existing sketch generation schemes are all based on the VAE model, and the generation framework is single

[0004] (2) At present, Sketch-pix2seq, the best model for generating sketches, cannot learn the long-distance relationship between strokes when learning sketch features, and cannot better grasp the feature information of sketches from a global perspective.

[0005] The difficulty of solving the above technical problems: At present, the research on hand-drawn sketches has become a branch of computer vision. Previous research has mainly focused on the recognition of hand-drawn sketches, and achieved good results to a certain extent, but in hand-drawn sketches There is only a small amount of research on the generation of sketches, and there are few verifiable materials; in addition, there is no clear standard for the quality evaluation of sketches, and the evaluation of sketches is mainly based on human subjective evaluation.

Images