Method and device for generating image through text with introduced class information

Abstract

The invention discloses a method and device for generating an image by a text with introduced class information, the method for generating the image by the text with introduced class information comprises a training stage and a testing stage, the training stage is based on a generative adversarial network, and a natural language text describing an image, the class label of the text, a corresponding real image training generator and a discriminator are utilized; and in the test stage, a corresponding image is generated in a generator by using the text and the class label thereof. The method and device have the advantages that text semantic image features and class information image features are generated through respective transcoding according to text information codes and class information codes, then the two levels of image features are fused to decode and generate the image, corresponding class information is introduced in the image generation process to enhance the correlation between the generated image and the text, and meanwhile, through the multi-stage generation process in the training process, the higher-resolution image is gradually generated, and the training difficulty of directly generating the high-resolution image is reduced.

Description

technical field [0001] The invention relates to the technical field of deep learning generation models, in particular to a method and device for generating images from texts by introducing class information. Background technique [0002] Image generation from text is an important problem and has wide applications, such as computer-aided medicine, news image generation, etc. [0003] The research on image generation methods from text is mainly based on two generative models, Conditional Variational Auto-Encoder (CVAE for short) and Conditional Generative Adversarial Networks (CGAN for short). Among them, the pictures generated by the CVAE method often have the problem of blurred pictures. Now the mainstream methods are all based on the CGAN model. [0004] Due to the instability of GAN's own training, it is very difficult to directly generate high-resolution images from text descriptions. Therefore, the hierarchical generation confrontation network (StackGAN) proposes a meth...

AI Technical Summary

Problems solved by technology

[0007] The present invention aims at the deficiencies of the above-mentioned existing image generation methods, and provides a text generation image method and device that introduces class information, which can introduce the class information to which the text belongs during the image generation process, and use the class information to constrain the same class of text to generate pictures Relevance, while solving the problem that a single text description may not be comprehensive

Images