Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Text-to-image generation method based on generative adversarial network

A technology for image generation and image generation, applied in image coding, image data processing, instruments, etc., to achieve the effect of rich details

Active Publication Date: 2020-06-09
SOUTH CHINA UNIV OF TECH
View PDF3 Cites 20 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

This problem becomes more severe when the scene and objects that need to be generated are more complex

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Text-to-image generation method based on generative adversarial network
  • Text-to-image generation method based on generative adversarial network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0037] The present invention will be further described in detail below in conjunction with the embodiments and the accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0038] Such as figure 1 , 2 , a text-to-image generation method based on generative adversarial networks, including the following steps:

[0039] 1) Input a meaningful text description into the network, which can be a description of representative attributes such as the type, size, quantity, color, shape, position, etc. of one or more entity objects. By using a bidirectional long short-term memory network (bi-directional LSTM), the two hidden states corresponding to each word in the text description are concatenated to represent the semantics of the word. The last hidden state is connected to the global sentence vector, and the other hidden states are concatenated to obtain the word feature matrix.

[0040] 2) Obtain the image feature matrix, the specific process is a...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a text-to-image generation method based on a generative adversarial network, and the method comprises the following steps: 1) inputting a text description into the network, andgenerating a word feature matrix and a sentence feature vector according to the text description; 2) adding a condition and a noise vector to the sentence feature vector to obtain an image feature matrix; 3) calculating a word context matrix of the image features; 4) calculating in the generative adversarial network by utilizing the image feature matrix and the word context matrix, and graduallygenerating images with increasingly high resolutions in three stages; 5) acquiring a local image feature matrix according to the generated image; and 6) evaluating the similarity between the generatedimage and the text description, and optimizing the next image generation. The image generation method not only can ensure that the content of the generated image is consistent with the semantics of the text description, but also can ensure that the generated image has more optimized image details, the resolution of the generated image can be effectively improved, and the diversity of the generated image is increased.

Description

technical field [0001] The invention relates to the field of image generation, in particular to a text-to-image generation method based on a generation confrontation network. Background technique [0002] Generating high-resolution and realistic images based on text descriptions is a very interesting research. In the industry, it not only provides help for deeper visual understanding for related research in the field of computer vision, but also has a wide range of real-world applications. In academia, it has become one of the most popular research directions in the field of computer vision in recent years, and has achieved remarkable results. Recurrent neural networks (RNNs) and generative adversarial networks (GANs) are often combined to generate realistic images based on natural language descriptions. These methods have been able to produce satisfactory results in some domains, such as creating beautiful images of flowers or birds. [0003] The original GAN ​​model con...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T9/00
CPCG06T9/002Y02T10/40
Inventor 田安捷陆璐
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com