Knowledge graph guided multi-scene image generation method

Abstract

The invention provides a knowledge graph guided multi-scene image generation method, and belongs to the field of image generation. According to the method, the knowledge graph is used for assisting in completing an image generation task for the first time, firstly, the knowledge graph containing object layout relations is constructed, then, a set of object labels are input into the graph, and a plurality of layout relation graphs conforming to facts is obtained through a layout search module; Knowledge information in the graph is extracted from the layout relation graph through a pre-trained knowledge module, and finally a generator and a discriminator are trained in combination with a knowledge object matrix and a global knowledge vector obtained in the knowledge module when the layout relation graph passes through an image generation module, so as to generate a scene image corresponding to each relation graph. According to the invention, the knowledge graph is utilized to realize a one-to-many task of generating a plurality of images by using a group of labels, and the image generation quality is improved by embedding knowledge representation information. The present invention is evaluated using a real image data set, and the improvement on the most advanced baseline is observed.

Description

technical field [0001] The invention belongs to the field of image generation, and in particular relates to a method for generating multiple scene images guided by knowledge graphs. Background technique [0002] The knowledge graph is a database in units of triples, which store entity information and relationship information between entities. In the application of knowledge graph methods, the KG2E method in the trans series is a classic One of the knowledge representation methods. This method can embed the entities and relationships in the map into a high-dimensional Gaussian distribution. When the model is trained, the KL divergence is used to make the distribution of the head entity minus the tail entity as close as possible to the relationship. distributed. This knowledge representation method can introduce the information in the map into other models in the form of distribution, and it is also the method used in the present invention to extract map information. [0003...

AI Technical Summary

Problems solved by technology

[0004] Most of the current text synthesis scene image methods have the following problems: (1) the current text is often a sentence, and the user needs to give a sentence to generate an image in practical applications in life, which is inconvenient; (2) ) There should be more than one image that matches the description text, but most current methods can only achieve one-to-one generation tasks, and do not perform well when generating complex scene images with many objects, and cannot generate good layouts

Images