Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures

a neural network and image synthesis technology, applied in the field of image synthesis, can solve problems such as the actual synthesis of such sources

Active Publication Date: 2018-03-08
ARTOMATIX LTD
View PDF0 Cites 117 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Benefits of technology

The patent describes a system for generating a synthesized image by combining content from a source content image and texture from a source style image. The system uses a neural network to determine a localized loss function for each pixel in the source content image and the source style image. The localized loss function is used to optimize the value of each pixel in the synthesized image to a content loss function and a style loss function. The system can be performed on a server system or on a user device. The technical effect of this invention is to provide a more efficient and effective method for generating high-quality synthesized images.

Problems solved by technology

However, the actual synthesis of such sources is a hard problem that raises a variety of difficulties.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures
  • Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures
  • Systems and Methods for Providing Convolutional Neural Network Based Image Synthesis Using Stable and Controllable Parametric Models, a Multiscale Synthesis Framework and Novel Network Architectures

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0056]Turning now to the drawings, systems and methods for providing Convolutional Neural Network (CNN) based image synthesis in accordance with some embodiments of the invention are described. In many embodiments, processes for providing CNN-based image synthesis may be performed by a server system. In accordance with several embodiments, the processes may be performed by a “cloud” server system. In still further embodiments, the processes may be performed on a user device.

[0057]In accordance with many embodiments, the loss functions may be modeled using Gram matrices. In a number of embodiments, the loss functions may be modeled using covariance matrices. In accordance with several embodiments, the total loss may further include mean activation or histogram loss.

[0058]In accordance with sundry embodiments, a source content image, including desired structures for a synthesized image and a source style image, including a desired texture for the synthesized image, are received. A CNN...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

Systems and methods for providing convolutional neural network based image synthesis using localized loss functions is disclosed. A first image including desired content and a second image including a desired style are received. The images are analyzed to determine a local loss function. The first and second images are merged using the local loss function to generate an image that includes the desired content presented in the desired style. Similar processes can also be utilized to generate image hybrids and to perform on-model texture synthesis. In a number of embodiments, Condensed Feature Extraction Networks are also generated using a convolutional neural network previously trained to perform image classification, where the Condensed Feature Extraction Networks approximates intermediate neural activations of the convolutional neural network utilized during training.

Description

CROSS REFERENCED APPLICATION[0001]This application claims priority to U.S. Provisional Application Ser. No. 62 / 383,283, filed Sep. 2, 2016, U.S. Provisional Application Ser. No. 62 / 451,580, filed Jan. 27, 2017, and U.S. Provisional Application Ser. No. 62 / 531,778, filed Jul. 12, 2017. The contents of each of these applications are hereby incorporated by reference as if set forth herewith.FIELD OF THE INVENTION[0002]This invention generally relates to image synthesis and more specifically relates to image synthesis using convolutional neural networks based upon exemplar images.BACKGROUND[0003]With the growth and development of creative projects in a variety of digital spaces (including, but not limited to, virtual reality, digital art, as well as various industrial applications), the ability to create and design new works based on the combination of various existing sources has become an area of interest. However, the actual synthesis of such sources is a hard problem that raises a v...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G06T11/00G06T7/45
CPCG06T11/001G06T7/45G06T5/004G06T2207/20084G06T5/40G06T2207/20221G06T5/20G06T11/00G06T5/75
Inventor RISSER, ERIC ANDREW
Owner ARTOMATIX LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com