Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Method and device for generating dynamic human free viewpoint video based on neural network

A neural network and human body technology, applied in neural learning methods, biological neural network models, neural architectures, etc., can solve problems such as slow reconstruction sequences, slow speeds, and complex equipment

Active Publication Date: 2022-07-29
杭州新畅元科技有限公司
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, high-quality, high-fidelity human body free-viewpoint video acquisition usually relies on expensive laser scanners or multi-camera array systems to model the human body. Although the effect is more realistic, there are also some obvious shortcomings: first, The equipment is complicated, and these methods often require the construction of multi-camera arrays; second, the speed is slow, and it often takes at least 10 minutes to several hours to reconstruct a 3D human body model, and the reconstruction sequence is even slower

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Method and device for generating dynamic human free viewpoint video based on neural network
  • Method and device for generating dynamic human free viewpoint video based on neural network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0053] The following describes in detail the embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, and are intended to be used to explain the present application, but should not be construed as a limitation to the present application.

[0054] The following describes the method and apparatus for generating a dynamic human free-viewpoint video based on a neural network according to the embodiments of the present application with reference to the accompanying drawings.

[0055] figure 1 This is a schematic flowchart of a method for generating a dynamic human free-viewpoint video based on a neural network according to an embodiment of the present application.

[0056] Specifically, the present app...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The present application proposes a method and device for generating a dynamic human body free-viewpoint video based on a neural network, which relates to the technical field of computer vision and computer graphics, wherein the method includes: reconstructing a pre-scan model of a single human body; Describe a single human body shot to obtain a sequence of RGB images; deform the pre-scan model so that the deformed pre-scan model matches each frame of RGB image; sample the pre-scan model, and define a hidden code at each sampling point, And jointly optimize the hidden code and network parameters based on the neural network; obtain any rendering perspective, and generate free-view video based on the arbitrary rendering perspective. As a result, a sequence of RGB images is captured based on multiple RGB cameras, and a temporally continuous and dynamic free-viewpoint video is generated based on this sequence, resulting in more realistic and dynamic rendering results.

Description

technical field [0001] The present application relates to the technical fields of computer vision and computer graphics, and in particular, to a method and device for generating a dynamic human free-viewpoint video based on a neural network. Background technique [0002] Dynamic human free-view video generation is a key problem in the fields of computer graphics and computer vision. High-quality human free-view video has broad application prospects and important application value in the fields of film and television entertainment, human body digitization and so on. However, high-quality, high-fidelity human free-view video acquisition usually relies on expensive laser scanners or multi-camera array systems to model the human body. Although the effect is more realistic, it also has some obvious shortcomings: first, The equipment is complex, and these methods often require the construction of multi-camera arrays; second, the speed is slow, and it often takes at least 10 minut...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06T7/13G06T7/181G06T7/90G06N3/04G06N3/08
CPCG06T7/13G06T7/181G06T7/90G06N3/04G06N3/08G06T2207/10016G06T2207/20081G06T2207/30196
Inventor 刘烨斌李哲于涛
Owner 杭州新畅元科技有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products