Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A large-scale panoramic viewpoint synthesis method based on single-view rgb-d images

An RGB-D, viewpoint synthesis technology, applied in the field of image processing, can solve the problems of overlapping synthetic viewpoint structures, large area hole filling, deformation and distortion, etc., to achieve the effect of good visual perception, smaller hole area, and improved image quality.

Active Publication Date: 2022-07-12
XI AN JIAOTONG UNIV
View PDF6 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0007] Finally, for the problems of large-scale hole filling, composite viewpoint structure overlap, deformation and distortion in the process of large-scale viewpoint synthesis based on single-viewpoint RGB-D images, etc.
For the above-mentioned problems in the process of large-scale view synthesis of single-view RGB-D images, there are still many deficiencies in the existing methods

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A large-scale panoramic viewpoint synthesis method based on single-view rgb-d images
  • A large-scale panoramic viewpoint synthesis method based on single-view rgb-d images
  • A large-scale panoramic viewpoint synthesis method based on single-view rgb-d images

Examples

Experimental program
Comparison scheme
Effect test

preparation example Construction

[0051] see figure 1 , a large-scale panoramic viewpoint synthesis method based on a single viewpoint RGB-D image of the present invention, comprising the following steps:

[0052] S1. Perform 3D-Warping on the single-frame color image and the corresponding depth map respectively;

[0053] First, the single frame RGB-D image is P 0 (RGB 0 ,Depth 0 ) for 3D-Warping, the principle is to use the depth information and camera parameters to map the image of the known reference viewpoint to the imaging plane of the virtual camera, and obtain the virtual viewpoint and mask image with holes through 3D-Warping.

[0054] S2. Filling of virtual viewpoint holes based on CNN convolutional neural network;

[0055] When the viewpoint is transformed from a reference viewpoint to a virtual viewpoint position, a part of the area that is visible in the virtual viewpoint but not visible in the original reference viewpoint is revealed, resulting in the loss of a large amount of information, maki...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a large-scale panoramic viewpoint synthesis method based on a single-viewpoint RGB-D image. First, 3D-Warping is performed on an input single-frame color image and a corresponding depth map, respectively, to obtain a virtual viewpoint image with holes and a mask. film image. Next, the virtual viewpoint image and mask image with holes are used as the input of the CNN neural network. The output result of the network model is the image after filling the virtual viewpoint hole. Next, the newly generated virtual viewpoint in the horizontal direction is used as the new input. , through 3D-Warping and CNN neural network hole filling to obtain a virtual viewpoint image at any position in the space. The invention realizes the overall improvement of the quality of virtual viewpoints, and has important guiding significance for improving the synthesis of large parallax virtual viewpoints.

Description

technical field [0001] The invention belongs to the technical field of image processing, and in particular relates to a large-scale panoramic viewpoint synthesis method based on a single viewpoint RGB-D image. Background technique [0002] With the development of science and technology and the improvement of people's living standards, the visual experience brought by the traditional two-dimensional video screen has become more and more unable to meet people's viewing needs. Therefore, many research institutions and industries have paid more and more attention to the improvement of the vividness of video screens, and omnidirectional video screens, free-view video screens, and interactive stereoscopic video screens have emerged as the times require. Among them, free-view video screens have been widely used because they can meet the needs of users to watch video screens from any viewpoint. In recent years, the rapid development of image acquisition technology, network communic...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06T5/00G06T7/50G06N3/04G06N3/08
CPCG06T5/005G06T7/50G06N3/08G06T2207/10004G06T2207/10024G06T2207/10028G06T2207/20081G06T2207/20084G06N3/045
Inventor 杨勐马勇郑南宁
Owner XI AN JIAOTONG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com