Method for joint estimation of scene depth and semantics by single image

Abstract

The invention belongs to the field of computer vision and computer graphics. An iterative network is designed to jointly estimate depth information and semantic information, and mutual prediction results are improved by utilizing complementary characteristics between the depth information and semantic information. According to the technical scheme, the method for jointly estimating the scene depthand the semantics through the single image comprises the following steps: performing photographing through any device with a monocular camera, and using an obtained color image as input of a network;and 2) iteration network: inputting a color image into a framework formed by a multi-task depth convolution network and used for depth estimation and semantic segmentation iteration combined optimization, estimating depth and semantic information of the color image, using the depth information to reconstruct a three-dimensional scene, and using the semantic information to realize understanding ofthe scene. The method is mainly applied to an image processing occasion.

Description

Technical field [0001] The invention belongs to the field of computer vision and computer graphics, and particularly relates to using a deep learning method to estimate the depth information and semantic information of a scene. Background technique [0002] In the field of computer vision, monocular depth estimation has always been a long-standing and discussed topic. Depth information is of great help in applications such as 3D reconstruction, virtual reality, and navigation. Nowadays, although there are many hardwares that can directly obtain the depth map, they all have their own shortcomings. For example, 3D LIDAR equipment is very expensive; structured light-based depth camera like Kinect, etc. cannot be used outdoors, the measurement distance is limited and the depth map noise is relatively large; the binocular camera needs to use a stereo matching algorithm, which requires a large amount of calculation and is not rich in texture The scene effect is not good. The monocul...

AI Technical Summary

Problems solved by technology

For example, 3D LIDAR equipment is very expensive; depth cameras based on structured light, such as Kinect, cannot be used outdoors, and the measurement distance is limited and the depth image noise is relatively large; binocular cameras need to use stereo matching algorithms, which require a large amount of calculation and are not rich in textures. scene doesn't work well

Images