Real-time depth completion method based on pseudo depth map guidance

Abstract

The invention discloses a real-time depth completion method based on pseudo depth map guidance. The method includes: enabling an RGB camera to collect an RGB image, and enabling a laser radar to detect to obtain a sparse depth image; performing data processing of morphological operation on the sparse depth map to generate a pseudo depth map; establishing a neural network structure, and processing the pseudo depth map, the RGB map and the sparse depth map to obtain a dense depth map; training the neural network structure, setting total loss function optimization supervision, and obtaining parameter values of all parameters in the neural network structure; and loading the parameter values of all parameters into the neural network structure, inputting a single-frame sparse depth map to be measured into the neural network structure, and outputting a dense depth map. According to the method, the precision of depth completion is effectively improved, wrong pixels of a sparse depth map can be corrected, structural similarity supervision is provided, the depth residual image is regressed during prediction, and a high-precision depth completion result can be obtained under a high real-time condition.

Description

technical field [0001] The present invention relates to a depth complement method based on laser radar and RGB camera in the technical field of unmanned vehicles and robot visual perception, in particular to a real-time depth complement method guided by a pseudo-depth map. Background technique [0002] Depth completion refers to the technology of projecting the point cloud collected by lidar into the sparse depth map formed by RGB image space to complete, so as to obtain a dense depth map with the same data density as the corresponding RGB image. It is the key technology for complex systems such as unmanned driving and autonomous robots to efficiently perceive the three-dimensional environment. Lidar is a common distance sensor, and the collected point cloud provides accurate 3D information, but the depth map projected by the original Lidar point cloud is very sparse, only about 3% to 4% of the dense image compared with the medium resolution. There is a depth value on the p...

AI Technical Summary

Problems solved by technology

However, these methods have many shortcomings: the introduction of pre-training models requires more data annotation, complex post-processing networks will increase the running time of single-frame depth maps, and the introduction of 3D convolution will also increase computing resources.

Images