Multi-sensor deep fusion 3D target detection method for automatic driving

Abstract

The invention discloses a multi-sensor deep fusion 3D target detection method for automatic driving, which effectively solves the problem of space-time desynchrony of sensors through an edge calibration technology, effectively solves the problem of depth estimation model training when labels are insufficient through target center data enhancement, and improves accuracy of target center data enhancement. Through the uncertainty estimation technology and the scene correlation loss function, the robustness and precision of the depth estimation model under the scene change condition are effectively improved. Through 2D-3D lossless information conversion, a voxel conversion-to-point technology, coarse-to-fine depth alignment and multi-branch supervision flow, the problems that different modal feature spaces are inconsistent and information loss exists in conversion are effectively solved, and a final architecture can enhance the complementary effect of multi-modal information in an automatic driving scene. Interference among multi-modal information is reduced, and good performance is achieved under the scenes of shielding, sparse point cloud, illumination variation and the like.

Description

technical field [0001] The invention relates to the technical field of machine vision, in particular to a multi-sensor deep fusion 3D target detection method for automatic driving. Background technique [0002] 3D object detection has a wide range of applications in unmanned driving, robots, augmented reality and other scenarios. Compared with ordinary 2D detection, 3D detection additionally provides the length, width, height and deflection angle information of the target object. It is an important perception basis for 3D scene understanding and autonomous decision-making and planning. Currently there are several representative fusion schemes as follows. [0003] MV3D is the first work to fuse LiDAR BEV features and camera front view features, but the results are not good due to feature misalignment. [0004] Multi-task multi-sensor fusion technology (Ming Liang, Bin Yang, Yun Chen, Rui Hu, Raquel Urtasun; Proceedings of the IEEE / CVF Conference on Computer Vision and Patt...

AI Technical Summary

Problems solved by technology

Therefore, its performance is limited by the 2D detector

[0007] 3D-CVF (3D-CVF: Generating Joint Camera and LiDAR Features Using Cross-view Spatial Feature Fusion for 3D Object Detection ECCV 2020pp 720-736) extracts 2D image features, projects them into 3D LiDAR space, and passes adaptive attention network These two features are fused, and finally the results are optimized in two stages based on the 3D region of interest (ROI), and the alignment method from 2D to 3D is adopted. Since it is aligned in 3D space, the effect is better, but the image features are in the projection Multiple downsampling will be performed before entering the LiDAR space, and the fine-grained spatial correspondence between dense 2D pixels and 3D space will be lost

However, many of their fusion modules are tightly coupled to specific LiDAR detection networks, cannot be cropped as needed when applied, and since they often lose information during projection between different spaces, there is no fine-grained pixel-level Alignment, the performance of the above fusion network will be limited

Images