Binocular parallax calculation method based on 3D convolutional neural network

Abstract

The invention relates to a binocular parallax calculation method based on a 3D convolutional neural network. The method comprises the steps of S1, performing feature extraction on input left and rightviews according to a defined multi-scale feature extraction method; s2, stacking the characteristics of the corresponding positions of the corresponding parallax of the left image and the right imageto obtain 4D cost volume; s3, performing cost aggregation by using a 3D CNN sub-network to obtain log-likelihood estimation of a disparity value, upsampling to an original image resolution to obtainlog-likelihood estimation of a possible disparity value of each pixel, and performing logarithm normalization operation to obtain new log-likelihood estimation; s4, calculating the real distribution of the settings; s5, performing back propagation training; s6, obtaining the disparity log-likelihood distribution of each pixel, and converting the disparity log-likelihood distribution into probability to obtain disparity probability distribution; s7, finding a disparity value corresponding to the maximum probability, and S8, obtaining normalized probability distribution according to the left disparity value, the right disparity value and the disparity probability distribution; and S9, obtaining a final estimation value of each pixel parallax through a weighted average operation. According tothe method, the parallax calculation precision can be effectively improved.

Description

technical field [0001] The invention relates to the field of binocular vision system processing, and more particularly, to a binocular disparity calculation method based on a 3D convolutional neural network. Background technique [0002] As a low-cost method to obtain depth, binocular vision system has important applications in many fields of robotics. Including mapping, obstacle avoidance, positioning, etc. Specifically, it has important applications in autonomous driving, augmented reality and other fields, such as 3D object detection, 3D environment perception, etc. It has the characteristics of low cost, high robustness and strong anti-interference. [0003] Traditional disparity estimation methods usually consist of four parts: feature extraction, cost calculation, cost aggregation, and disparity optimization. With the development of convolutional neural networks and related hardware, CNN estimation of disparity has become a better application. But there are still m...

AI Technical Summary

Problems solved by technology

But there are still many problems, such as the extraction of multi-scale features, high-precision optimization of disparity, etc.

Images