End-to-end stereo matching method based on convolutional neural network

Abstract

The invention discloses an end-to-end stereo matching method based on a convolutional neural network. The end-to-end stereo matching method comprises the following steps: respectively extracting respective feature maps of left and right images through a residual convolutional neural network; respectively extracting feature information of the left and right feature maps on multiple scales by usingthe feature pyramid to obtain final feature maps of the left and right images; fusing the final feature maps of the left and right images to form a four-dimensional cost amount; using a three-dimensional convolutional neural network stacked by a multi-scale hourglass network to carry out cost normalization on a four-dimensional cost amount, and obtaining a disparity map through up-sampling and disparity regression. According to the method, global information can be fully utilized, so that a more accurate disparity map is obtained; compared with a traditional stereo matching algorithm, the problem that the matching effect in an ill-conditioned area is poor is greatly improved, the algorithm robustness is better, and the generalization ability is higher. Compared with other stereo matching algorithms based on the convolutional neural network, the matching effect of the details of the disparity map is effectively improved, and the corresponding mismatching rate is lower.

Description

technical field [0001] The invention relates to the fields of stereo vision and deep learning, in particular to an end-to-end stereo matching method based on a convolutional neural network. Background technique [0002] Stereo matching is essential for many computer vision applications, such as autonomous driving, robot navigation, augmented reality, and 3D reconstruction. By finding pixel-level correspondences between two images, stereo matching algorithms aim to construct a disparity map from a pair of rectified stereo images. First, the left and right image pairs are obtained by the binocular camera, and after image correction, they are sent to the stereo matching module to obtain an accurate disparity map, and there is a one-to-one correspondence between the disparity and the depth, which is inversely proportional to each other, and can be calculated based on the disparity map Get the depth information of the object. It can be applied to various practical scenarios. ...

AI Technical Summary

Problems solved by technology

For the existing stereo matching network, spatial features are obtained through spatial pyramid pooling, but the pooling operation loses a lot of feature information, and the matching effect on the details of objects is not ideal

Images