System and method for progressive stereo matching of digital images

Abstract

A method and a system for progressive stereo matching of digital images representing a scene. In general, the present invention uses a progressive iterative technique that includes a disparity gradient limit principle and a least commitment strategy. Generally unambiguous pixel matches are found by beginning with a few reliable pixel matches and finding progressively more unambiguous pixel matches. Unambiguous pixel matches are used to define the search ranges for each pixel to guide matching in the current iteration. Unambiguous pixel matches then are found using a novel correlation technique and based on a correlation score associated with a pixel match. The search range is capable of being rotated, and is part of a novel correlation technique that provides a more robust estimate of pixel match reliability. Potential pixel matches found in the search ranges are tested for ambiguity and unambiguous matches are selected and added to the reliable pixel matches.

Description

BACKGROUND OF THE INVENTION [0001] 1. Field of the Invention [0002] The present invention relates in general to computer vision and more particularly to a method and a system for progressive stereo matching of digital images that represent a scene. [0003] 2. Related Art [0004] Stereo matching of digital images is widely used in many computer vision applications (such as, for example, fast object modeling and prototyping for computer-aided drafting (CAD), object segmentation and detection for human-computer interaction (HCI), video compression, and visual surveillance) to provide three-dimensional (3-D) depth information. Stereo matching obtains images of a scene from two or more cameras positioned at different locations and orientations in the scene. These digital images are obtained from each camera at approximately the same time and points in each of the image are matched corresponding to a 3-D point in space. In general, points from different images are matched by searching a por...

AI Technical Summary

Benefits of technology

[0008] To overcome the limitations in the prior art as described above and other limitations that will become apparent upon reading and understanding the present specification, the present invention includes a method and system for performing progressive stereo matching of digital images. In particular, the present invention uses a progressive iterative technique that uses a disparity gradient limit principle and a least commitment strategy to determine generally unambiguous pixel matches. The progressive iterative method of the present invention begins with a few reliable pixel matches and during successive iterations finds progressively more unambiguous pixel matches based on a novel correlation technique and a correlation score associated with a pixel match. Unambiguous pixel matches found in a previous iteration are used to define search ranges for subsequent pixels to reliably and efficiently determine matches in a current iteration. The present invention eliminates the problems of dense matching techniques whereby the same search range is used for the entire image. In the present invention, potentially false matches are reduced by dynamically changing the search range based on unambiguous matches found in previous iterations.

[0009] The present invention provides reliable and accurate pixel matching that begins with a reliable set of pixel matches and quickly adds a greater number of reliable pixel matches based on a pixel correlation. In addition, because at each step of the process the pixel matches are unambiguous, the present invention is highly efficient and fast because backtracking due to unreliable pixel matches is eliminated. Unlike many prior stereo matching techniques, the present invention works well with images containing featureless regions and is capable of changing an orientation of a search range based on the current set of reliable pixel matches.

[0010] In general, the method of the present invention includes determining a set of reliable (or unambiguous) pixel matches and progressively adding a greater number of unambiguous pixel matches to the set. The initial set of reliable pixel matches may be obtained either automatically (using a suitable matching technique that yields reliable pixel matches) or manually (such as through user input). Pixel matches are sought by defining a search range for each image and examining pixels within the search ranges for potential matches. The search range is capable of being rotated, and is part of a novel correlation technique of the present invention that provides a more robust estimate of pixel match reliability. After an iteration the set of reliable pixel matches acts as a constraint and helps define a new search range. This dynamic search range reduces potential matching ambiguities and increases the robustness of the process. Potential pixel matches found in the search ranges are tested for ambiguity and any unambiguous matches are selected and added to the set of reliable pixels matches. The ambiguity testing includes determining a correlation score for the pixel match and classifying the match based on the correlation score. The present invention also includes a system for progressive image matching that incorporates the method of the present invention.

Problems solved by technology

One disadvantage of feature matching, however, is that only a small subset of image pixels is used to match features.

This means that if image pixels used in the matching process are unreliable then only a coarse and inaccurate 3-D representation of the actual scene is produced.

Although this assumption may be valid for relatively textured portions of an image and for image pairs having only small differences, the assumption may lead to unreliable matching at occlusion boundaries and within featureless regions of an image.

In addition, template matching yields many false matches due in part to the unreliability of matched image pixels.

These and other existing stereo matching techniques have the disadvantage of having difficulty specifying an appropriate search range and being unable to adapt the search range depending on the observed scene structure.

Existing dense matching techniques use the same search range (such as a small search window) for the entire image and thus may yield many false matches.

In addition, some matching techniques consider only one match at a time and propagate the match in a small area, which also has the disadvantage of yield many false matches.

Images