1327 results about "Stereo matching" patented technology

There are provided a method and an apparatus for generating a multi-viewpoint depth map, and a method for generating a disparity of a multi-viewpoint image. A method for generating a multi-viewpoint depth map according to the present invention includes the steps of: (a) acquiring a multi-viewpoint image constituted by a plurality of images by using a plurality of cameras (b) acquiring an image and depth information by using a depth camera; (c) estimating coordinates of the same point in a space in the plurality of images by using the acquired depth information; (d) determining disparities in the plurality of images with respect to in the same point by searching a predetermined region around the estimated coordinates; and (e) generating a multi-viewpoint depth map by using the determined disparities. According to the above-mentioned present invention, it is possible to generate a multi-viewpoint depth map within a shorter time and generate a multi-viewpoint depth map having higher quality than a multi-viewpoint depth map generated by using known stereo matching.

This invention provides a system and method for determining the three-dimensional alignment of a modeledobject or scene. After calibration, a 3D (stereo) sensor system views the object to derive a runtime 3D representation of the scene containing the object. Rectified images from each stereo head are preprocessed to enhance their edge features. A stereo matching process is then performed on at least two (a pair) of the rectified preprocessed images at a time by locating a predetermined feature on a first image and then locating the same feature in the other image. 3D points are computed for each pair of cameras to derive a 3D point cloud. The 3D point cloud is generated by transforming the 3D points of each camera pair into the world 3D space from the world calibration. The amount of 3D data from the point cloud is reduced by extracting higher-level geometric shapes (HLGS), such as line segments. Found HLGS from runtime are corresponded to HLGS on the model to produce candidate 3D poses. A coarse scoring process prunes the number of poses. The remaining candidate poses are then subjected to a further more-refined scoring process. These surviving candidate poses are then verified by, for example, fitting found 3D or 2D points of the candidate poses to a larger set of corresponding three-dimensional or two-dimensional model points, whereby the closest match is the best refined three-dimensional pose.

An imaging apparatus includes: a plurality of imaging elements; a plurality of solid lenses that form images on the plurality of imaging elements; a plurality of optical axis control units that control the directions of the optical axes of light that is incident to each of the plurality of imaging elements; a plurality of video processing units that convert photoelectric converted signals output from each of the plurality of imaging elements to video signals; a stereo image processing unit that, by performing stereo matching processing based on the plurality of video signals converted by the plurality of video processing units, determines the amount of shift for each pixel, and generates compositing parameters in which the shift amounts that exceed the pixel pitch of the plurality of imaging elements are normalized to the pixel pitch; and a video compositing processing unit that generates high-definition video by compositing the video signals converted by the plurality of video processing units, based on the compositing parameters generated by the stereo image processing unit.

A vehicle environment recognition system includes stereo-image taking means for taking images of an environment around a subject vehicle and for outputting the images as a reference image and a comparative image, first stereo matching means for forming a first distance image on the basis of the reference image and the comparative image or on the basis of two images obtained by preprocessing the reference image and the comparative image, second stereo matching means for forming a second distance image on the basis of two images obtained by preprocessing the reference image and the comparative image in a different manner, detection means for detecting objects in the reference image on the basis of the first and second distance images, and selection means for selecting one of the results of detection based on the first and second distance images.

A two-dimensional moving track calculating unit 45 is provided for calculating a two-dimensional moving track of each individual person in each of a plurality of video images by tracking the position on each of the plurality of video images which is calculated by a person position calculating unit 44, and a three-dimensional moving track calculating unit 46 carries out stereo matching between two-dimensional moving tracks in the plurality of video images, which are calculated by the two-dimensional moving track calculating unit 45, to calculate a degree of match between the two-dimensional moving tracks, and calculates a three-dimensional moving track of each individual person from two-dimensional moving tracks each having a degree of match equal to or larger than a specific value.

The invention discloses a parallax optimization algorithm-based binocular stereo vision automatic measurement method. The method comprises the steps of 1, obtaining a corrected binocular view; 2, matching by using a stereo matching algorithm and taking a left view as a base map to obtain a preliminary disparity map; 3, for the corrected left view, enabling a target object area to be a colorized master map and other non-target areas to be wholly black; 4, acquiring a complete disparity map of the target object area according to the target object area; 5, for the complete disparity map, obtaining a three-dimensional point cloud according to a projection model; 6, performing coordinate reprojection on the three-dimensional point cloud to compound a coordinate related pixel map; 7, using a morphology method to automatically measure the length and width of a target object. By adopting the method, a binocular measuring operation process is simplified, the influence of specular reflection, foreshortening, perspective distortion, low textures and repeated textures on a smooth surface is reduced, automatic and intelligent measuring is realized, the application range of binocular measuring is widened, and technical support is provided for subsequent robot binocular vision.

The invention discloses a binocular stereo vision three-dimensional measurement method based on line structured light scanning, which comprises the steps of performing stereo calibration on binocularindustrial cameras, projecting laser light bars by using a line laser, respectively acquiring left and right laser light bar images, extracting light bar center coordinates with sub-pixel accuracy based on a Hessian matrix method, performing light bar matching according to an epipolar constraint principle, and calculating a laser plane equation; secondly, acquiring a line laser scanning image of aworkpiece to be measured, extracting coordinates of the image of the workpiece to be measured, calculating world coordinates of the workpiece to be measured by combining binocular camera calibrationparameters and the laser plane equation, and recovering the three-dimensional surface topography of the workpiece to be measured. Compared with a common three-dimensional measurement system combininga monocular camera and line structured light, the binocular stereo vision three-dimensional measurement method avoids complicated laser plane calibration. Compared with the traditional stereo vision method, the binocular stereo vision three-dimensional measurement method reduces the difficulty of stereo matching in binocular stereo vision while ensuring the measurement accuracy, and improves the robustness and the usability of a visual three-dimensional measurement system.

The invention discloses a binocular stereo vision based intelligent three-dimensional human face rebuilding method and a system; the method comprises: preprocessing operations including image normalization, brightness normalization and image correction are carried out to a human face image; a human face area in the human face image which is preprocessed is obtained and human face characteristic points are extracted; the object is rebuilt by projection matrix, so as to obtain internal and external parameters of a vidicon; based on the human face characteristic points, gray level cross-correlation matching operators are expanded to color information, and a parallax image generated by stereo matching is calculated according to information including polar line restraining, human face area restraining and human face geometric conditions; a three-dimensional coordinate of a human face spatial hashing point cloud is calculated according to the vidicon calibration result and the parallax image generated by stereo matching, so as to generate a three-dimensional human face model. By adopting the steps, more smooth and vivid three-dimensional human face model is rebuilt in the invention.

A method and apparatus for tracking objects used in connection with athletic activities or sporting events, especially, balls, pucks, and the like. The method includes the steps of differencing present and previous frames of a video image including the, for example, ball to obtain motion regions, converting the motion regions to HSV color space, extracting the region corresponding to the ball based on empirical color data about the ball, obtaining a motion vector based on the motion of the ball region from a previous frame to the current frame, and updating the ball trajectory based on the newest motion vector obtained. The method also preferably includes a step of identifying completed trajectories based on preset constraints. The method is preferably expanded on by using at least one pair of cameras to provide a three-dimensional trajectory, and sometimes preferable expanded on by using a plurality of cameras, especially a plurality of pairs of cameras. An apparatus according to the present invention includes at least one camera connected to a computer which operates to difference previous and current frames, compute the ball track, convert ball regions to HSV color space and output the tracking and video information. In a case where one or more pairs of cameras are used, the computer is preferably also provided with a stereo matching device or module for matching the tracking results from respective cameras and / or respective pairs of cameras.

The present invention discloses a face in-vivo detection method and system. The face in-vivo detection method comprises the steps of: simultaneously and respectively acquiring two images of an identification object by two cameras; positioning face regions in the two images by adopting a face classifier so as to acquire face images which respectively correspond to the two images; carrying out feature point positioning on the two face images; carrying out feature point rapid stereo matching on the face images subjected to feature point positioning; calculating parallax errors of the two face image at match feature points and acquiring depth information values of the match feature points; and according to the depth information values of a plurality of match feature points, judging whether the face of the identification object is a living body. According to the present invention, excessive cooperation of a user is not required; safety is high; hidden performance is high; attitude adaptability is high; the application range is wide; a matching speed is high; and user experience is good.

A method for determining an augmented reality scene by a mobile device includes estimating 3D geometry and lighting conditions of the sensed scene based on stereoscopic images captured by a pair of imaging devices. The device accesses intrinsic calibration parameters of a pair of imaging devices of the device independent of a sensed scene of the augmented reality scene. The device determines two dimensional disparity information of a pair of images from the device independent of a sensed scene of the augmented reality scene. The device estimates extrinsic parameters of a sensed scene by the pair of imaging devices, including at least one of rotation and translation. The device calculates a three dimensional image based upon a depth of different parts of the sensed scene based upon a stereo matching technique. The device incorporates a three dimensional virtual object in the three dimensional image to determine the augmented reality scene.

The present invention provides a method for generating a high-resolution virtual-focal-plane image which is capable of generating a virtual-focal-plane image with an arbitrary desired resolution simply and rapidly by using multi-view images.The method of the present invention comprises a disparity estimating process step for estimating disparities by performing the stereo matching for multi-view images consisting of multiple images with different capturing positions and obtaining a disparity image; a region selecting process step for selecting an image among multi-view images as a basis image, setting all remaining images as reference images, and selecting a predetermined region on the basis image as a region of interest; a virtual-focal-plane estimating process step for estimating a plane in the disparity space for the region of interest based on the disparity image, and setting the estimated plane as a virtual-focal-plane; and an image integrating process step for obtaining image deformation parameters used for deforming each reference image to the basis image for the virtual-focal-plane, and generating the virtual-focal-plane image by deforming multi-view images with obtained image deformation parameters.

An image processing device comprising: a plurality of image pickup sections that pick-up, from respectively different positions, a same object of sensing which carries out an instructing action, and outputting image information thereof; a position information sensing section which, by carrying out stereo matching on the plurality of image information outputted from the plurality of image pickup sections by using, as an object, two-dimensional regions, which correspond to a search space which is a three-dimensional space set in advance as a space in which the instructing action can be carried out, senses position information expressing matching positions which are positions of the object of sensing in the three-dimensional space at points corresponding to one another in the two-dimensional regions; and a specific position sensing section that senses, among the matching positions expressed by the position information, a specific position which is a matching position which matches predetermined conditions, is provided.

The invention discloses a method for acquiring a range image by stereo matching of multi-aperture photographing based on color segmentation. The method comprises the following steps of: (1) carrying out image correction on all input images; (2) carrying out the color segmentation on a reference image, and extracting an area with consistent color in the reference image; (3) respectively carrying out local window matching on the multiple input images to obtain multiple parallax images; (4) removing mismatched points which are generated during the matching by applying a bilateral matching strategy; (5) synthesizing the multiple parallax images into a parallax image, and filling parallax information of the mismatched points; (6) carrying out post-treatment optimization on the parallax image to obtain a dense parallax image; and (7) converting the parallax image into a range image according to the relationship between parallax and depth. By acquiring range information from multiple viewpoint images and utilizing image information provided by the multiple viewpoint images, the invention can not only solve the problem of mismatching brought by periodic repetitive textural features and shelter and the like in the images, but also can improve matching precision and obtain an accurate range image.

Disclosed are a method and apparatus for feature-based stereo matching. A method for stereo matching of a reference image and at least one comparative image captured by at least two cameras from different points of view using a computer device includes collecting the reference image and the at least one comparative image, extracting feature points from the reference image, tracking points corresponding to the feature points in the at least one comparative image using an optical flow technique, and generating a depth map according to correspondence-point tracking results.

The invention provides a binocular vision obstacle detection method based on three-dimensional point cloud segmentation. The method comprises the steps of synchronously collecting two camera images of the same specification, conducting calibration and correction on a binocular camera, and calculating a three-dimensional point cloud segmentation threshold value; using a three-dimensional matching algorithm and three-dimensional reconstruction calculation for obtaining a three-dimensional point cloud, and conducting image segmentation on a reference map to obtain image blocks; automatically detecting the height of a road surface of the three-dimensional point cloud, and utilizing the three-dimensional point cloud segmentation threshold value for conducting segmentation to obtain a road surface point cloud, obstacle point clouds at different positions and unknown region point clouds; utilizing the point clouds obtained through segmentation for being combined with the segmented image blocks, determining the correctness of obstacles and the road surface, and determining position ranges of the obstacles, the road surface and unknown regions. According to the binocular vision obstacle detection method, the camera and the height of the road surface can be still detected under the complex environment, the three-dimensional segmentation threshold value is automatically estimated, the obstacle point clouds, the road surface point cloud and the unknown region point clouds can be obtained through segmentation, the color image segmentation technology is ended, color information is integrated, correctness of the obstacles and the road surface is determined, the position ranges of the obstacles, the road surface and the unknown regions are determined, the high-robustness obstacle detection is achieved, and the binocular vision obstacle detection method has higher reliability and practicability.

A method of and apparatus for detecting people using a stereo camera. The method includes: calculating three-dimensional information regarding a moving object from a pair of image signals received from the stereo camera using stereo matching and creating a height map for a specified discrete volume of interest (VOI) using the three-dimensional information; detecting a people candidate region estimated as including one or more persons by finding connected components from the height map using a predetermined algorithm; and generating a histogram with respect to the people candidate region, detecting different height regions using the histogram, and detecting a head region by analyzing the different height regions using a tree structure.

The invention discloses a binocular stereoscopic vision matching method combining depth characteristics. The binocular stereoscopic vision matching method comprises: obtaining a depth characteristic pattern from left and right images through a convolutional neural network; calculating a truncation similarity measurement degree of pixel depth characteristics by taking the depth characteristics as the standard, and constructing a truncation matching cost function combining color, gradients and depth characteristics to obtain a matched cost volume; processing the matched cost volume by adopting a fixed window, a variable window and a self-adaptive weight polymerization or guide filtering method to obtain a cost volume polymerized by a matching cost; selecting an optimal parallax error of the cost volume by adopting WTA (Wireless Telephony Application) to obtain an initial parallax error pattern; then finding a shielding region by adopting a double-peak test, left-right consistency detection, sequence consistency detection or shielding constraint algorithm, and giving a shielding point to a parallax error value of a same-row point closest to the shielding point to obtain a parallax error pattern; and filtering the parallax error pattern by adopting a mean value or bilateral filter to obtain a final parallax error pattern. By adopting the binocular stereoscopic vision matching method combining the depth characteristics, the incorrect matching rate of three-dimensional matching can be effectively reduced, the images are smooth and image edges including edges of small objects are effectively kept.

The invention discloses a regional depth edge detection and binocular stereo matching-based three-dimensional reconstruction method, which is implemented by the following steps: (1) shooting a calibration plate image with a mark point at two proper angles by using two black and white cameras; (2) keeping the shooting angles constant and shooting two images of a shooting target object at the same time by using the same camera; (3) performing the epipolar line rectification of the two images of the target objects according to the nominal data of the camera; (4) searching the neighbor regions of each pixel of the two rectified images for a closed region depth edge and building a supporting window; (5) in the built window, computing a normalized cross-correlation coefficient of supported pixels and acquiring the matching price of a central pixel; (6) acquiring a parallax by using a confidence transmission optimization method having an acceleration updating system; (7) estimating an accurate parallax by a subpixel; and (8) computing the three-dimensional coordinates of an actual object point according to the matching relationship between the nominal data of the camera and the pixel and consequently reconstructing the three-dimensional point cloud of the object and reducing the three-dimensional information of a target.

In accordance with at least some embodiments of the present disclosure, a processor for performing stereo matching of a first image and a second image is described. The processor may include a first pipeline stage configured to generate data costs associated with a first tile selected from the first image, wherein the data costs is generated based on pixels in the first tile and corresponding pixels in the second image. The processor may include a second pipeline stage configured to generate disparity values associated with the first tile and an outbound message from the first tile to one of neighboring tiles in the first image, wherein the disparity values and the outbound message are generated based on the data costs and inbound messages from the neighboring tiles to the first tile. The processor may further include a third pipeline stage configured to store the disparity values and the outbound message in a memory, wherein the outbound message is used by the second pipeline stage as one of the inbound messages during processing of a second tile selected from the first image.

The invention discloses a depth camera based on structured light and stereo vision A depth camera system uses a structured light illuminator and multiple sensors such as infrared light detectors, such as in a system which tracks the Motion of a user in a field of view. One sensor can be optimized for shorter range detection while another sensor is optimized for longer range detection. The sensors can have a different baseline distance from the illuminator, as well as a different spatial resolution, exposure time and sensitivity. In one approach, depth values are obtained from each sensor by matching to the structured light pattern, and the depth values are merged to obtain a final depth map which is provided as an input to an application. The merging can involve unweighted averaging, weighted averaging, accuracy measures and / or confidence measures. In another approach, additional depth values which are included in the merging are obtained using stereoscopic matching among pixel data of the sensors.

The invention relates to a color image three-dimensional reconstruction method based on three-dimensional matching, comprising the following steps of: (1) simultaneously and respectively taking an image from proper angles by using two color cameras; (2) respectively calibrating the internal parameter matrixes and the external parameter matrixes of the two cameras; (3) carrying out polar line correction and image transformation according to calibrated data; (4) working out matching cost for each pixel point in the two corrected images by applying a self-adaption weight window algorithm and acquiring an initial parallax image; (5) marking the reliability coefficient of the pixel initial matching result by adopting matching cost reliability detection and left and right consistency verification; (6) carrying out color segmentation on the images through a Mean-Shift algorithm; (7) carrying out global optimization by a selective confidence propagation algorithm on the basis of color segmentation and pixel reliability classification results to obtain a final parallax image; and (8) working out the three-dimensional coordinates of actual object points on the images according to the calibrated data and the matching relation, thereby reconstructing the three-dimensional point cloud of an object.

A method of revising depth of a three-dimensional (3D) image is disclosed. The method comprises the following steps: firstly, at least one initial depth map associated with one image of the 3D image pair based on stereo matching technique is received, wherein the one image comprises a plurality of pixels, and the initial depth map carries an initial depth value of each pixel. Then, the inconsistence among the pixels of the one image of the 3D image pair is detected to estimate a reliable map. Finally, the initial depth value is interpolated according to the reliable map and the proximate pixels, so as to generate a revised depth map by revising the initial depth value.

The invention provides a method for barrier perception based on airborne binocular vision. The method comprises the following steps of setting a coordinate system of an airborne binocular vision camera, calculating a formula for conversion between the coordinate system and computer image coordinates of an image obtained by the airborne binocular vision camera according to the coordinate system, wherein the airborne binocular vision camera comprises a left camera and a right camera, extracting characteristic points of an image obtained by the airborne binocular vision camera, carrying out a characteristic vector description process on the characteristic points, carrying out stereo matching of left and right images according to characteristic vectors of the characteristic points to obtain preliminary matching point pairs, eliminating error matching in the preliminary matching point pairs to obtain final matching point pairs, creating a disparity map according to the final matching point pairs, and carrying out barrier perception according to the disparity map. The method for barrier perception based on airborne binocular vision has the advantages of strong adaptability, good instantaneity and good concealment performance.

A system and method for registering stereoscopic images comprising: obtaining at least two sets of stereoscopic images, each one of the at least two sets including at least two images that are taken from different angles, determining at least two groups of images, each one of the groups including at least two images that are respective images of at least two of the sets or are derived therefrom. For each one of the groups, calculating a respective optimal entities list and stereo-matching at least two images, each one being or derived from different one of the at least two groups and same or different sets, using at least four optimal entities from each one of the optimal entities list, thereby giving rise to at least one pair of registered stereoscopic images.

An apparatus and method for obtaining a three-dimensional image. A first multi-view image may be generated using patterned light of infrared light, and a second multi-view image may be generated using non-patterned light of visible light. A first depth image may be obtained from the first multi-view image, and a second depth image may be obtained from the second multi-view image. Then, stereo matching may be performed on the first depth image and the second depth image to generate a final depth image.

A change discrimination device capable of discriminating an alteration of a photographing target only from an aerial photograph or irrespectively of a difference in lighting conditions or photographing conditions at the time of taking a photo, and at minute distance intervals on a pixel basis, which receives input of a plurality of aerial image data at a new time point and an old time point, generates three-dimensional data (DSM) by subjecting the applied aerial image data to stereo-matching processing, generates ortho-image data and ortho-DSM data by normalizing the aerial image data and the generated DSM data, compares colors by using the generated ortho-image of the new time point and ortho-image of the old time point and compares heights by using the generated ortho-DSM data of the new time point and ortho-DSM data of the old time point to discriminate an alteration of a feature on the earth.

The invention discloses a three-dimensional photographing process based on a laser probe array and a device utilizing the same, and belongs to the field of three-dimensional photographing and measuring. Thousands of laser probes are projected to preset space positions according to the digital optical phase conjugation principle, reflection of the laser probes on surfaces of articles is monitored by a general two-dimensional camera, and then three-dimensional coordinate measurement is realized. Meanwhile, since the laser probe array is combined with a pair of three-dimensional cameras, more three-dimensional coordinates can be acquired by means of three-dimensional reconstruction, and accurate and compact three-dimensional coordinate measurement for large sites can be realized. Accordingly, three-dimensional matching treatment accuracy of three-dimensional images in the three-dimensional reconstruction process can be improved, matching treatment time is shortened, images picked up by the three-dimensional cameras can be compensated at the post stage, error in focal length and installation of the two cameras is eliminated, and requirements for the previous stage installation and synchronization of the focal length in the actual photographing process of the three-dimensional cameras are lowered. The three-dimensional photographing process based on the laser probe array and the device utilizing the same are applicable to fields of three-dimensional films, robots, intelligent driving, quick detection for obstacles, automatic measurement for industrial parts, and the like.

Multi view images are generated with reduced flickering. A first depth map is generated from stereo images by stereo-matching. When stereo-matching is poor or varies too much from frame to frame, disparity fallback selects a second depth map that is generated from a single view without stereo-matching, preventing stereo-matching errors from producing visible artifacts or flickering. Flat or textureless regions can use the second depth map, while regions with good stereo-matching use the first depth map. Depth maps are generated with a one-frame delay and buffered. Low-cost temporal coherence reduces costs used for stereo-matching when the pixel location selected as the lowest-cost disparity is within a distance threshold of the same pixel in a last frame. Hybrid view synthesis uses forward mapping for smaller numbers of views, and backward mapping from the forward-mapping results for larger numbers of views. Rotated masks are generated on-the-fly for backward mapping.

The invention provides a human face living detection method, device and system. The method includes the steps that a first image acquired by a main camera and a second image acquired by an auxiliary camera are acquired, when the first image comprises characteristic information of a human face image, the first image is corrected, the corrected first image and the second image are subjected to three-dimension matching, a disparity map between the human face image and the main camera is obtained, a fitting plane of key points of the human face image is obtained through fitting according to three-dimensional relation matrixes between the disparity map and the main camera and between the disparity map and the auxiliary camera, and therefore the corresponding fitting deviation can be calculated; the fitting deviation is compared with a set threshold, and therefore whether the human face image comes from a living face or not can be judged. According to the technical scheme, operation is easy, the anti-outside-interference capacity is high, precision is high, and a model training part in a machine learning method is avoided.