52 results about "2D to 3D conversion" patented technology

A system and method is provided for model fitting and registration of objects for 2D-to-3D conversion of images to create stereoscopic images. The system and method of the present disclosure provides for acquiring at least one two-dimensional (2D) image, identifying at least one object of the at least one 2D image, selecting at least one 3D model from a plurality of predetermined 3D models, the selected 3D model relating to the identified at least one object, registering the selected 3D model to the identified at least one object, and creating a complementary image by projecting the selected 3D model onto an image plane different than the image plane of the at least one 2D image. The registering process can be implemented using geometric approaches or photometric approaches.

In general, a “Stereoscopic Video Converter” (SVC) provides various techniques for automatically converting arbitrary 2D video sequences into perceptually plausible stereoscopic or “3D” versions while optionally generating dense depth maps for every frame of the video sequence. In particular, the automated 2D-to-3D conversion process first automatically estimates scene depth for each frame of an input video sequence via a label transfer process that matches features extracted from those frames with features from a database of images and videos having known ground truth depths. The estimated depth distributions for all image frames of the input video sequence are then used by the SVC for automatically generating a “right view” of a corresponding stereoscopic image for each frame (assuming that each original input frame represents the “left view” of the stereoscopic image).

A system and method for region classification of two-dimensional images for 2D-to-3D conversion of images to create stereoscopic images are provided. The system and method of the present disclosure provides for acquiring a two-dimensional image, identifying a region of the 2D image, extracting features from the region, classifying the extracted features of the region, selecting a conversion mode based on the classification of the identified region, converting the region into a 3D model based on the selected conversion mode, and creating a complementary image by projecting the 3D model onto an image plane different than an image plane of the 2D image. A learning component optimizes the classification parameters to achieve minimum classification error of the region using a set of training images and corresponding user annotations.

Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

A method and apparatus for converting a two dimensional image or video to three dimensional image or video. The method includes segmenting objects in at least one of a two dimensional image and video, performing, in the digital processor, a depth map generation based on low-level features, performing face detection and scene classification on at least one of a two dimensional image and video, and utilizing the face detection and scene classification in enhancing the depth map and for converting the at least one of a two dimensional image and video to three dimensional image and video.

An apparatus for 2D to 3D conversion using a panorama image includes an image receiving unit for receiving and storing an input image, a user interface for receiving an input of a user who performs a 3D conversion work, a panorama image generating unit for extracting feature points of a plurality of images, a depth setting unit for recording scribbles including depth information in at least one of a plurality of pixels of the panorama image in response to the input of the user received through the user interface, a depth information propagating unit for calculating depth values of other pixels, a depth information remapping unit for mapping a depth value with respect to each of the plurality of images, and a stereo image generating unit for generating a stereo image pair for each of the plurality of images.

Systems and methods for generating a stereoscopic pair of images from a monoscopic image input are described. At least one brushstroke input corresponding to a location in the monoscopic image is received. A saliency map and edge map of the monoscopic image are computed. A first image warp and a second image warp are computed using the at least one brushstroke, the saliency map, and the edge map. A stereoscopic pair of images are generated from the first image warp and the second image warp.

Machine learning method that learns to convert 2D video to 3D video from a set of training examples. Uses machine learning to perform any or all of the 2D to 3D conversion steps of identifying and locating objects, masking objects, modeling object depth, generating stereoscopic image pairs, and filling gaps created by pixel displacement for depth effects. Training examples comprise inputs and outputs for the conversion steps. The machine learning system generates transformation functions that generate the outputs from the inputs; these functions may then be used on new 2D videos to automate or semi-automate the conversion process. Operator input may be used to augment the results of the machine learning system. Illustrative representations for conversion data in the training examples include object tags to identify objects and locate their features, Bézier curves to mask object regions, and point clouds or geometric shapes to model object depth.

The invention discloses a dynamic grating for 2D-to-3D (two-dimension to three-dimension) conversion and a control method of the dynamic grating. The dynamic grating comprises a first control layer, a second control layer, a grating material and a control module. The first control layer comprises a first transparent substrate and a first transparent electrode. The second control layer comprises a second transparent substrate and a second transparent electrode. The second transparent electrode is a planar electrode and is arranged on the upper surface of the second transparent substrate. The grating material is arranged between the first control layer and the second control layer. The control module controls voltage between the first transparent electrode and the second transparent electrode and is used for adjusting the dynamic grating to be in a transparent state to realize a 2D display effect or adjusting the dynamic grating to have a nontransparent area and become a light and shade alternate grating to realize a 3D display effect. The dynamic grating for 2D-to-3D conversion has the advantages that the structure is simple, the cost is low, the 2D display and the 3D display can be simultaneously realized in different areas on a screen, the conversion switching between 2D and 3D is easy to realize and the using effect of display devices is improved.

The invention belongs to the technical field of computer vision and relates to a 2D to 3D conversion method, in particular to a single image-based 2D to 3D conversion method. The method comprises the following four steps: subjecting an image to be processed to grayscale preprocessing; calculating a two-dimensional Gaussian convolution mask according to a two-dimensional Gaussian filtration formula; performing the Gaussian smoothing filtration of the grayscale image by using the obtained two-dimensional gauss convolution mask to reduce the noises in the image; and performing the Laplace filtration of the image subjected to the two-dimensional Gaussian filtration. The method has the advantages of processing by starting from the image, avoiding the complex input of multiple images or multi-view stereo images, having high applicability, not requiring camera parameters, avoiding complex calibration, correction and other operation, ensuring automatic generation in a whole process, requiring a small amount of calculation, and having convenient application on occasions when high speed is required.

In general, a “Stereoscopic Video Converter” (SVC) provides various techniques for automatically converting arbitrary 2D video sequences into perceptually plausible stereoscopic or “3D” versions while optionally generating dense depth maps for every frame of the video sequence. In particular, the automated 2D-to-3D conversion process first automatically estimates scene depth for each frame of an input video sequence via a label transfer process that matches features extracted from those frames with features from a database of images and videos having known ground truth depths. The estimated depth distributions for all image frames of the input video sequence are then used by the SVC for automatically generating a“right view” of a corresponding stereoscopic image for each frame (assuming that each original input frame represents the “left view” of the stereoscopic image).

A multi-view image generating method adapted to a 2D-to-3D conversion apparatus is provided. The multi-view image generating method includes the following steps. A pair of images is received. The pair of images is captured from different angles by a single image capturing apparatus rotating a rotation angle. A disparity map is generated based on one of the pair of images. A remapped disparity map is generated based on the disparity map by using a non-constant function. A depth map is generated based on the remapped disparity map. Multi-view images are generated based on the one of the pair of images and the depth map. Furthermore, a multi-view image generating apparatus adapted to the 2D-to-3D conversion apparatus is also provided.

Mathematical relationships between the scene geometry, camera parameters, and viewing environment are used to control stereography to obtain various results influencing the viewer's perception of 3D imagery. The methods may include setting a horizontal shift, convergence distance, and camera interaxial parameter to achieve various effects. The methods may be implemented in a computer-implemented tool for interactively modifying scene parameters during a 2D-to-3D conversion process, which may then trigger the re-rendering of the 3D content on the fly.

A 2D-to-3D delay compensation method for making a pair of shutter glasses be synchronization with a display device's playback of a video source is disclosed. The display device is coupled with a 2D-to-3D conversion box. The method includes the following steps: firstly, the 2D-to-3D conversion box sends out a calibration pattern to the display device. Then, a shutter control signal is sent out after waiting a delay time to switch on and off the left and right lenses of the pair of shutter glasses. Subsequently, the calibration pattern is detected. Finally, determine whether the delay time must be adjusted or not according to the detecting result.

A depth fusion method adapted for a 2D-to-3D conversion image processing apparatus is provided. The depth fusion method includes the following steps. Respective motion-based depths of a plurality of blocks in an image frame are obtained. An original image-based depth of each of the blocks is obtained. The original image-based depth of each of the blocks is converted to obtain a converted image-based depth of each of the blocks. The motion-based depth and the converted image-based depth of each of the blocks are fused block by block to obtain a fusion depth of each of the blocks. Furthermore, a depth fusion apparatus is also provided.

A semi-automatic 2D to 3D method with fault tolerance comprises the following steps: key frame extraction, user annotation, annotation extraction and depth conversion, super pixel segmentation, planefitting, representative pixel extraction, structural difference description, fault tolerance mechanism, energy function construction with fault-tolerant structure correlation, generalized iterative re-weighted least squares method and depth map-based drawing. The method has the following beneficial effects: a deep propagation energy model related to a fault-tolerant structure is established by means of robust estimation, and local and global constraint relationships are combined, so that the prior knowledge of a scenario can be utilized more effectively, and the quality of the estimated depthmap is enhanced while the user annotation difficulty in semi-automatic 2D to 3D conversion is reduced.

A three dimensional system including additional 2 dimensional to three dimensional conversion.

A method and apparatus for converting a two dimensional image or video to three dimensional image or video. The method includes segmenting objects in at least one of a two dimensional image and video, performing, in the digital processor, a depth map generation based on low-level features, performing face detection and scene classification on at least one of a two dimensional image and video, and utilizing face detection and scene classification in enhancing the depth map and for converting the at least one of a two dimensional image and video to three dimensional image and video.

A device and method are provided for two dimension (2D) to three dimension (3D) conversion. A 2D to 3D conversion device receives a 2D image data. The 2D to 3D conversion device assigns position data of a predetermined window. The 2D to 3D conversion device generates a depth map including a depth data of the 2D image data according to the 2D image data and the position data of the predetermined window. The 2D to 3D conversion device converts the 2D image data into a 3D image data according to the depth data of the depth map and the position data of the predetermined window.

Embodiments of the invention disclose a method and a device for carrying out 2D to 3D conversion on an image. The 2D to 3D conversion efficiency and effect of the image can be improved. The method comprises the steps of S1, acquiring a to-be-processed 2D image, inputting the to-be-processed 2D image into a pre-built and trained parallax information extraction model to obtain a parallax informationimage; and S2, carrying out three-dimensional rendering on the to-be-processed 2D image in combination with the parallax information image, and performing three-dimensional reconstruction on the to-be-processed 2D image.

In the 2D-to-3D conversion method presented, a series of fast-executing 2D image- or pixel-processing operations are used to transform flat 2D RGB images and video into Red-Cyan Stereoscopic 3D Anaglyph images and video. The output of this process is viewable in stereoscopic 3D on any color screen or display device, using passive red-cyan lensed Anaglyph 3D glasses. The method presented is designed to be deployed as both a hardware “2D-to-3D conversion chip”, and as a short, fast, highly parallelizable realtime algorithm running on programmable graphics hardware like a GPU. The method presented upgrades 2D visual content to stereoscopic 3D, improves the lighting, contrast and color of the content, and also increases its optical sharpness by as much as 200%. Both the left-eye and right-eye image created by the method are “synthetic”—digitally generated. The method produces fairly accurate colors, despite 3D glasses with colored lenses being used.

A method, system and medium for improving the quality of 2D-to-3D automatic conversion in a computing device by using machine learning techniques is provided, comprising: receiving new 2D image from user input; recognizing multiple objects in the new 2D image by machine learning algorithm; obtaining the hierarchical relations among objects according to the features of the said objects; improving the quality of 3D image according to the hierarchical relations. The implementation of this invention provides a more distinct classification of objects in a 2D image, thereby improving the 2D-to-3D conversion in the computing device.

The present invention relates to the field of 2D to 3D conversion, in particular discloses a method of converting 2D into 3D based on image motion information. The method comprises: S1, obtaining a depth value of each pixel of the input 2D image based on a method of motion estimation; S2, accumulating the depth value of each pixel in accordance with a luminance value of each pixel to obtain a depth image of the input 2D image; S3, reconstructing a left eye and / or a right eye image based on a reconstruction of depth image in accordance with the depth image obtained in the step of S2; S4, combining the left eye image and the right eye image obtained in the step of S4 and outputting a combined image to obtain the 3D image. In the method herein, due to the accumulation process of the depth value obtained by the motion estimation, the resulted depth image is continuous and dense, which improves the quality of the reconstructed image and the 3D visual effect.

The invention relates to a method for converting 2D (two-dimensional) video signals to 3D (three-dimensional) video signals, which is characterized in that the method comprises the steps of (1) dividing input 2D video signals into a frame of picture according to a frame and cutting the frame of picture into two equal parts left and right in a vertical direction; (2) adding N data into data in the left part of the frame or reducing N data from the data in the left part of the frame according to intervals, wherein the N is more than or equal to 2 and is less than or equal to 20; (3) reducing N data from data in the right part of the frame or adding N data into the data in the right part of the frame according to intervals, in a way of corresponding to the treatment of the date in the left part, wherein the N is more than or equal to 2 and is less than or equal to 20; (4) employing the frames treated by the step 3 or the step 4 as left eye data or right eye data of output 3D videos; (5) outputting the input 2D video signals as right eye data or left eye data, so that the conversion from 2D video signals to 3D video signals can be realized; and (6) receiving the next frame of data and repeating the step 1 to step 5. The method provided by the invention takes the middle of an image as the center, and spreads / contracts towards two sides, so as to generate left / right eye view and then combine the left eye view and the right eyes view, so that a 3D effect is achieved.

Method and apparatus for encoding, decoding and rendering 3D media content are provided. An apparatus for rendering three-dimensional (3D) media content includes a communication interface configured to receive a multimedia stream, and one or more processors operably coupled to the communication interface, the one or more processors configured to parse the multimedia stream into 2D video bitstreamsincluding geometry frames and texture frames, 2D to 3D conversion metadata for rendering 3D points from 2D frames, and scene description metadata describing 6 degree of freedom (6DoF) relationships among objects in a 6DoF scene, decode the 2D video streams including geometry data and texture data to generate 2D pixel data, covert the 2D pixel data into 3D voxel data using the 2D to 3D conversionmetadata; and generate the 6DoF scene from 3D voxel data using the scene description metadata.

A method (300) and apparatus (400) for three-dimensional television (3DTV) image adjustment includes loading (342, 344) default 2D-to-3D image setting values from a default settings memory to a user adjustment settings memory, annunciating (346) the default 2D-to-3D image setting values, receiving (361, 362) a 2D-to-3D image settings value adjustment, saving (370) the 2D-to-3D image settings value adjustment in the user adjustment settings memory, and applying (390) the 2D-to-3D image settings value adjustment to a 2D-to-3D converted image. These methods and apparatuses allow individual users to set 3DTV image settings to their personal preferences to compensate for brightness reductions caused by 3DTV glasses, depth perception sensitivities, and other image quality factors.

The invention relates to a motion-vector extraction method easy for hardware implementation in a 2D-to-3D conversion technology. The motion-vector extraction method comprises the following steps of: extracting motion vectors from image information of two continuous frames, and generating a vector diagram; when the images of the two frames are matched, carrying out motion matching on points on the edges of objects in the images firstly; then blocking the images by image layering and connected domain labeling, wherein the same block is considered to have the same motion vector; and obtaining a final motion vector diagram by combination of a blocking result and the previsou edge motion vector. The motion-vector extraction method has the advantages that on the basis of obtaining an accurate motion vector detection result, not only can the operation time be saved, but also the operation amount is greatly reduced compared with that of the method for direct full-image matching, and the implementation on hardware is facilitated.