A method and a device for performing 3D fusion display on a real scene and a virtual object

Abstract

The invention discloses a method and a device for carrying out 3D fusion display on a real scene and a virtual object. The method comprises the steps: obtaining two 2D left-eye and right-eye image images, collected by a binocular camera in real time, of a real scene; Determining a first target mark position of the three-dimensional virtual object on the image picture, and obtaining a depth value of the first target mark position; Determining world space vectors of all the target mark positions of the three-dimensional virtual object on the image according to the depth value of the first targetmark position; Transforming the world space vectors of all the target mark positions according to the internal parameter matrix and the external parameter matrix of the binocular camera to obtain a target mark clipping space position vector corresponding to each target mark position; Cutting a space position vector according to the two 2D left-right eye image images and the target mark corresponding to each target mark position to obtain two 2D left-right eye virtual-real fusion images after virtual-real fusion; And performing three-dimensional display according to the two 2D left-right eye virtual-real fusion images.

Description

technical field [0001] The present invention relates to Augmented Reality (Augmented Reality, AR) and 3D display technology, in particular to a method and device for 3D fusion display of a real scene and a virtual object. Background technique [0002] In the field of augmented reality, how to more realistically integrate the real scene picture previewed by the camera with the virtual object, and make the observer feel that the fused virtual object is fused in the real scene has always been an issue. research problems in the field. [0003] Most of the current technologies use a pair of images captured by a monocular camera or two images captured by a binocular camera, and add some virtual special effects to the acquired images for virtual and real fusion, but the final effect of this synthesis is only added to the real scene. The 2D image of the virtual object cannot present the three-dimensional effect of fusion display to the observer. [0004] Another fusion display tec...

AI Technical Summary

Problems solved by technology

[0003] Most of the current technologies use a pair of images captured by a monocular camera or two images captured by a binocular camera, and add some virtual special effects to the acquired images for virtual and real fusion, but the final effect of this synthesis is only added to the real scene. The 2D image of the virtual object does not present the three-dimensional effect of fusion display to the observer

[0004] Another fusion display technology is to place the two-dimensional markers trained in advance in the real scene to be previewed, and draw the virtual object at the image position corresponding to the two-dimensional markers by detecting the posture of the two-dimensional markers. This method has great limitations, because for many applications that require high real-time performance, it is impossible to perform lengthy marker training operations in advance

Images