Projection type physical interaction desktop system and method for middle school experiments

Abstract

The invention discloses a projection type physical interaction desktop system and method for middle school experiments. The system comprises a desktop; the real object which is obtained through 3D printing according to the experimental content and is placed on a desktop; the 3D somatosensory camera which is used for shooting a depth image of the real object in real time and transmitting the depthimage to the computer; the computer which is used for constructing a virtual experiment scene according to the experiment content, adjusting internal and external parameters of the virtual camera, identifying the pose of the real object in real time through the depth image, and adjusting the pose of the virtual experiment scene in real time to be registered with the real object; and the projectorwhich projects the adjusted virtual experiment scene to a desktop in real time to realize virtual-real fusion presentation. The system provided by the invention supports real-time interaction and virtual-real fusion projection display of a real object.

Description

technical field [0001] The invention relates to the technical field of space augmented reality, in particular to a middle school experiment-oriented projection type physical object interactive desktop system and method. Background technique [0002] Some microscopic or macroscopic experimental phenomena are beyond the scope of human vision and are difficult to present in experiments. Therefore, virtual reality (Virtual Reality) and augmented reality (Augmented Reality) technologies are widely used in experimental teaching. Its advantage is to display things or phenomena that are difficult to observe in a virtual environment or a real environment, making some abstract content images Visualization, the unrepresentable becomes visible. [0003] Virtual reality and augmented reality technology integrate teaching content into interactive media, and each learner will have a different learning process due to different interaction methods during the use process. Virtual reality an...

AI Technical Summary

Problems solved by technology

This method lacks the sense of operation of real experiments; the interactive experimental teaching of VR handles means that students wear virtual helmets and immerse themselves in the virtual environment to conduct experiments, and interact through handles, which is not the way of interaction in real scenes. The interaction does not coincide with the operation of the real experiment; the interactive method of relocating the interactive piece is to assign the pose of the artificial marker to the virtual object (references: AzmandianM, Hancock M, Benko H, et al.Haptic Re-targeting: Dynamic Repurposing of Passive Haptics for Enhanced Virtual Reality Experiences[C] / / the 2016 CHI Conference.ACM, 2016.), this method of experimental teaching is based on the real environment that students are in contact with, superimposing some virtual scenes, and constructing a combination of virtual and real, three-dimensional In the new immersive environment, students can achieve the fusion and interaction of real scenes and virtual objects through screen devices such as mobile phones, tablets or helmets. Although it is convenient to view digital elements through the screen, it has caused important limitations in the field of view, making Students' interactive operation and visual presentation are separated. Obviously, this method is not flexible to apply. At the same time, the possession of these devices affects the user's sense of experience and immersion to a certain extent.

Images