Monocular and IMU fused stable motion tracking method and device based on mobile terminal

Abstract

The invention discloses a monocular and IMU fused stable motion tracking method and device based on a mobile terminal, belonging to the technical field of AR / VR motion tracking. The method comprises the following steps of: judging whether the number of tracking feature points of a current frame of an image is greater than a pre-set threshold value or not, if so, performing feature point tracking by adopting an optical flow method so as to obtain the current pose of a camera, if not, obtaining feature points by adopting a FAST feature detection operator, and performing feature matching of the image by adopting a BRIEF algorithm calculation descriptor so as to obtain the current pose of the camera; performing Kalman filtering of the current pose of the camera so as to obtain a visual pose; obtaining acceleration and angular speed values generated by an IMU in a three-dimensional space, and performing integral operation of the acceleration and angular speed values so as to obtain the pose of the IMU; and performing Kalman fusion of the visual pose and the pose of the IMU, and performing motion tracking. Compared with the prior art, more stable and rapid motion tracking can be obtained on mobile terminal equipment.

Description

technical field [0001] The present invention relates to the technical field of AR / VR motion tracking, in particular to a stable motion tracking method and device based on mobile monocular and IMU fusion. Background technique [0002] Motion tracking technology is to measure, track, and record the trajectory of objects in three-dimensional space. It mainly obtains the information of the motion scene through sensor technology, and calculates the attitude of the tracked object in space in real time. It is mainly used in fields such as AR (Augmented Reality, Augmented Reality) / VR (Virtual Reality, Virtual Reality), wearable devices, robots, and autonomous driving navigation. At present, motion tracking such as mobile AR / VR mainly uses handles for interaction, and only the gyroscope of the mobile phone is used for rotation tracking during the interaction process. Since Nister first proposed the concept of visual odometry in 2004, the method based on visual odometry has become th...

AI Technical Summary

Problems solved by technology

From the perspective of the sensor itself, due to the limitation of the imaging quality and frame rate of the mobile terminal, when there are few feature points in the environmental image, it is easy to cause tracking failure; when the device moves quickly, it will cause motion blur in the image, resulting in motion tracking failure; In principle, based on monocular motion tracking, it can only incrementally represent the relative motion trend of the camera, without absolute scale information.

These two aspects have greatly restricted the current specific practical application based on the mobile terminal.

Images