Method and apparatus for device orientation tracking using a visual gyroscope

Abstract

A method for tracking device orientation on a portable device is disclosed. The method comprises initializing a device orientation to a sensor orientation, wherein the sensor orientation is based on information from an inertial measurement unit (IMU) sensor. It also comprises initiating visual tracking using a camera on the portable device and capturing a frame. Next, it comprises determining a plurality of visual features in the frame and matching the frame to a keyframe, wherein capture of the keyframe precedes capture of the frame. Subsequently, it comprises computing a rotation amount between the frame and the keyframe. Responsive to a determination that a rotational distance between the frame and the keyframe exceeds a predetermined threshold, promoting the frame to a keyframe status and adding it to a first orientation map and adjusting the frame with all prior captured keyframes.

Description

CROSS-REFERENCE TO RELATED APPLICATIONSRelated Applications[0001]The present application is related to U.S. patent application Ser. No. ______, filed ______, entitled “A METHOD AND APPARATUS FOR LONG TERM IMAGE EXPOSURE WITH IMAGE STABILIZATION ON A MOBILE DEVICE,” naming Syed Zahir Bokari, Josh Abbott, and Jim van Welzen as inventors, and having attorney docket number NVID-PDU-13-0254-US1. That application is incorporated herein by reference in its entirety and for all purposes.FIELD OF THE INVENTION[0002]Embodiments according to the present invention generally relate to augmented reality systems and more specifically to device orientation tracking for augmented reality systems.BACKGROUND OF THE INVENTION[0003]While augmented reality technologies have improved considerably in recent years, the improvements typically come at the cost of computationally intensive procedures implemented on expensive hardware. The high cost of implementing effective augmented reality systems is a barri...

AI Technical Summary

Benefits of technology

The patent text describes a system and method for tracking the orientation of a mobile device using a camera and without needing to create a map of the environment or calculate the device's position. This approach is more efficient and faster, and avoids common issues with other methods like noise and drift error. The system uses markers to improve accuracy, but can also use IMU sensors as a fallback option if vision-based tracking fails. Overall, this technology allows for more precise and efficient orientation tracking on mobile devices.

Problems solved by technology

While augmented reality technologies have improved considerably in recent years, the improvements typically come at the cost of computationally intensive procedures implemented on expensive hardware.

The high cost of implementing effective augmented reality systems is a barrier to entry that prevents casual users from having access to such systems on everyday devices that have relatively low processing capability, e.g., tablet computers, phones and other hand-held devices.

Procedures implemented in conventional augmented reality applications for tracking device orientation with expected levels of robustness, speed and precision are computationally expensive.

Accordingly, they are not ideal for handheld mobile devices.

Building a 3D map of the environment can be significantly intensive from a computation standpoint, in part, because it involves tracking the device's position as well as orientation.

Because of the considerable computational requirements for implementing SLAM, it is not a suitable procedure for implementation on a mobile device, such as a smart phone.

Other conventional augmented reality systems are unsuitable for determining device orientation on mobile devices because they suffer from problems such as drift error.

A major disadvantage of using IMUs is that they typically suffer from accumulated error.

Because the guidance system is continually adding detected changes to its previously-calculated positions, any errors in measurement, however small, are accumulated from point to point.

This leads to drift error, or an ever-increasing difference between where the system thinks it is located and the actual location, where, as stated above, location includes both the device's position and also its orientation.

Stated differently, drift is a problem because integration of the orientation tracking's relative measurements accumulates the small errors in each measurement.

This can, consequently, create significant differences between the estimated and actual orientation.

Further, measurements from IMU sensors tend to be error prone and noisy.

Thus, IMU sensors are typically not suitable for an immersive augmented reality environment.

Conventional methods of tracking device orientation for augmented reality systems, therefore, are unsuitable for use on mobile devices because they are either too expensive and computationally intensive or simply unsuitable as a result of problems such as noise and drift error.

Images