Low-latency virtual reality display system

Abstract

A low-latency virtual reality display system that provides rapid updates to virtual reality displays in response to a user's movements. Display images generated by rendering a 3D virtual world are modified using an image warping approximation, which allows updates to be calculated and displayed quickly. Image warping is performed using various rerendering approximations that use simplified 3D models and simplified 3D to 2D projections. An example of such a rerendering approximation is a shifting of all pixels in the display by a pixel translation vector that is calculated based on the user's movements; pixel translation may be done very quickly, possibly using hardware acceleration, and may be sufficiently realistic particularly for small changes in a user's position and orientation. Additional features include techniques to fill holes in displays generated by image warping, synchronizing full rendering with approximate rerendering, and using prediction of a user's future movements to reduce apparent latency.

Description

technical field [0001] One or more embodiments of the invention relate to the field of virtual reality systems. More specifically, but not limitation, one or more embodiments of the invention implement a low-latency virtual reality display system that tracks a user's movements and rapidly renders a virtual reality display based on those movements. Background technique [0002] Virtual reality systems are well known in the art. Such systems generate a virtual world for a user that responds to the user's movements. Examples include various types of VR headsets and goggles worn by users, and dedicated rooms with multiple monitors. Virtual reality systems typically include sensors that track the user's head, glasses, or other body parts and modify the virtual world based on the user's movements. Virtual worlds consist of three-dimensional models, either computer-generated or captured from real-world scenes. An image of the 3D model is generated based on the user's position a...

AI Technical Summary

Problems solved by technology

3D rendering is complex and processor intensive operation which can potentially take hundreds of milliseconds

The result is a significant delay between when the user perceives their motion and when the updated virtual environment is rendered on their display

Three technology trends are exacerbating this challenge: (1) As more 3D data is captured and generated, the complexity of 3D models is increasing

(2) The resolution of virtual reality displays is increasing, requiring more computing power to render images

As a result of these trends, high latency in rendering VR displays has become a major factor limiting the adoption and application of VR technology

Given these technical limitations, no known system provides sufficiently low-latency rendering and display to produce a highly responsive virtual reality environment

Images