Mobile Virtual Reality for Head-mounted Displays With Interactive Streaming Video and Likelihood-based Foveation
Teams: Microsoft
Writers: Eduardo Cuervo
Publication date: June 2016
Abstract
Immersive virtual reality has long been an aspiration of many. For a truly immersive VR experience, three properties are essential: quality, responsiveness and mobility. By quality, we mean that realistic and life-like visual portrayals in a virtual environment heighten our sense of immersion. By responsiveness, we mean that any motion, especially of the head, must be reflected as quickly as possible in visual feedback because ocular proprioception sensitivity is very high. By mobility, we mean that we ought to be able to move untethered in physical space, free to explore our virtual world. Unfortunately mobile devices like phones are tablets are power-constrained and cannot produce acceptable framerate. While cloud offloading allows them to deliver high-quality and framerate it places high bandwidth requirements and introduces an unacceptable amount of latency.
This demo introduces Matia (formerly Irides), a stereo HMD system that simultaneously attains quality, responsiveness and mobility. Matia achieves this by offloading rendering work to a high-end GPU across the WAN. The HMD client is mobile and only requires a low-end GPU, yet receives high quality imagery. To overcome WAN latencies, Matia borrows speculative execution techniques inspired by previous work Outatime [1]. However, speculative execution alone is insufficient. This is because HMD sensitivities are substantially more stringent along several key dimensions: responsiveness and quality.These factors suggest that we ought to greatly benefit from maximally utilizing all available bandwidth between client and server to deliver the highest resolution imagery possible. At the same time, any latency or bandwidth changes must be handled very responsively.
