FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization

Title: FlashBack: Immersive Virtual Reality on Mobile Devices via Rendering Memoization

Teams: Microsoft

Writers: Kevin Boos David Chu Eduardo Cuervo

Publication date: June 2016

Abstract

Virtual reality head-mounted displays (VR HMDs) are attracting users with the promise of full sensory immersion in virtual environments. Creating the illusion of immersion for a near-eye display results in very heavy rendering workloads: low latency, high framerate, and high visual quality are all needed. Tethered VR setups in which the HMD is bound to a powerful gaming desktop limit mobility and exploration, and are difficult to deploy widely. Products such as Google Cardboard and Samsung Gear VR purport to offer any user a mobile VR experience, but their GPUs are too power-constrained to produce an acceptable framerate and latency, even for scenes of modest visual quality.

We present FlashBack, an unorthodox design point for HMD VR that eschews all real-time scene rendering. Instead, FlashBack aggressively precomputes and caches all possible images that a VR user might encounter. FlashBack memoizes costly rendering effort in an offline step to build a cache full of panoramic images. During runtime, FlashBack constructs and maintains a hierarchical storage cache index to quickly lookup images that the user should be seeing. On a cache miss, FlashBack uses fast approximations of the correct image while concurrently fetching more closely-matching entries from its cache for future requests. Moreover, FlashBack not only works for static scenes, but also for dynamic scenes with moving and animated objects.

We evaluate a prototype implementation of FlashBack and report up to a 8× improvement in framerate, 97x reduction in energy consumption per frame, and 15× latency reduction compared to a locally-rendered mobile VR setup. In some cases, FlashBack even delivers better framerates and responsiveness than a tethered HMD configuration on graphically complex scenes