FarfetchFusion: Towards Fully Mobile Live 3D Telepresence Platform

In remote rendered virtual reality (VR), the rendering of the application is moved to the cloud enabling high quality real-time content to be consumed on low-powered standalone head mounted displays (HMDs). The rendered frames are encoded to a video stream and streamed to a thin client which relays user’s input to the server and decodes and displays the incoming video. Latency and high bandwidth requirements are key challenges for remote rendered graphics. Foveation can be used to optimize the quality of the transmitted frames to be in line with the human visual system (HVS). In this paper we evaluate multiple different strategies on how to apply foveation to spatially compress video frames, i.e., reduce their resolution, before transmission. We also show how the foveation methods can be used together with super resolution to alleviate the bandwidth usage of real-time remote rendered VR and optimize the perceived image quality.