Virtual Agents from 360° Video for Interactive Virtual Reality

PubDate: July 2019

Teams: MINES ParisTech

Writers: Grégoire Dupont de Dinechin；Alexis Paljic

PDF: Virtual Agents from 360° Video for Interactive Virtual Reality

Abstract

Creating lifelike virtual humans for interactive virtual reality is a difficult task. Most current solutions rely either on crafting synthetic character models and animations, or on capturing real people with complex camera setups. As an alternative, we propose leveraging efficient learning-based models for human mesh estimation, and applying them to the popular form of immersive content that is 360° video. We demonstrate an implementation of this approach using available pre-trained models, and present user study results that show that the virtual agents generated with this method can be made more compelling by the use of idle animations and reactive verbal and gaze behavior.