Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment

Title: Room2Room: Enabling Life-Size Telepresence in a Projected Augmented Reality Environment

Teams: Microsoft

Writers: Tomislav Pejsa Julian Kantor Hrvoje Benko Eyal Ofek Andrew D. Wilson

Publication date: February 2016

Abstract

Room2Room is a telepresence system that leverages projected augmented reality to enable life-size, co-present interaction between two remote participants. Our solution recreates the experience of a face-to-face conversation by performing 3D capture of the local user with color + depth cameras and projecting their life-size virtual copy into the remote space. This creates an illusion of the remote person’s physical presence in the local space, as well as a shared understanding of verbal and non-verbal cues (e.g., gaze, pointing.) In addition to the technical details of two prototype implementations, we contribute strategies for projecting remote participants onto physically plausible locations, such that they form a natural and consistent conversational formation with the local participant. We also present observations and feedback from an evaluation with 7 pairs of participants on the usability of our solution for solving a collaborative, physical task.