XRTB: A Cross Reality Teleconference Bridge to incorporate 3D interactivity to 2D Teleconferencing

PubDate: May 2021

Teams: University of Auckland；Informatics University of California Irvine

Writers: Prasanth Sasikumar；Max Collins；Huidong Bai；Mark Billinghurst

PDF: XRTB: A Cross Reality Teleconference Bridge to incorporate 3D interactivity to 2D Teleconferencing

Abstract

We present XRTeleBridge (XRTB), an application that integrates a Mixed Reality (MR) interface into existing teleconferencing solutions like Zoom. Unlike conventional webcam, XRTB provides a window into the virtual world to demonstrate and visualize content. Participants can join via webcam or via head mounted display (HMD) in a Virtual Reality (VR) environment. It enables users to embody 3D avatars with natural gestures and eye gaze. A camera in the virtual environment operates as a video feed to the teleconferencing software. An interface resembling a tablet mirrors the teleconferencing window inside the virtual environment, thus enabling the participant in the VR environment to see the webcam participants in real-time. This allows the presenter to view and interact with other participants seamlessly. To demonstrate the system’s functionalities, we created a virtual chemistry lab environment and presented an example lesson using the virtual space and virtual objects and effects.