Demonstrating Adaptive Many-to-Many Immersive Teleconferencing for Volumetric Video

Date：May 2024

Teams:Ghent University - imec

Writers:Matthias De Fré,Jeroen van der Hooft,Tim Wauters,Filip De Turck

PDF:Demonstrating Adaptive Many-to-Many Immersive Teleconferencing for Volumetric Video

Abstract

In today's world, the use of video conferencing applications has risen significantly. However, with the introduction of affordable head-mounted displays (HMDs), users are now seeking new immersive and engaging experiences that enhance the 2D video conferencing applications with a third dimension. Immersive video formats such as light fields and volumetric video aim to enhance the experience by allowing for six degrees-of-freedom (6DoF), resulting in users being able to look and walk around in the virtual space. We present a novel, open source, many-to-many streaming architecture using point cloud-based volumetric video. To ensure bitrates that satisfy contemporary networks, the Draco codec encodes the point clouds before they are transmitted using web real-time communication (WebRTC), all while ensuring that the end-to-end latency remains acceptable for real-time communication. A multiple description coding (MDC)-based quality adaptation approach ensures that the pipeline can support a large number of users, each with varying network conditions.