Local Free-View Neural 3D Head Synthesis for Virtual Group Meetings

PubDate: April 2022

Teams: Universität Hamburg

Writers: Sebastian Rings; Frank Steinicke

PDF: Local Free-View Neural 3D Head Synthesis for Virtual Group Meetings

Abstract

Virtual group meetings provide enormous potential for remote group communication. However, today’s video conferences incur numer-ous challenges compared to face-to-face meetings. For instance, perception of correct gaze, deictic relations, or eye-to-eye contact is impeded due to the fact that the camera is offset from the eyes of the other users’ avatars and that the gallery view is different for each group member. In this paper, we describe how 3D neural heads can be synthesized to overcome these limitations. Therefore, we generate different head poses using a generative adversarial network for a given source image frame using state-of-the-art technology. These head poses can then be viewed in a local space to freely control the gaze of the head poses. We introduce and discuss five use cases for these synthesized head poses that aim to improve intelligent agents and virtual avatar representations in regular video group meetings.