How Important are Detailed Hand Motions for Communication for a Virtual Character Through the Lens of Charades?

PubDate: Jan 2023

Teams: Clemson University;Bryn Mawr College;Meta;University of Birmingham

Writers: Alex Adkins；Aline Normoyle；Lorraine Lin；Yu Sun；Yuting Ye；Massimiliano Di Luca；Sophie Jörg

PDF: How Important are Detailed Hand Motions for Communication for a Virtual Character Through the Lens of Charades?

Abstract

Detailed hand motions play an important role in face-to-face communication to emphasize points, describe objects, clarify concepts, or replace words altogether. While shared virtual reality (VR) spaces are becoming more popular, these spaces do not, in most cases, capture and display accurate hand motions. In this paper, we investigate the consequences of such errors in hand and finger motions on comprehension, character perception, social presence, and user comfort. We conduct three perceptual experiments where participants guess words and movie titles based on motion captured movements. We introduce errors and alterations to the hand movements and apply techniques to synthesize or correct hand motions. We collect data from more than 1000 Amazon Mechanical Turk participants in two large experiments, and conduct a third experiment in VR. As results might differ depending on the virtual character used, we investigate all effects on two virtual characters of different levels of realism. We furthermore investigate the effects of clip length in our experiments.