Touch with Foreign Hands: The Effect of Virtual Hand Appearance on Visual-Haptic Integration

Title: Touch with Foreign Hands: The Effect of Virtual Hand Appearance on Visual-Haptic Integration

Teams: Facebook

Writers: Valentin Schwind, Lorraine Lin, Massimiliano Di Luca, Sophie Jörg, James Hillis

Publication date: May 10, 2019

Abstract

Hand tracking and haptics are gaining more importance as key technologies of virtual reality (VR) systems. For designing such systems, it is fundamental to understand how the appearance of the virtual hands influences user experience and how the human brain integrates vision and haptics. However, it is currently unknown whether multi-sensory integration of visual and haptic feedback can be influenced by the appearance of virtual hands in VR. We performed a user study in VR to gain insight into the effect of hand appearance on how the brain combines visual and haptic signals using a cue-conflict paradigm. In this paper, we show that the detection of surface irregularities (bumps and holes) sensed by eyes and hand is affected by the rendering of avatar hands. However, sensitivity changes do not correlate with the degree of perceived limb ownership. Qualitative feedback provides insights into potentially distracting cues in visual-haptic integration.