Multimodal Feedback Pen Shaped Interface and MR Application with Spatial Reality Display

PubDate: December 2021

Teams: Sony Interactive Entertainment

Writers: Jun Momose;Yuta Koda;Hideki Mori;Morio Kakiuchi;Kotaro Imamura;Makoto Wakabayashi

PDF: Multimodal Feedback Pen Shaped Interface and MR Application with Spatial Reality Display

Abstract

Multimodal interface is essential to enrich the reality of drawing in the virtual 3D environment. We propose the pen shaped interface capable of providing the following multimodal feedbacks: (1) Linear motion force feedback to express contact pressure of virtual object, (2) Rotational force feedback to simulate friction of rubbing virtual surface and tangential contact force with virtual object, (3) Vibrotactile feedback, and (4) auditory feedback to express contact information and texture of virtual object. We developed Mixed Reality (MR) interaction system with pen shaped interface and Spatial Reality Display. This system will display virtual pen tip extended from the actual pen shaped interface, and user can use it to draw into virtual workspace as well as interact with virtual objects. This demonstrates the advantage of our proposal by improving the reality of virtual space interaction.