ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality

Title: ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality

Teams: Microsoft

Writers: Daniel Schneider Alexander Otte Travis Gesslein Philipp Gagel Bastian Kuth Mohamad Shahm Damlakhi Oliver Dietz Eyal Ofek Michel Pahud Per Ola Kristensson Jörg Müller Jens Grubert

Publication date: October 2019

Abstract

Physical keyboards are common peripherals for personal computers and are efficient standard text entry devices. Recent research has investigated how physical keyboards can be used in immersive head-mounted display-based Virtual Reality (VR). So far, the physical layout of keyboards has typically been transplanted into VR for replicating typing experiences in a standard desktop environment. In this paper, we explore how to fully leverage the immersiveness of VR to change the input and output characteristics of physical keyboard interaction within a VR environment. This allows individual physical keys to be reconfigured to the same or different actions and visual output to be distributed in various ways across the VR representation of the keyboard. We explore a set of input and output mappings for reconfiguring the virtual presentation of physical keyboards and probe the resulting design space by specifically designing, implementing and evaluating nine VR-relevant applications: emojis, languages and special characters, application shortcuts, virtual text processing macros, a window manager, a photo browser, a whack-a-mole game, secure password entry and a virtual touch bar. We investigate the feasibility of the applications in a user study with 20 participants and find that, among other things, they are usable in VR. We discuss the limitations and possibilities of remapping the input and output characteristics of physical keyboards in VR based on empirical findings and analysis and suggest future research directions in this area.