Creating and Augmenting Keyboards for Extended Reality with the Keyboard Augmentation Toolkit

PubDate: April 2022

Teams: University of Glasgow

Writers: Mark McGill;Stephen Brewster;Daniel Pires De Sa Medeiros;Sidney Bovet;Mario Gutierrez;Aidan Kehoe

PDF: Creating and Augmenting Keyboards for Extended Reality with the Keyboard Augmentation Toolkit

Abstract

This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display—2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity—supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts—allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.