A Virtual Mouse Interface for Supporting Multi-user Interactions

PubDate: July 2020

Teams: Rensselaer Polytechnic Institute;IBM Thomas J Watson Research Center

Writers: Matthew PevelerJeffery O. KephartXiangyang MouGordon ClementHui Su

PDF: A Virtual Mouse Interface for Supporting Multi-user Interactions

Abstract

Traditionally, two approaches have been used to build intelligent room applications. Mouse-based control schemes allow developers to leverage a wealth of existing user-interaction libraries that respond to clicks and other events. However, systems built in this manner cannot distinguish among multiple users. To realize the potential of intelligent rooms to support multi-user interactions, a second approach is often used, whereby applications are custom-built for this purpose, which is costly to create and maintain. We introduce a new framework that supports building multi-user intelligent room applications in a much more general and portable way, using a combination of existing web technologies that we have extended to better enable simultaneous interactions among multiple users, plus speech recognition and voice synthesis technologies that support multi-modal interactions.