A Web-based Real-Time Kinect Application for Gestural Interaction with Virtual Musical Instruments

PubDate: September 2018

Teams: National Technical Univ. of Athens，Inst. for Language and Speech Proc.

Writers: Athanasia Zlatintsi；Panagiotis P. Filntisis；Christos Garoufis；Antigoni Tsiami；Kosmas Kritsis；M. A. Kaliakatsos-Papakostas；Aggelos Gkiokas；Vassilis Katsouros；Petros Maragos

PDF: A Web-based Real-Time Kinect Application for Gestural Interaction with Virtual Musical Instruments

Abstract

We present a web-based real-time application that enables gestural interaction with virtual instruments for musical expression. Skeletons of the users are tracked by a Kinect sensor, while the performance of the virtual instruments is accomplished using gestures inspired from their corresponding physical counterparts. The application supports the virtual performance of an air guitar and an upright bass, as well as a more abstract conductor-like performance with two instruments, while collaborative playing of two or more players is also allowed. The multimodal virtual interface of our application, which includes 3D avatars, allows users, even if not musically educated, to engage in innovative interactive musical activities, while its web-based architecture improves its accessibility and performance. The application was qualitatively evaluated by 13 users, in terms of its usability and enjoyability, among others, accomplishing high ratings and positive feedback.