Project MultiLeap: Making Multiple Hand Tracking Sensors to Act Like One

PubDate: December 2021

Teams: Czech Technical University in Prague

Writers: Tomáš Nováček; Marcel Jiřina

PDF: Project MultiLeap: Making Multiple Hand Tracking Sensors to Act Like One

Abstract

We present a concept that provides hand tracking for virtual and extended reality, only with the use of optical sensors, without the need for the user to hold any physical controller. In this article, we propose five new algorithms further to extend our previous research and the possibilities of the hand tracking system whilst also making it more precise. The first algorithm deals with the need to calibrate the tracking system. Thanks to the new approach, we improved tracking precision by 37% over our previous solution. The second algorithm allows us to compute the precision of the hand tracking data when multiple sensors are used. The third algorithm further improves the computation of hand tracking data confidence by correctly handling the edge cases, for example, when the tracked hand is at the edge of the sensor’s field of view. The fourth algorithm provides a new way to fuse the hand tracking data by using only the hand tracking data with the highest hand tracking data confidence. The fifth algorithm deals with the issue when the optical sensor misclassifies the hand chirality.