Occlusion-aware Hand Posture Based Interaction on Tabletop Projector

Title: Occlusion-aware Hand Posture Based Interaction on Tabletop Projector

Teams: Sony

Writers: Eisuke Fujinawa；Kenji Goto；Atsushi Irie；Songtao Wu；Kuanhong Xu

Publication date: October 2019

Abstract

Conventional camera-based hand interaction technique suffered from self-occlusion among fingers, which lowers the detection accuracy of fingertip positions, leading to uncomfortable UI controls. Based on observations, self-occlusion depends on hand postures. We design an interaction framework in which interaction is decided in response to a recognized hand posture. Using a tabletop projection system that has a projector and a depth sensor, we implement the framework by integrating five touch and in-air interactions that balance its stability and usability.