Acustico: Surface Tap Detection and Localization using Wrist-based Acoustic TDOA Sensing

PubDate: October 19, 2020

Teams: Facebook Reality Labs；Dartmouth College

Writers: Jun Gong, Aakar Gupta, Hrvoje Benko

PDF: Acustico: Surface Tap Detection and Localization using Wrist-based Acoustic TDOA Sensing

Abstract

In this paper, we present Acustico, a passive acoustic sensing approach that enables tap detection and 2D tap localization on uninstrumented surfaces using a wrist-worn device. Our technique uses a novel application of acoustic time differences of arrival (TDOA) analysis. We adopt a sensor fusion approach by taking both “surface waves” (i.e., vibrations through surface) and “sound waves” (i.e., vibrations through air) into analysis to improve sensing resolution. We carefully design a sensor configuration to meet the constraints of a wristband form factor. We built a wristband prototype with four acoustic sensors, two accelerometers and two microphones. Through a 20- participant study, we evaluated the performance of our proposed sensing technique for tap detection and localization. Results show that our system reliably detects taps with an F1-score of 0.9987 across different environmental noises and yields high localization accuracies with root-mean-square-errors of 7.6mm (X-axis) and 4.6mm (Y-axis) across different surfaces and tapping techniques.