EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

PubDate: June 2020

Teams: Tsinghua University , Max Planck Institute for Informatics, Hong Kong University of Science and Technology

Writers: Lan XU, Weipeng Xu, Vladislav Golyanik, Marc Habermann, Lu Fang, Christian Theobalt

PDF: EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

Project: EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera

Abstract

The high frame rate is a critical requirement for capturing fast human motions. In this setting, existing markerless image-based methods are constrained by the lighting requirement, the high data bandwidth and the consequent high computation overhead. In this paper, we propose EventCap — the first approach for 3D capturing of high-speed human motions using a single event camera. Our method combines model-based optimization and CNN-based human pose detection to capture high frequency motion details and to reduce the drifting in the tracking. As a result, we can capture fast motions at millisecond resolution with significantly higher data efficiency than using high frame rate videos. Experiments on our new event-based fast human motion dataset demonstrate the effectiveness and accuracy of our method, as well as its robustness to challenging lighting conditions.