Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

PubDate: Apr 2020

Teams: Ariel AI, Twitter

Writers: Dominik Kulon, Rıza Alp Güler, Iasonas Kokkinos, Michael Bronstein, Stefanos Zafeiriou

PDF: Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Project: Weakly-Supervised Mesh-Convolutional Hand Reconstruction in the Wild

Abstract

We introduce a simple and effective network architecture for monocular 3D hand pose estimation consisting of an image encoder followed by a mesh convolutional decoder that is trained through a direct 3D hand mesh reconstruction loss. We train our network by gathering a large-scale dataset of hand action in YouTube videos and use it as a source of weak supervision. Our weakly-supervised mesh convolutions-based system largely outperforms state-of-the-art methods, even halving the errors on the in the wild benchmark.