Reconstructing Part-Level 3D Models From a Single Image

PubDate: June 2020

Teams: Beihang University

Writers: Dingfeng Shi; Yifan Zhao; Jia Li

PDF: Reconstructing Part-Level 3D Models From a Single Image

Abstract

Understanding an image with 3D representations has been an increasingly attractive topic in computer vision. The state-of the-art 3D reconstruction methods usually focus on the reconstruction of the holistic object, while missing important part information, which is crucial in robotic interaction and virtual reality applications. To solve this issue, we make the first attempt to reconstruct the 3D models with part-level representations in a unified framework. With the input of the singleview images, we first develop a feature enhancement encoder to incorporate discriminative local features into the feature representation. The local features are selected adaptively by a learnable local awareness module. Then the enhanced local features are fused with the global branch to form the 3D representations. We then develop a 3D part generator to decode the image priors to 3D parts with a 3D focal loss, which enables the representations of small parts. Experimental results indicate that our model generates reliable part-level structures while achieving state-of-the-art performance in object-level recovering.