3D Scene Reconstruction from RGB Images Using Dynamic Graph Convolution for Augmented Reality

PubDate: April 2022

Teams: National Taiwan University

Writers: Tzu-Hsuan Weng; Robin Fischer; Li-Chen Fu

PDF: 3D Scene Reconstruction from RGB Images Using Dynamic Graph Convolution for Augmented Reality

Abstract

The 3D scene reconstruction task aims to reconstruct the object shape, object pose, and the 3D layout of the scene. In the field of augmented reality, this information is required for interactions with the surroundings. In this paper, we develop a holistic end-to-end scene reconstruction system using only RGB images. We further designed an architecture that can adapt to different types of objects through our graph convolution network during object surface generation. Moreover, a scene-merging strategy is proposed to alleviate the occlusion problem by merging different views continuously. This also allows our system to reconstruct the complete surroundings in a room.