Context-Consistent Generation of Indoor Virtual Environments based on Geometry Constraints

PubDate: September 2021

Teams: Tsinghua University；Tianjin University of technology；Cardiff University

Writers: Yu He; Yingtian Liu; Yihan Jin; Song-Hai Zhang; Yu-Kun Lai; Shi-Min Hu

PDF: Context-Consistent Generation of Indoor Virtual Environments based on Geometry Constraints

Abstract

In this paper, we propose a system that can automatically generate immersive and interactive virtual reality (VR) scenes by taking real-world geometric constraints into account. Our system can not only help users avoid real-world obstacles in virtual reality experiences, but also provide context-consistent contents to preserve their sense of presence. To do so, our system first identifies the positions and bounding boxes of scene objects as well as a set of interactive planes from 3D scans. Then context-compatible virtual objects that have similar geometric properties to the real ones can be automatically selected and placed into the virtual scene, based on learned object association relations and layout patterns from large amounts of indoor scene configurations. We regard virtual object replacement as a combinatorial optimization problem, considering both geometric and contextual consistency constraints. Quantitative and qualitative results show that our system can generate plausible interactive virtual scenes that highly resemble real environments, and have the ability to keep the sense of presence for users in their VR experiences.