Computational Glasses: Vision Augmentations Using Computational Near-Eye Optics and Displays

PubDate: May 2020

Teams: OPPO US Research Center

Writers: Celong Liu; Zhong Li; Shuxue Quan; Yi Xu

PDF: Computational Glasses: Vision Augmentations Using Computational Near-Eye Optics and Displays

Abstract

In this paper, we present a method that estimates the real-world lighting condition from a single RGB image of an indoor scene, with information of support plane provided by commercial Augmented Reality (AR) frameworks (e.g., ARCore, ARKit, etc.). First, a Deep Neural Network (DNN) is used to segment the foreground. We only focus on the foreground objects to reduce computation complexity. Then we introduce Differentiable Screen-Space Rendering (DSSR), a novel approach for estimating the normal and lighting condition jointly. We recover the most plausible lighting condition using spherical harmonics. Our approach provides plausible results and considerably enhances the visual realism in AR applications.