Light4AR: a Shadow-based Estimator of Multiple Light Sources in Interactive Time for More Photorealistic AR Experiences
PubDate: November 2021
Teams: InterDigital
Writers: Matthieu Fradet; Patrice Hirtzlin; Pierrick Jouet; Anthony Laurent; Caroline Baillard
Abstract
We present Light4AR, a light source estimation solution based on the detection of real cast shadows in an image captured by a mobile device. Given the camera pose of the image and a local 3D model of the scene, the approach consists in analyzing shadows cast by a reference object onto its supporting plane to determine the 3D position and the intensity of multiple light sources. By creating virtual point lights based on the resulting parameters and adding them to the AR scene, all the virtual objects can be illuminated and cast virtual shadows consistent with the real environment lighting, thereby enhancing user experience and object presence. In addition to offering the ability to share results over multiple users while preserving device resources, the server-based GPU implementation provides results in interactive time and makes photorealistic AR experiences accessible to most mobile devices, not limiting them to recent ones or specific OS. The proposed approach requires small manual input, limited to placing in the scene a reference object of known geometry, then selecting a region including the shadows cast by this object. We show the potential of this approach on several challenging scenes with various lighting configurations and background textures.
