Sensor Simulation for Monocular Depth Estimation using Deep Neural Networks

PubDate: November 2021

Teams: Fraunhofer IDMT；Ilmenau University of Technology

Writers: Christon R. Nadar; Christian Kunert; Tobias Schwandt; Wolfgang Broll

PDF: Sensor Simulation for Monocular Depth Estimation using Deep Neural Networks

Abstract

Depth estimation is one of the basic building blocks for scene understanding. In the case of monocular depth estimation using neural networks, many such approaches are highly hardware dependent because they result in a task- and environment-specific optimizing problem. Most DNN methods use commonly available datasets which leads to overfitting on particular sensor properties. Finding a generalized model with the consideration of different hardware properties of sensors and platforms is challenging if not impossible. For this reason, it is desirable to adapt existing and well-trained models into a new domain in order to let them simulate different depth sensors without the need for large datasets and time-consuming learning. Therefore, a small dataset has been created with the Structure Sensor for evaluating the transferable structural characteristic between neural networks. Finally, two input feature representations for the neural networks are considered to mimic the depth sensor including its artifacts including holes. The results show that a simple domain adaptation technique and a small dataset are adequate to simulate and adapt to a specific domain from a target domain. Therefore, the network is able to accurately predict depth maps as if they were created by a specific depth sensor. This also includes unique artifacts of the sensor, thereby allowing for a plausible simulation of specific depth sensing hardware which is beneficial for areas like prototyping in the context of Augmented Reality.