Mobile AR Depth Estimation: Challenges & Prospects

Date：Feb 2024

Teams:University of Toulouse III

Writers:Ashkan Ganj,Yiqin Zhao,Hang Su,Tian Guo

PDF:Mobile AR Depth Estimation: Challenges & Prospects

Abstract

Accurate metric depth can help achieve more realistic user interactions such as object placement and occlusion detection in mobile augmented reality (AR). However, it can be challenging to obtain metricly accurate depth estimation in practice. We tested four different state-of-the-art (SOTA) monocular depth estimation models on a newly introduced dataset (ARKitScenes) and observed obvious performance gaps on this real-world mobile dataset. We categorize the challenges to hardware, data, and model-related challenges and propose promising future directions, including (i) using more hardware-related information from the mobile device’s camera and other available sensors, (ii) capturing high-quality data to reflect real-world AR scenarios, and (iii) designing a model architecture to utilize the new information.