Demo: Attention and Mobility-Aware On-Device Perception for Mobile Augmented Reality

Date：Dec 2023

Teams:New Jersey Institute of Technology

Writers:Mayooran Thavendra,Kasthuri Jayarajah

PDF:Demo: Attention and Mobility-Aware On-Device Perception for Mobile Augmented Reality

Abstract

With the ability to seamlessly integrate physical and virtual worlds to provide immersive experiences, Mobile Augmented Reality, is a promising technology for several futuristic applications such as on-the-job training and guidance in smart factories, human-autonomy teaming, etc. However, the hardware limitations of most modern AR/VR headsets are prohibitive in that the processing latencies for common scene understanding tasks, a foundational primitive for such applications, can be in the order of 1sec leading to poor quality of experience (QoE). The problem is further exacerbated by contention for resources between processing and rendering. State-of-the-art in perception for these headsets rely on assistance from nearby edge servers [1, 2] which may not always be available in practical settings. In this demonstration, we show how user attention and mobility-aware scheduling can improve the processing latency without significant loss in perception accuracy and QoE for on-device perception.