Managing Localization Delay for Cloud-assisted AR Applications Via LSTM-driven Overload Control

PubDate: December 2021

Teams: Budapest University of Technology and Economics；Ericsson Research

Writers: János Czentye; Balázs Péter Gerő; Balázs Sonkoly

PDF: Managing Localization Delay for Cloud-assisted AR Applications Via LSTM-driven Overload Control

Abstract

Simultaneous Localization and Mapping plays a key role in different Augmented and Mixed Reality applications to determine the pose, that is, the position and orientation of the AR user in a 3D coordinate system, in relation with the rendered digital objects. To meet the ever-growing resource demands of these SLAM algorithms, the localization task can be offloaded to edge cloud platforms. Although pose calculation techniques are optimized for resource-limited robotic environments, the volatile nature of cloud platforms with the strict requirements of realtime AR applications can still lead to deteriorated performance. In this paper, we propose an LSTM-driven overload control mechanism that can effectively improve the worst-case response time of an edge-assisted SLAM by predicting overloaded periods in advance. Our main contribution is threefold. First, we identify factors influencing the response time of edge-assisted SLAMs fed by real-time AR applications and propose two applicable control actions. Second, we present our control architecture including an encoder-decoder LSTM model that can forecast response time degradation by using a specific image quality metric. Third, we demonstrate the applicability and performance of our proposed control methods along with their effects on the pose accuracy by performing dedicated experiments with challenging motion patterns and the widely-known EuRoC benchmarking dataset.