Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolution

Omnidirectional video (ODV) provides an immersive visual experience and is widely utilized in virtual reality and augmented reality. However, restricted capturing devices and transmission bandwidth lead to low-resolution ODVs. Video super-resolution (SR) is proposed to enhance resolution, but practical ODV spatial projection distortions and temporal flickering are not well addressed directly applying existing methods. To achieve better ODV-SR reconstruction, we propose a Spatio-Temporal Distortion Aware Network (STDAN) oriented to ODV characteristics. Specifically, a spatially continuous distortion modulation module is introduced to improve discrete projection distortions. Next, we design an interlaced multi-frame reconstruction mechanism to refine temporal consistency across frames. Furthermore, we incorporate latitude-saliency adaptive weights during training to concentrate on regions with higher texture complexity and human-watching interest. In general, we explore inference-free and real-world viewing matched strategies to provide an application-friendly method on a novel ODV-SR dataset with practical scenarios. Extensive experimental results demonstrate the superior performance of the proposed STDAN over state-of-the-art methods.