A mixed path tracing and NeRF approach for optimizing rendering in XR Displays

Date：Dec 2023

Teams:Universidade Federal Fluminense

Writers:Horácio Henriques,Eder Oliveira,Esteban Clua,Daniela Trevisan

PDF:A mixed path tracing and NeRF approach for optimizing rendering in XR Displays

Abstract

Real-time effects achieved by path tracing are essential for creating highly accurate illumination effects in interactive environments. However, due to its computational complexity, it is essential to explore optimization techniques like Foveated Rendering. Recent advancements in the field of Neural Radiance Fields allow for reconstructing scenes while maintaining adequate frame rates and keeping global and dynamic illumination while in Virtual Reality. The present paper proposes the usage of hybrid rendering: real path tracing at the fovea region and NeRF baked from offline path-traced images at the peripheries. The main idea is to reduce the size of the path-traced rendered image only for a small amount of pixels related to the fovea area, while at the same time providing less costly reconstructions in the periphery. To assess the effectiveness of our method, we presented users with images of a particular scene generated using both conventional rendering and our unique approach. We then asked for their evaluation regarding any perceived degradation in the images. Our work contributes to the development of rendering techniques for XR experiences that demand low latency and high visual quality through path traced effects.