Analysing Hybrid Neural and Ray Tracing Perception for Foveated Rendering

Foveated Rendering is a fundamental approach for optimizing latency and achieving high fidelity graphics in Virtual Reality. With the advent of real-time ray tracing, different global illumination effects are becoming feasible in interactive media, although still require solutions for optimally displaying such effects. At the same time, neural rendering techniques, such as Neural Radiance Fields (NeRF) and Gaussian Splatting are featuring realistic reconstruction of objects and scenes through real or pre-rendered images. Previous works introduced the possibility of hybrid rendering approaches, using full Ray Tracing techniques for the Fovea regions of a Head Mounted Display and Neural Rendering approaches, using baked and previously trained Networks for inferring realistic light effects in a more optimal and faster way for the peripheral region. In this work we make a deep analysis of human perception related to five different lighting conditions, commonly achieved by Ray Tracing techniques when combining NeRF for approaching the peripheral region. To this end we conducted a Double-Stimulus Impairment Scale (DSIS) test with 42 users, half of whom were tested on their perception of lighting effects with the reticule present in the image to fixate the user’s gaze, and half of whom were tested without the reticule. Results analysis enhance our understanding of visual perception in virtual reality and the limitations of NeRF on reproducing ray tracing effects for VR environments.