Real-Time Light Field Path Tracing

Abstract

Light field rendering and displays are emerging technologies that produce more immersive visual 3D experiences than the conventional stereoscopic 3D technologies, as well as provide a more comfortable virtual or augmented reality (VR/AR) experience by mitigating the vergence-accommodation conflict. Path tracing photorealistic synthetic light fields in real time is extremely challenging, since it involves rendering a large amount of viewpoints for each frame. However, these viewpoints are often spatially very close to each other, especially in light field AR glasses or other near-eye light field displays. In this paper, we propose a practical real-time light field path tracing pipeline and demonstrate it by rendering a 6x6 grid of 720p viewpoints at 18 frames per second on a single GPU, through utilizing denoising filters and spatiotemporal sample reprojection. In addition, we discuss how the pipeline can be scaled to yield higher-quality results if more parallel computing resources are available. We also show that our approach can be used to simultaneously serve multiple clients with varying light field grid sizes, with the quality remaining constant across clients.