Light Field Reconstruction from Sparse Input Data: Comparing NeRF and Gaussian Splatting Approaches

Abstract

Light fields represent all light rays emitted or reflected from a scene. This allows for realistic image creation from different viewpoints. This characteristic makes them useful for virtual and augmented reality applications. However, capturing dense light fields usually needs special hardware or a large number of input images. This thesis explores if modern neural rendering techniques can create high-quality light field reconstructions using sparse input data.

Three methods were compared. Instant-NGP as a Neural Radiance Field approach, while two Gaussian Splatting variants were Splatfacto and Feature Splatting. The experiments involved two synthetic light field datasets with different numbers of training views ranging from 3 to 25 images.

The results show that Gaussian Splatting methods significantly outperform NeRF in reconstruction quality, achieving an average of 35-40 dB PSNR. The two Gaussian Splatting methods also rendered at 7-14 FPS, while Instant-NGP did not reach even 1 FPS. Interestingly, using more training views did not always lead to better reconstruction quality.

These results indicate that Gaussian Splatting is a promising method for practical light field reconstruction. It offers both high quality and real-time rendering. However, further research is needed beyond this thesis.