Visibility-Based Geometry Pruning of Neural Plenoptic Scene Representations

Abstract

The need for more realistic 3D scene representations has fomented the development of models for a wide range of applications. In this context, solutions that attempt to model the light’s behavior through the plenoptic function have provided considerable advancements using neural-based approaches, often presenting a trade-off between rendering time and model sizes. In this work, we propose a pruning framework to reduce the sizes of these models by computing the visibility over the training data, applicable to different 3D scene representations. In particular, we implement first a solution suitable for the 3D Gaussian Splatting, and then we exemplify the solution for the Neural Radiance Fields (NeRF)-style of rendering using PlenOctrees. We show that our pruning solution produces smaller models in terms of the number of elements-be they voxels, points, or Gaussians-with minimal losses in terms of rendering novel views. We further assess our solution by combining it with state-of-the-art (SOTA) compression solutions for both rendering schemes. Results over the NeRF-Synthetic dataset show comparable metrics to the SOTA for PlenOctrees, achieving marginal gains for lower bitrates. For 3DGS, the combination of our pruning method and compression solutions achieves a compression ratio of up to 37.5 times over the uncompressed 3DGS models, with only a 0.5 dB decrease in rendering quality. When compared against other SOTA compression methods, our solution produces models 1.4 times smaller, with less than a 0.1 dB loss over novel views for synthetic data, and models 1.9 times smaller with less than 0.2 dB loss when synthesizing novel views on real-world, outdoor content.