Sensing and Reconstruction of Plenoptic Point Clouds

Abstract

This thesis provides an overview of the plenoptic function and how it relates to volumetric data, through plenoptic scene representations. It reviews the existing methods that introduce such capability explicitly and implicitly, in the form of a Plenoptic Point Cloud (PPC) and Radiance Fields (RF), respectively. These are presented from the point of view of the challenges of practicality of these methods for applications of streaming for interactive content, namely the size and speed.

Initially, we incorporated the plenoptic capability for MPEG’s geometry-based encoder (G-PCC) by compressing PPCs using a combination of the linear transforms over the color vector of the different camera viewpoints combined G-PCC’s predictive attribute coders.

We follow this by addressing the size disadvantages of real-time rendering implementations of NeRF-based methods, by introducing a compression pipeline to the PlenOctrees model.

Afterwards, we introduce a method to systematically generate PPCs and compare them directly against RF solutions with conventionally used render-based metrics.

Finally, we leverage the underlying geometry of the RF models to orient their pruning for a more efficient compression.