Applying 3D Gaussian Splatting In Real World Point Cloud Rendering

Abstract

3D rendering is transforming 3D models into 2D images and capturing an accurate por-trayal of the object from a specific viewpoint. High-fidelity 3D rendering requires simulating light interaction and depth of an object on to a flat screen. The quality of the render is directly proportional to time and computational resources required. To project 3D vertices to a 2D plane, pixel positions for the shape must be calculated and the visible surfaces of the object must be determined. This process is called rasterization.

This thesis explores how 3D Gaussian splatting or 3DGS can improve the efficiency and realism rendering sparse point clouds in real-time applications. 3DGS is a technique developed for volume-based rendering and view synthesis, it uses 3D Gaussian functions, or splats, to model real and artificial 3D scenes without relying on neural networks. Unlike radial field meth-ods, Gaussian splatting achieves fast, real-time rendering at Full HD (1080p) resolution, while reducing the computational load. In the context of point clouds, 3D Gaussian Splatting has multiple advantages. It enables fast render times using splat rasterization, a more efficient pro-cess than classical ray tracing and is well suited for performance-critical applications.

Gaussian Splatting can also model physical properties such as surface normals, BRDF (bidirectional reflectance distribution function) parameters, colour, opacity and incoming illumi-nation, allowing realistic shadows and scattering of light to be modelled in scenes. The inherent flexibility of point clouds also makes scenes easy to edit and animate, increasing the adapta-bility of this rendering process.

The aim of this thesis is to recreate a room as a 3D point cloud using methods like Structure from Motion and Stereo Vision. We then apply a 3D Gaussian Splatting (3DGS) pipe-line to this point cloud. Lastly, we re-illuminate a virtual object inside the room using the ambi-ent lighting of the room and compare the results to a 3DGS scene created form dense point cloud. The aim of this thesis is to explore 3DGS rendering and rasterization methods, evaluat-ing their robustness and effectiveness in realistic re-illumination.