A GPU-Accelerated Pipeline for Adaptive Meshing from Multiple RGB-D Streams
Käpylä, Jani (2025)
2025
Sähkötekniikan DI-ohjelma - Master's Programme in Electrical Engineering
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
Hyväksymispäivämäärä
2025-07-30
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202507297884
https://urn.fi/URN:NBN:fi:tuni-202507297884
Tiivistelmä
Volumetric video is dynamic 3D content that can be viewed from arbitrary angles, making it ideal for applications such as virtual reality, telepresence, and other immersive media. Capturing and processing such content in real-time presents a significant computational challenge due to both the high data throughput and the complexity of the volumetric video pipeline, which spans multiple stages from capture to rendering. This thesis addresses a key component of that pipeline: converting multi-view RGB-D data into 3D mesh representations suitable for volumetric video applications.
The goal of this work is to design and implement a full GPU-based pipeline for real-time meshing of multiple RGB-D streams. We study and integrate novel methods, and specifically ones aimed at efficient meshing. The system aims at meeting three primary requirements: 1) real-time execution on GPU, 2) adaptive meshing, and 3) meshing on individual depth image planes, followed by merging into a unified representation.
To achieve this, we propose a complete GPU pipeline comprising three main components: filtering, meshing, and zippering, each composed of smaller functional elements. The filtering stage is a critical preprocessing step that performs overall smoothing and merges different views by projecting points onto the Moving Least Squares (MLS), along with applying other supporting filters. In the meshing stage, a Greedy Vertex Insertion (GVI) algorithm triangulates each depth image in smaller blocks. A final optimization pass then refines the mesh using the Lawson's Local Optimization Procedure (LOP), guided by a Data-Dependent Triangulation (DDT) criterion. Finally, the boundaries of the individual meshes are merged using a lightweight zippering strategy. An additional projection-based acceleration strategy is employed to expedite neighbor searches across various stages of the pipeline.
The pipeline is evaluated using data from a system comprising four RGB-D cameras. The results indicate that the approach is promising and yields a satisfactory final output in real-time for this dataset. The filtering stage, particularly the MLS, effectively reduces noise and outliers while also merging the views, thereby enabling successful downstream processing. The use of GVI and DDT proved effective, yielding adaptive and detailed mesh results. However, the zippering block would still benefit from further development, as it may produce visible artifacts in certain areas. Several improvements and directions for future work have been identified, including the selection of different DDT criteria and further application-specific optimization of the pipeline.
The goal of this work is to design and implement a full GPU-based pipeline for real-time meshing of multiple RGB-D streams. We study and integrate novel methods, and specifically ones aimed at efficient meshing. The system aims at meeting three primary requirements: 1) real-time execution on GPU, 2) adaptive meshing, and 3) meshing on individual depth image planes, followed by merging into a unified representation.
To achieve this, we propose a complete GPU pipeline comprising three main components: filtering, meshing, and zippering, each composed of smaller functional elements. The filtering stage is a critical preprocessing step that performs overall smoothing and merges different views by projecting points onto the Moving Least Squares (MLS), along with applying other supporting filters. In the meshing stage, a Greedy Vertex Insertion (GVI) algorithm triangulates each depth image in smaller blocks. A final optimization pass then refines the mesh using the Lawson's Local Optimization Procedure (LOP), guided by a Data-Dependent Triangulation (DDT) criterion. Finally, the boundaries of the individual meshes are merged using a lightweight zippering strategy. An additional projection-based acceleration strategy is employed to expedite neighbor searches across various stages of the pipeline.
The pipeline is evaluated using data from a system comprising four RGB-D cameras. The results indicate that the approach is promising and yields a satisfactory final output in real-time for this dataset. The filtering stage, particularly the MLS, effectively reduces noise and outliers while also merging the views, thereby enabling successful downstream processing. The use of GVI and DDT proved effective, yielding adaptive and detailed mesh results. However, the zippering block would still benefit from further development, as it may produce visible artifacts in certain areas. Several improvements and directions for future work have been identified, including the selection of different DDT criteria and further application-specific optimization of the pipeline.