Real-Time Panoramic Video Stitching Software
Tasanen, Saku (2023)
2023
Tietotekniikan DI-ohjelma - Master's Programme in Information Technology
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2023-05-04
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202305014899
https://urn.fi/URN:NBN:fi:tuni-202305014899
Tiivistelmä
Typically, video production requires a lot of manual labor. Sport events can be several hours long and during that time the camera equipment requires an operator. The cost of labor and large amount of time spent on video production increases demand for automated video production solutions. However, automatic video production is not a simple task. Sport venues are typically quite large and thus capturing the whole venue with a single static camera is often not possible.
A solution to increasing the field of view is to create a panoramic video using multiple synchronized cameras. The main problem with this approach is that panorama stitching is computationally expensive. Another challenge is creating natural looking panoramas. Images captured using different cameras can have color or exposure differences and final panoramas might have visible seams or alignment errors. Although image stitching methods have been known for a long time, software solutions capable of creating high-quality panorama videos are lacking. Most software implementations focus on still image stitching rather than video stitching.
This thesis presents a fully automated cloud-based video production system that produces panoramic videos for sports events. The system consists of multiple software systems: the video recorder, the event recording manager, the video processing and the video archive. The focus of this thesis is on the video processing software’s image processing pipeline. The pipeline is implemented as a graph, where each processing step is implemented as a node. The processing steps are distortion correction, cylindrical projection, color and exposure compensation, image blending and panorama composition.
The implemented software is capable of stitching four 2160p video streams into a 7200×3584 resolution panorama stream in real time using an NVIDIA Tesla P4 graphics card. This makes high-quality broadcasts of sport events possible. Compared to traditional broadcasts, the panorama video has the advantage of capturing the whole venue, enabling new virtual reality applications with richer viewing experience.
A solution to increasing the field of view is to create a panoramic video using multiple synchronized cameras. The main problem with this approach is that panorama stitching is computationally expensive. Another challenge is creating natural looking panoramas. Images captured using different cameras can have color or exposure differences and final panoramas might have visible seams or alignment errors. Although image stitching methods have been known for a long time, software solutions capable of creating high-quality panorama videos are lacking. Most software implementations focus on still image stitching rather than video stitching.
This thesis presents a fully automated cloud-based video production system that produces panoramic videos for sports events. The system consists of multiple software systems: the video recorder, the event recording manager, the video processing and the video archive. The focus of this thesis is on the video processing software’s image processing pipeline. The pipeline is implemented as a graph, where each processing step is implemented as a node. The processing steps are distortion correction, cylindrical projection, color and exposure compensation, image blending and panorama composition.
The implemented software is capable of stitching four 2160p video streams into a 7200×3584 resolution panorama stream in real time using an NVIDIA Tesla P4 graphics card. This makes high-quality broadcasts of sport events possible. Compared to traditional broadcasts, the panorama video has the advantage of capturing the whole venue, enabling new virtual reality applications with richer viewing experience.