Calibration of a multi-kinect system

Abstract

The recent introduction of low cost RGB-D depth sensors has made available 3D sensing capabilities to many researchers and applications developers. Although the 3D sensing performance of these sensors is able to satisfy the needs of most common applications, there are applications, which require higher accuracy and coverage area. This thesis presents an evaluation of a low cost RGB-D sensor and proposes a multi-sensor system in order to overcome the limitations of a single sensor. The Kinect sensor was chosen as the target of the evaluation process due to its popularity, tools availability and support. The thesis started with an analysis of the 3D sensing mechanism and operation of the sensor, identified its limitations and proceeded with a performance evaluation. The Kinect sensor was evaluated taking into account the behavior of accuracy, precision and resolution and its variation along the distance to the target scene. As a step towards the solution of the limitations of the sensor, a multi-sensor system is proposed and analyzed. The main idea is to combine the data from several sensors located at different positions in order to improve the accuracy of the information and increase the coverage of the scene. The method proposed to combine the information of the multi-sensor system is based on the Kinect performance evaluation results. An experimental testbed with two Kinect sensors has been assembled and tested. It is shown that performance strongly depends on proper calibration of the system. Method for calibration has been developed and experimentally evaluated. Results show good performance of the system. The main outcome of the thesis is the evaluation of the measurements of the Kinect sensor, an automatic calibration system for several sensors and a method to fuse the input from several sensors taking into account limitation on data precision.