Prediction of Camera Calibration and Distortion Parameters Using Deep Learning and Synthetic Data

Abstract

In this research, the gap in calibration data has been addressed through rigorous experimentation with synthetic data. The focus of the study is to predict camera calibration and distortion parameters from a single image. Applications dependent on 3D geometry, such as autonomous driving, robotics, and augmented reality, require calibrated cameras. Traditional calibration methods often require multiple images of a calibration object, such as a chessboard pattern, captured from different angles. However, most publicly available datasets do not include such images, which hinders the accurate calibration of cameras, especially in diverse and uncontrolled environments. This research focuses on the development of deep learning models that have been trained on synthetic datasets generated using a simulation platform, the AILiveSim. We have used AILiveSim’s simulator to generate considerable number of synthetic images with different Horizontal Field-Of-View (H-FOV) and lens distortion parameters This large dataset was generated and used to train a Residual Networks (ResNets) model to learn eight camera distortion and calibration parameters k1, k2, k3, p1, p2 from the Brown-Conrady model, the principal axes cx and cy of the image and H-FOV). The ResNet architecture, known for its ability to handle complex image-based tasks, is adapted for regression tasks. This involves modifying the network’s output layers and loss functions to predict continuous values rather than categorical labels.