Deep-BrownConrady: Prediction of Camera Calibration and Distortion Parameters Using Deep Learning and Synthetic Data

Abstract

This research addresses the challenge of camera calibration and distortion parameter prediction from a single image using deep learning models. The main contributions of this work are: (1) demonstrating that a deep learning model, trained on a mix of real and synthetic images, can accurately predict camera and lens parameters from a single image, and (2) developing a comprehensive synthetic dataset using the AILiveSim simulation platform. This dataset includes variations in focal length and lens distortion parameters, providing a robust foundation for model training and testing. The training process predominantly relied on these synthetic images, complemented by a small subset of real images, to explore how well models trained on synthetic data can perform calibration tasks on real-world images. Traditional calibration methods require multiple images of a calibration object from various orientations, which is often not feasible due to the lack of such images in publicly available datasets. A deep learning network based on the ResNet architecture was trained on this synthetic dataset to predict camera calibration parameters following the Brown-Conrady lens model. The ResNet architecture, adapted for regression tasks, is capable of predicting continuous values essential for accurate camera calibration in applications such as autonomous driving, robotics, and augmented reality. Note to Practitioners - This paper introduces a deep learning approach to predict camera calibration and distortion parameters directly from a single image, addressing the limitations of traditional methods that require structured calibration objects and multiple images. Using synthetic datasets generated with a simulation platform, the model predicts essential parameters such as field of view, principal points, and Brown-Conrady distortion coefficients. A key innovation is incorporating image size into the learning process, enabling the model to generalize well to real-world scenarios. This approach simplifies the calibration process, making it suitable for dynamic and unstructured environments, such as autonomous driving and robotics, where traditional calibration methods are not feasible. As an important result, the proposed method enables the use of synthetic data to overcome data scarcity in real-world applications by adapting to the camera parameters of the underlying physical system. By leveraging synthetic data and deep learning, the method offers a modern, flexible alternative that practitioners can adopt to enhance camera calibration workflows in real-world applications. Extensive experiments are reported that showcase a successful solution based on the famous Brown-Conrady camera lens model and are validated on a real-world dataset. We believe a similar methodology can be used and extended as future work to enable camera parameter estimation and the use of synthetic data for other camera models.