Long-term Visual Place Recognition Under Varying Conditions

Abstract

The objective of this Ph.D. dissertation is to study the problem of visual place recognition (VPR) under challenging conditions using deep learning methods. VPR is the ability of a vision-based navigation system to determine a match within the previously visited places with resilience to perceptual aliasing as well as seasonal, illumination and viewpoint variations.

VPR is a challenging problem due to environmental changes in which the appearance of the places can vary over time. It is one of the essential and challenging problems in the field of robotics and computer vision. In the last few years, enormous improvements in visual sensing capabilities, an increasing concentration on long-term mobile robot autonomy, and the ability to conduct state-of-the-art research have all contributed to significant advances in VPR systems. Long-term robot autonomy has revealed that changing conditions can be a significant factor in its failure. Therefore, it is crucial to come up with more advanced solutions which considers a variety of changes within the environment, including seasonal, weather, illumination and occlusion.

Compared to the traditional machine learning approaches, deep learning methods are currently among the most widespread and successful research topics in several disciplines, including computer vision and robotics. Deep learning is a powerful tool for obtaining representations automatically from raw data required for various recognition tasks such as VPR. This dissertation investigates and explores deep Convolutional Neural Networks (CNN) to obtain abstract and distinctive feature representations of the input images as an essential tool for image retrieval and VPR, in which an algorithm finds the matches of the gallery sample similar to the query sample.

Towards the goals, first, we compared the performance of deep CNN architectures coupled with BatchNorm layers using architectures primarily trained for image classification and object detection as holistic feature descriptors for VPR. Second, we investigated the performance of learned global features when trained using three different loss functions in an end-to-end manner for learning the parameters of the architectures in terms of the fraction of the correct matches during deployment. Next, we evaluated two state-of-the-art deep metric learning methods for VPR using vision and LiDAR sensor modalities along with ablation studies on the crucial parameters of deep architectures. Finally, we looked into the topic of deep long-term VPR as a systematic approach to propose an experimental benchmark using deep CNN architectures in order to obtain the discriminative feature representation.

Our results which are provided extensively in the publications and briefly in this dissertation highlight the outperformance capabilities of the deeply learned feature representations when fine-tuned for the VPR task under a wide variety of challenging conditions both for indoor and outdoor environments.