Learning Based No-Reference Image Quality Assessment

Abstract

This thesis aims to address the critical challenges of creating precise and efficient learning-based methods for image analysis. Specifically, our focus lies in creating robust and efficient techniques for assessing image quality, known as No-reference Image Quality Assessment (NR-IQA) models, NR-IQA refers to the evaluating and quantifying the perceived quality of an image without the need for a reference image for comparison. Moreover, we present approaches for the estimation of noise parameters, acknowledging noise as an important artifact in images that significantly influences image quality.

Deep learning forms the foundation for many recently developed NR-IQA methods, and datasets play an essential role in deep learning. However, in the NR-IQA field, acquiring extensive No-Reference datasets (NR datasets) with accurate quality scores (ground truth quality of image knows as Mean Opinion Score) is challenging. Most available datasets are relatively small, posing a risk of overfitting. To address this issue, transfer learning has become widely used. However, transfer learning’s performance is limited to pre-existing models and only fine-tuning the pre-trained network on the desired smaller dataset. To tackle this issue, this thesis introduces a Novel Transfer Learning approach and its modified version, Iterative Transfer learning, which allows us to use transfer learning for a customized deep structure, utilizing a large unlabeled dataset (no ground truth target value is available) for training. Consequently, this approach effectively overcomes the limitations of relying on off the- shelf models and the lack of large NR datasets that own Mean Opinion Scores (MOS).

Ensuring fast and efficient implementation is another essential criterion in designing NR-IQA systems, particularly for real-time application performance. This thesis addresses this challenge by introducing a Two-Branch CNN (TBCNet). This architecture is lightweight, featuring minimal training parameters compared to state-of- the-art NR-IQA models. Furthermore, it is designed to mimic the comprehension of the Human Visual System by incorporating two types of inputs—high-level and low-level features. Experimental results demonstrate that our proposed TBCNet, trained through Iterative Transfer Learning is robust. Its accuracy in estimating image quality matches the state-of-the-art, and in terms of speed and efficiency, it outperforms the current state-of-the-art.

In addition to the design of NR-IQA for general quality assessment purposes, this thesis places particular emphasis on noise estimation and its impact on NRIQAs’ performance. Noise presents a challenging characteristic as an image artifact, introducing high-frequency features to an image. Distinguishing noise from image deblurring tools, especially in low-noise scenarios, is essential. Various noise parameter estimation techniques have been developed, with a focus on models like AWGN and Poisson noise. Recently, attention has shifted to non-stationary noise prevalent in real-world scenarios. To address this, we introduce two autoencoder-based models crafted for estimating non-stationary noise. Our proposed noise estimators can effectively calculate noise parameters in the form of a matrix representing noise variance, providing a depiction of the noise level for all regions within an image. Experimental results and comparisons with state-of-the-art methods indicate that our model excels, particularly in low-level noise scenarios.

Subsequently, we broadened our study on noise and NR-IQA together by investigating the role of noise in the performance of NR-IQA and improving the sensitivity of NR-IQA to the existence of noise. To enhance the sensitivity of NR-IQA on the influence of noise, we integrated noise estimators into the NR-IQA framework. The results indicate a noticeable improvement in general performance of image quality metrics when considering the influence of noise.