Measuring Image Quality for Automatic Image Signal Processor Parameter Optimization
Aho, Jere (2025)
2025
Tietotekniikan DI-ohjelma - Master's Programme in Information Technology
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2025-04-09
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202504083502
https://urn.fi/URN:NBN:fi:tuni-202504083502
Tiivistelmä
The image signal processor (ISP) is a crucial component of a camera system. Traditionally, it includes a set of sequential algorithms, each with its own tunable parameters, which must be optimized to obtain good image quality (IQ) from a camera. The optimal parameters depend on the input data to the ISP, as well as the operating conditions.
This thesis explores the use of Camera System Image Quality (CSIQ) metrics for automatically optimizing ISP parameters related to noise filtering and sharpening. It aims to answer the questions: "How can the CSIQ metrics be selected and combined to obtain a meaningful single loss?" and "What are the limitations of the CSIQ metrics relating to ISP parameter optimization?" The thesis reviews the standard CSIQ metrics with mappings to the perceptually correlated unit, Just Noticeable Difference (JND).
The selected metrics, along with a proposed oversharpening metric are combined into a single loss function, which is tested in an automatic ISP parameter optimization framework. Assuming that the loss function defines the desired IQ well enough, the optimization should converge to an improved solution. The CSIQ-based loss is evaluated against the Deep Image Structure and Texture Similarity (DISTS), a Full-Reference Image Quality Assessment (FRIQA) metric, which in other studies is found to perform well in image enhancement tasks.
The research shows that the proposed CSIQ approach for automatic ISP parameter optimization works particularly when the image has moderate noise level. Increasing the weight of the noise metric in the proposed loss improves convergence in high-noise cases, leading to better results. The DISTS performed better in the high-noise image, giving more consistent results over different amounts of noise. The results highlight weaknesses in the proposed CSIQ-based loss, such as inaccuracy in the presence of high noise and errors due to incorrect annotation. This inaccuracy due to noise is especially bad, as high noise mismeasured image may be incorrectly judged as high quality, resulting in unusable parameters. With the methodology used, neither metric was able to effectively assess chroma noise filtering.
This thesis offers insight into automatic optimization of the ISP parameters. The proposed CSIQ-based loss function serves as a foundation for further research and development on CSIQ metrics in this subject. The proposed loss can be improved by adding more metrics to cover more IQ attributes, improving the robustness of the current metrics, and modifying the JND mappings.
This thesis explores the use of Camera System Image Quality (CSIQ) metrics for automatically optimizing ISP parameters related to noise filtering and sharpening. It aims to answer the questions: "How can the CSIQ metrics be selected and combined to obtain a meaningful single loss?" and "What are the limitations of the CSIQ metrics relating to ISP parameter optimization?" The thesis reviews the standard CSIQ metrics with mappings to the perceptually correlated unit, Just Noticeable Difference (JND).
The selected metrics, along with a proposed oversharpening metric are combined into a single loss function, which is tested in an automatic ISP parameter optimization framework. Assuming that the loss function defines the desired IQ well enough, the optimization should converge to an improved solution. The CSIQ-based loss is evaluated against the Deep Image Structure and Texture Similarity (DISTS), a Full-Reference Image Quality Assessment (FRIQA) metric, which in other studies is found to perform well in image enhancement tasks.
The research shows that the proposed CSIQ approach for automatic ISP parameter optimization works particularly when the image has moderate noise level. Increasing the weight of the noise metric in the proposed loss improves convergence in high-noise cases, leading to better results. The DISTS performed better in the high-noise image, giving more consistent results over different amounts of noise. The results highlight weaknesses in the proposed CSIQ-based loss, such as inaccuracy in the presence of high noise and errors due to incorrect annotation. This inaccuracy due to noise is especially bad, as high noise mismeasured image may be incorrectly judged as high quality, resulting in unusable parameters. With the methodology used, neither metric was able to effectively assess chroma noise filtering.
This thesis offers insight into automatic optimization of the ISP parameters. The proposed CSIQ-based loss function serves as a foundation for further research and development on CSIQ metrics in this subject. The proposed loss can be improved by adding more metrics to cover more IQ attributes, improving the robustness of the current metrics, and modifying the JND mappings.