Software Implementation of Contrast-Based Autofocus in Mobile Camera System
Jantunen, Heikki (2014)
2014
Tietotekniikan koulutusohjelma
Tieto- ja sähkötekniikan tiedekunta - Faculty of Computing and Electrical Engineering
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2014-06-04
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tty-201405231213
https://urn.fi/URN:NBN:fi:tty-201405231213
Tiivistelmä
Autofocus is an important part of a modern digital camera system. Lens of the camera redirects light onto the surface of the imaging sensor. The distance between the lens and the sensor is in direct relation to the distance at which the scene appears sharp in the captured image. The purpose of focusing is to move the camera lens so that the region of interest in the image is sharp. Autofocus aims to do this automatically, without user interaction. Contrast-based autofocus algorithm works as a part of the image processing pipeline and uses metrics provided by the image signal processor (ISP) to analyse the sharpness and moves the based on this analysis.
In this thesis, the target was to create an autofocus system that works independent of the ISP and calculates the metrics on the CPU of the target device, Nokia Lumia 1520. The benefit of a pure software implementation is that it will remove the need for the ISP hardware for autofocus and adds flexibility to the metrics calculation process because configurability is not limited by the particular hardware implementation of the ISP. By preprocessing the image data before metrics calculation, it is possible to enhance the low-light performance of the system. However, the challenge of replacing a dedicated piece of hardware with software processing lies in creating an implementation that is efficient enough to be practical.
An autofocus framework was implemented. It provides a background processing system for calculating the metrics and possible preprocessing. Threading is utilised as means of optimization so that the image is processed in parts. The metrics are processed during the exposure of the next frame, which leads to latency in the availability of the metrics. To take this into account, also a simple sweep-based single pass autofocus algorithm was implemented.
For calculating the metrics, three focus operators and three preprocessing methods were implemented and evaluated. The techniques varied in the heaviness of calculation and they were optimized using NEON which is a single instruction multiple data (SIMD) extension of ARM instruction set architecture. MATLAB simulation was used to evaluate the output of the implemented methods.
While all of the focus operators produced very similar results, using median filter for preprocessing provided a significant improvement for low-light focusing. The autofocus system was also run on the target device with combinations of the implemented metrics processing techniques. Processing times were measured and the framework was proved to be applicable with any combination of the techniques.
In this thesis, the target was to create an autofocus system that works independent of the ISP and calculates the metrics on the CPU of the target device, Nokia Lumia 1520. The benefit of a pure software implementation is that it will remove the need for the ISP hardware for autofocus and adds flexibility to the metrics calculation process because configurability is not limited by the particular hardware implementation of the ISP. By preprocessing the image data before metrics calculation, it is possible to enhance the low-light performance of the system. However, the challenge of replacing a dedicated piece of hardware with software processing lies in creating an implementation that is efficient enough to be practical.
An autofocus framework was implemented. It provides a background processing system for calculating the metrics and possible preprocessing. Threading is utilised as means of optimization so that the image is processed in parts. The metrics are processed during the exposure of the next frame, which leads to latency in the availability of the metrics. To take this into account, also a simple sweep-based single pass autofocus algorithm was implemented.
For calculating the metrics, three focus operators and three preprocessing methods were implemented and evaluated. The techniques varied in the heaviness of calculation and they were optimized using NEON which is a single instruction multiple data (SIMD) extension of ARM instruction set architecture. MATLAB simulation was used to evaluate the output of the implemented methods.
While all of the focus operators produced very similar results, using median filter for preprocessing provided a significant improvement for low-light focusing. The autofocus system was also run on the target device with combinations of the implemented metrics processing techniques. Processing times were measured and the framework was proved to be applicable with any combination of the techniques.