Super-resolution image reconstruction using non-linear filtering techniques

Abstract

Super-resolution (SR) reconstruction is a filtering technique that aims to combine a sequence of under-sampled and degraded low-resolution images to produce an image at a higher resolution. The reconstruction attempts to take advantage of the additional spatio-temporal data available in the sequence of images portraying the same scene. The fundamental problem addressed in super-resolution is a typical example of an inverse problem, wherein multiple low-resolution (LR) images are used to solve for the original high-resolution (HR) image.

Super-resolution has already proved useful in many practical cases where multiple frames of the same scene can be obtained, including medical applications, satellite imaging and astronomical observatories. The application of super resolution filtering in consumer cameras and mobile devices shall be possible in the future, especially that the computational and memory resources in these devices are increasing all the time. For that goal, several research problems need to be investigated, i.e., precise modeling of the image capture process, fast filtering methods, accurate methods for motion estimation and optimal techniques for combining pixel values from the motion compensated images.

In this thesis, we investigate a number of topics related to the performance problems mentioned above. We develop novel solutions to improve the image quality captured by the sensors of a camera phone. Particularly, we present a framework for producing a high-resolution color image directly from a sequence of images captured by a CMOS sensor that is overlaid with a color filter array. In the proposed framework, we introduce a super-resolution algorithm that interpolates the subsampled color components and reduces the optical blurring. The results confirm that it is possible to improve the overall image quality by using few consecutive shots of the same scene.

Achieving accurate and fast registration of the input images is a critical step in super-resolution processing. Motivated by this basic requirement, we propose a novel recursive method for pixel-based motion estimation. We use recursive least mean square filtering (LMS) along different scanning directions to track the stationary shifts between a pair of LR images, which results in smooth estimates of the displacements at sub-pixel accuracy. The initial results indicate good performance, especially for tracking smooth global motion. One important advantage of the proposed method is that it can be easily integrated into super-resolution algorithms thanks to its relative low computational complexity.

The overall performance of super-resolution is particularly degraded in the presence of motion outliers. Therefore, it is essential to develop methods to enhance the robustness of the fusion process. Towards this goal, we propose an integrated adaptive filtering method to reject the outlier image regions. The proposed approach consists in applying non-linear filtering techniques to improve the performance and robustness against motion outliers. In particular, we applied median filtering for robust fusion of the LR images, and we used generalized order statistic filters (OSF) for the enhancement of binary text images. Compared with conventional super-resolution algorithms, the proposed algorithms preserved well the fine details in the images, additionally, the result images exhibited less artifacts in the presence of registration errors. This confirms the advantage of using order statistic filtering in image super-resolution.