Learning Computational Hyperspectral Imaging
Kim, Ayoung (2022)
Kim, Ayoung
2022
Master's Programme in Computing Sciences
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2022-11-17
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202210247783
https://urn.fi/URN:NBN:fi:tuni-202210247783
Tiivistelmä
This thesis discusses the computational hyperspectral imaging systems that incorporate end-to-end learnt diffractive optics and a color filter array in the camera hardware and a neural network at the post-processing stage. Two alternative post-processing networks are utilized: residual dense network (RDN) [1] and an unrolling network. The systems are designed to address two main challenges: First, 3D hyperspectral data, which consists of a one-dimensional spectral domain and a two-dimensional spatial domain, must be acquired via the 2D sensor of conventional cameras. This makes the problem severely ill-posed. Second, we search for a snapshot solution that is applicable to dynamic scenes without imposing limitations other than camera sensor’s available frame rate.
The camera model consisting of optimized diffractive optical element and color filter can capture a 3D hyperspectral data cube within a single detector integration period while preserving rich spectral information. Besides, we solve the severely ill-posed reconstruction problem using two different approaches: one is the deep learning method, and another is the unrolling method, which is a hybrid iterative optimization and deep learning approach. The proposed deep learning method is RDN applying residual learning mechanism. An unrolling algorithm utilizes data-driven priors based on a mathematical model. Lastly, the end-to-end training of camera parameters and post-processing plays a critical role in obtaining good performance.
We evaluate our method through simulations, considering the effects of each optical element, the reconstruction network structure, and end-to-end training. Moreover, we prove the noise tolerance of the proposed method by assuming the various training and test noise settings. Simulation results demonstrate high-quality hyperspectral image reconstruction capabilities through the proposed computational hyperspectral cameras, which advance the existing solutions for snapshot hyperspectral imaging.
The camera model consisting of optimized diffractive optical element and color filter can capture a 3D hyperspectral data cube within a single detector integration period while preserving rich spectral information. Besides, we solve the severely ill-posed reconstruction problem using two different approaches: one is the deep learning method, and another is the unrolling method, which is a hybrid iterative optimization and deep learning approach. The proposed deep learning method is RDN applying residual learning mechanism. An unrolling algorithm utilizes data-driven priors based on a mathematical model. Lastly, the end-to-end training of camera parameters and post-processing plays a critical role in obtaining good performance.
We evaluate our method through simulations, considering the effects of each optical element, the reconstruction network structure, and end-to-end training. Moreover, we prove the noise tolerance of the proposed method by assuming the various training and test noise settings. Simulation results demonstrate high-quality hyperspectral image reconstruction capabilities through the proposed computational hyperspectral cameras, which advance the existing solutions for snapshot hyperspectral imaging.