Attention Enhancement and Model Quantization for Efficient Differential Mobility Spectrometry Analysis

Abstract

An important data pre-processing step in the classification problem of Volatile Organic Compounds (VOCs) with Differential Mobility Spectrometry (DMS) is the extraction of alpha curves. The alpha curves represent distinct patterns that characterize ion mobility across a range of compensation and separation voltages in the DMS spectrum. Recent methods for extracting alpha curves include scaling, smoothing, and detecting local maxima. In this thesis, I intend to explore a deep learning neural network able to accurately detect and extract the alpha curves, which can subsequently be employed in VOC classification tasks. The implemented network architecture is UNet with different variants of attention: Self-Attention, Narrow-Attention, Broad-Attention, and Attention Gates (AG). The performance of each variant is studied and discussed to determine which configuration provides the optimal results. Furthermore, this research explores quantization techniques as a means to enable the deployment of these computationally demanding models on resource-constrained devices. Local processing on resource-constrained devices is highly attractive for handling VOC datasets given the potential privacy concerns and relevant legal requirements. In this thesis, two primary quantization techniques are explored in the trained UNet variants: Static and Dynamic Post Training Quantization (PTQ) and Quantization Aware Training (QAT). The results indicate that the UNet architecture incorporating AG consistently shows the best performance for segmentation of alpha curves. Broad-Attention was unable to fully converge in the training configuration and achieved the worst performance. QAT delivered the most balanced quantization with good model shrinkage and latency improvement, along with a slight improvement in segmentation accuracy. Static PTQ also followed similar patterns, whereas Dynamic PTQ achieved no model shrinkage but significantly higher latency improvement. In conclusion, no quantization solution proves to be universally optimal for deployment on resource-constrained hardware. Each method presents its own trade-offs, and the optimal approach is determined by the availability of specific resources and inference requirements.