Visualizing and Manipulating Artificial Intelligence Confidence in Virtual Reality
Rasouli, Nastaran (2026)
2026
Tietojenkäsittelyopin maisteriohjelma - Master's Programme in Computer Science
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2026-04-24
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202604244270
https://urn.fi/URN:NBN:fi:tuni-202604244270
Tiivistelmä
Artificial intelligence (AI) has significantly advanced medical image segmentation, especially with deep learning architectures such as the 3D U-Net. However, most segmentation systems rely on fixed decision thresholds or argmax-based class selection, offering limited transparency about model confidence. This lack of insight can undermine clinical trust and reduce the practical adoption of AI-generated segmentations. At the same time, virtual reality (VR) has emerged as a promising medium for interactive visualization of volumetric medical data, yet its potential for uncertainty exploration remains largely unexplored.
This thesis presents a framework that combines calibrated voxel-wise confidence estimation with an immersive VR interface, allowing clinicians to control the threshold themselves. A 3D U-Net is trained on mandibular structures in MRI scans from the AAPM-RT-MAC 2019 dataset and calibrated using temperature scaling to improve the reliability of predicted probabilities. The resulting confidence maps are explored in VR through two interaction modes: a single minimum threshold and a threshold range defined by minimum and maximum values. Users can directly manipulate these thresholds and observe how the segmentation changes in real time.
A user study with six clinicians evaluated the system using the UMUX-Lite questionnaire, interaction logs, and qualitative feedback. The results indicate that enabling interactive threshold control can increase perceived trust and improve usability compared to fixed-threshold approaches. Furthermore, providing a threshold range rather than a single cutoff enhanced users’ sense of control and allowed more detailed inspection of the segmentation.
The findings suggest that integrating calibrated confidence visualization with immersive interaction supports explainability and may improve acceptance of AI-assisted segmentation tools in clinical workflows. The work demonstrates the potential of user-centered, VR-based interfaces to bridge the gap between automated segmentation and clinician decision-making.
This thesis presents a framework that combines calibrated voxel-wise confidence estimation with an immersive VR interface, allowing clinicians to control the threshold themselves. A 3D U-Net is trained on mandibular structures in MRI scans from the AAPM-RT-MAC 2019 dataset and calibrated using temperature scaling to improve the reliability of predicted probabilities. The resulting confidence maps are explored in VR through two interaction modes: a single minimum threshold and a threshold range defined by minimum and maximum values. Users can directly manipulate these thresholds and observe how the segmentation changes in real time.
A user study with six clinicians evaluated the system using the UMUX-Lite questionnaire, interaction logs, and qualitative feedback. The results indicate that enabling interactive threshold control can increase perceived trust and improve usability compared to fixed-threshold approaches. Furthermore, providing a threshold range rather than a single cutoff enhanced users’ sense of control and allowed more detailed inspection of the segmentation.
The findings suggest that integrating calibrated confidence visualization with immersive interaction supports explainability and may improve acceptance of AI-assisted segmentation tools in clinical workflows. The work demonstrates the potential of user-centered, VR-based interfaces to bridge the gap between automated segmentation and clinician decision-making.