A multimodal drowsiness dataset using video, biometric, and behavioral data

Abstract

We present a comprehensive public dataset for driver drowsiness detection, integrating multimodal signals of facial, behavioral, and biometric indicators. Our dataset includes 3D facial video, infrared footage, posture videos, and biometric signals like heart rate, electrodermal activity, blood oxygen saturation, skin temperature, and accelerometer data. This data set provides grip sensor data and telemetry data to provide more information about drivers’ behavior while they are alert and drowsy. Drowsiness levels were self-reported every four minutes using the Karolinska Sleepiness Scale (KSS). Data were collected from 19 subjects in two conditions: when they were fully alert and when they exhibited signs of sleepiness. Unlike other datasets, our multimodal dataset has a continuous duration of 40 minutes for each data collection session per subject, contributing to a total length of 1,400 minutes. We recorded gradual changes in the driver state rather than discrete alert/drowsy labels. This study aims to create a publicly available multimodal dataset of driver drowsiness that captures a wider range of physiological, behavioral, and driving-related signals.