A Multi-Sensor Dataset for 3D Human Pose Estimation Using Imaging Radar

Abstract

This work focuses on 3D Human Pose Estimation (HPE) using imaging radar. We introduce DualRadarHPE, a new multi-sensor dataset containing calibrated and temporally aligned data from two widely adopted radar sensors– Texas Instruments (TI) and Bosch. The dataset includes three radar data representations from TI: raw ADC data, heatmap, and point-cloud; and point clouds from Bosch. The data was captured in both indoor and outdoor scenarios. These scenarios were designed to reflect challenges of real-world applications, such as presence of large metallic objects that can increase radar sensor noise, sensor motion, and multiple people in the scene. We overview state-of-the-art open-access datasets and benchmarks for radar-based 3D HPE and evaluate two popular point-cloud-based methods and the most recent heatmap-based method. We introduce necessary algorithmic adaptations to improve their performance. Finally, we propose a new RadarViT method that is more light-weight and outperforms the existing methods with respect to the mean per joint position error. We tested this model in outdoor conditions with and without re-training, in the presence of multiple persons, and under dynamic conditions. The dataset and the code will be made publicly available.