Radar-based Human Pose Estimation
Nguyen, Trung Jr (2025)
2025
Tietojenkäsittelyopin maisteriohjelma - Master's Programme in Computer Science
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
Hyväksymispäivämäärä
2025-06-13
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202506127116
https://urn.fi/URN:NBN:fi:tuni-202506127116
Tiivistelmä
Human pose estimation is a perception task that can be used in various applications from entertainment such as movies, video games, to security purposes. For medical purposes, human pose estimation can be applied to support rehabilitation patients and detect falls in elderly and infants.
With the increasing awareness in privacy issue, radar has become a suitable choice for human pose estimation. It is not only capable of keeping the anonymity of a person but also invariant to many environmental conditions such as lighting, weather, etc. This thesis concentrates on using the Frequency Modulated Continuous-Wave (FMCW) millimeter-wave (mmWave) radar to perform 3D human pose estimation. The task is also limited to find 3D positions of key points that are associated with human joints for verifying the skeleton structure of the subject.
This work aims to build a pipeline for data collection and deep learning method evaluation for 3D human pose estimation using FMCW mmWave MIMO radar. The dataset consists of 4D radar tensors, 3D radar point clouds with a complementary of 3D Ouster LiDAR point clouds, and stereo images from ZED2i camera. As a result, the thesis succeed in collecting 33 sequences that contains more than 40000 synced frames. The recording is performed on the multi-modal sensor box of Tampere University’s AMM group. The development framework includes also methods for the sensor box calibration and a pipeline for multi-modal data pre-processing. For annotation, the OptiTrack MoCap camera system is used with the help of Tampere University’s CIVIT Labs.
Additionally, to discover the limitation of the current methods in radar-based human pose estimation application, this work also embarks on challenging those methods with different data set setup. These challenges can motivate a new approach in providing a flexible and robust in human pose estimation using radar data in specific and radar perception application in general.
With the increasing awareness in privacy issue, radar has become a suitable choice for human pose estimation. It is not only capable of keeping the anonymity of a person but also invariant to many environmental conditions such as lighting, weather, etc. This thesis concentrates on using the Frequency Modulated Continuous-Wave (FMCW) millimeter-wave (mmWave) radar to perform 3D human pose estimation. The task is also limited to find 3D positions of key points that are associated with human joints for verifying the skeleton structure of the subject.
This work aims to build a pipeline for data collection and deep learning method evaluation for 3D human pose estimation using FMCW mmWave MIMO radar. The dataset consists of 4D radar tensors, 3D radar point clouds with a complementary of 3D Ouster LiDAR point clouds, and stereo images from ZED2i camera. As a result, the thesis succeed in collecting 33 sequences that contains more than 40000 synced frames. The recording is performed on the multi-modal sensor box of Tampere University’s AMM group. The development framework includes also methods for the sensor box calibration and a pipeline for multi-modal data pre-processing. For annotation, the OptiTrack MoCap camera system is used with the help of Tampere University’s CIVIT Labs.
Additionally, to discover the limitation of the current methods in radar-based human pose estimation application, this work also embarks on challenging those methods with different data set setup. These challenges can motivate a new approach in providing a flexible and robust in human pose estimation using radar data in specific and radar perception application in general.