Human Tracking and Activity Recognition with mmWave Radar
Minto, Md Robiul Islam (2020)
2020
Master's Programme in Electrical Engineering
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2020-12-04
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202011278255
https://urn.fi/URN:NBN:fi:tuni-202011278255
Tiivistelmä
Millimetre-wave (mmWave) is an extremely valuable sensing technology for the detection of objects and providing the range, velocity, and angle of these objects. A mmWave radar, having synergies with the multi-beam light detection and ranging (LiDAR) and cameras, has been con-sidered as a must-have sensor in the connected and autonomous vehicles (CAV) in the future intelligent transportation systems (ITS). Besides the traditional target detection and ranging functions, the mmWave radar is expected to perform more intelligent tasks to improve road safety, for example recognizing the targets, especially the vulnerable road users like pedestri-ans and cyclists. mmWave radars are also used in indoor environments due to its high capabil-ity of working in low visibility conditions, such as smoke and debris. mmWave radar can provide the exact location of the human presence in the indoor environment with very high accuracy.
The first part of the thesis addresses the radar basics and principles, followed by a detailed discussion on FMCW radar. Also, the first part describes the micro-Doppler (µ-D) in the radar system. The second part of the thesis concentrates on the machine learning basics followed by the detailed discussion on the convolutional neural networks (CNN), recurrent neural networks (RNN).
The third part of the thesis describes a simulation study of the µ-D signatures of the pedes-trian and cyclist based on mmWave vehicle radar and investigates the recognition capabilities through both the CNN, RNN and mixed convolutional and recurrent approach respectively. The result demonstrates the usability of the mmWave radar µ-D information and complementary with the video and laser data streams in the CAV auto-piloting. A paper ‘’Shallow Neural Net-works for mmWave Radar Based Recognition of Vulnerable Road Users’’ has been published in the IEEE Xplore with this simulation study.
The fourth part of the thesis concentrates on the experimental study of an object (human) detection in the indoor environment by using the Texas Instruments mmWave RADAR module. The experimental study results show the target movement in real-time by azimuth-static heatmap, range-doppler heatmap, and range-profile. The acquired data from the experiments are analyzed and demonstrated in the X-Y scatter plot which gives the analytical view of the target (human) movements.
The first part of the thesis addresses the radar basics and principles, followed by a detailed discussion on FMCW radar. Also, the first part describes the micro-Doppler (µ-D) in the radar system. The second part of the thesis concentrates on the machine learning basics followed by the detailed discussion on the convolutional neural networks (CNN), recurrent neural networks (RNN).
The third part of the thesis describes a simulation study of the µ-D signatures of the pedes-trian and cyclist based on mmWave vehicle radar and investigates the recognition capabilities through both the CNN, RNN and mixed convolutional and recurrent approach respectively. The result demonstrates the usability of the mmWave radar µ-D information and complementary with the video and laser data streams in the CAV auto-piloting. A paper ‘’Shallow Neural Net-works for mmWave Radar Based Recognition of Vulnerable Road Users’’ has been published in the IEEE Xplore with this simulation study.
The fourth part of the thesis concentrates on the experimental study of an object (human) detection in the indoor environment by using the Texas Instruments mmWave RADAR module. The experimental study results show the target movement in real-time by azimuth-static heatmap, range-doppler heatmap, and range-profile. The acquired data from the experiments are analyzed and demonstrated in the X-Y scatter plot which gives the analytical view of the target (human) movements.