Person Tracking and Region-of-Interest Generation for Human-Robot Interaction: Design and Evaluation of a ROS2 Pipeline for Indoor Scenarios

Abstract

Robots that operate alongside people must continuously know where those people are, making robust person detection and tracking a core requirement for safe and effective human-robot interaction (HRI). Although many person tracking algorithms exist, deploying them in practical indoor HRI settings remains challenging. Identity switches, bounding box jitter, and missed detections all undermine downstream robot behavior, and there is no standardized notion of what constitutes stable tracking or reliable region-of-interest (ROI) generation in this context.

This thesis therefore focuses on tracking accuracy together with ROI stability metrics, which existing benchmarks rarely address. This thesis develops and evaluates a ROS2-based person tracking and ROI generation pipeline built around a ZED 2i stereo camera and the ZED SDK body-tracking module, running on an NVIDIA Jetson AGX Orin. The pipeline publishes 3D positions, persistent tracking IDs, and 2D bounding boxes over a standardized ROS2 interface for downstream robot applications. Performance was evaluated against tracking KPIs from the CLEAR MOT family (MOTA, MOTP, ID switches, fragmentations) and ROI stability metrics (jitter, size stability, frame rate). Four indoor scenarios were tested in nine pipeline configurations: single-person approach, partial occlusion, two-person crossing, and a five-person crowded scene.

The pipeline tracked one or two people with high accuracy and stable identity (MOTA ≥ 0.94, essentially zero ID switches) at 9–12 FPS, including under partial occlusion. A reduced-cost configuration combining ULTRA depth, the HUMAN_BODY_FAST detector, body fitting, and the BODY_34 skeleton (FASTER) Pareto-improved over the NEURAL_PLUS baseline: throughput roughly doubled (6.1 → 11.7 FPS) and mean ROI jitter dropped by a third with no loss in tracking accuracy. ROI smoothing and a raised confidence threshold gave situational rather than universal gains. The five-person crowded scene exposed the pipeline’s main weakness, with MOTA falling to 0.58 due to detection dropouts when overlapping silhouettes were not separated. Within the one- and two-person regime, which covers the majority of close-range indoor HRI use, the pipeline meets the stability and real-time requirements set out in the thesis and is ready for integration via its ROS2 interface.