Symbolic Modeling of Single-Arm Demonstrations Using Probabilistic Methods in Learning from Demonstration

Abstract

Robots are highly reconfigurable machines with applications in manufacturing, healthcare, and many other domains. However, programming them for new tasks typically requires sufficient robotics expertise and is often time-consuming. Learning from Demonstration offers an intuitive and practical alternative to traditional programming methods by replacing the tedious programming process with a workflow in which the user simply demonstrates the desired task to the robot system. However, many existing Learning from Demonstration approaches lack full interpretability in the learning outcome and can suffer from the black box effect, wherein the learning process remains opaque to the user, thus making the system’s behavior difficult to analyze and debug. This further weakens the intuitiveness of the system.

This thesis investigates how symbolic Learning from Demonstration can be used to encode unimanual human manipulation demonstrations as interpretable, parameterized high-level action plans, such that the demonstrated tasks can later be replicated with a robot system incorporating an anthropomorphic robot hand. To achieve this, the proposed approach extends an existing probabilistic Learning from Demonstration framework based on Optimized Multivariate Multi-Order Markov Models through a ROS2-based system implementation with the following key contributions: a synchronized multimodal demonstration capture system for recording hand pose, finger joint angles, exerted forces, and visual cues; a demonstration preprocessing pipeline including, for example, grasp type recognition and 6D pose tracking of manipulated objects; new symbolic input variables for OM3M; enhanced segmentation algorithm for OM3M; extended optimization algorithm for OM3M; parameterized high-level action plans; and robotic execution with an anthropomorphic hand.

The developed system was evaluated both qualitatively and quantitatively. The qualitative evaluation confirmed stable performance despite certain limitations. The quantitative evaluation confirmed promising results within evaluated scope, with 99.52% grasp type recognition accuracy, 99.5% grasp direction recognition accuracy, 98.57% recognition accuracy for single-operation demonstrations, 85.38% fully correct segmentation accuracy for multi-operation demonstrations, and 95% success rate in robotic execution trials. The results demonstrate the feasibility of the proposed system in both symbolic encoding and robotic execution, indicating that symbolic Learning from Demonstration can support the transformation of unimanual human manipulation demonstrations into interpretable, parameterized, and robot-executable task representations.