Multi-Modal Interface for Offline Robot Programming

Abstract

This thesis presents an approach for improving robot offline programming using input methods based on the human natural skills. The approach is focused to teach basic assembly and manipulation operations using a pair of industrial robots in an existing simulation environment and is meant to be improved in future works, that are also proposed in this thesis.

In order to develop this approach, and regarding the available resources, an Add-In for the simulation and offline programming software RobotStudio was developed. This Add-In combines human pose, a graphical user interface and optionally speech to teach the robot a sequence of targets, along with the simulation environment, to automatically generate instructions.

Two different kinds of sensors, Kinect and Leap Motion Sensor have been evaluated based on references in order to select the most suitable one for the implementation of this work.

The executions of the programmed instructions have been evaluated in simulation.