Human-robot collaboration by speech in an industrial assembly task

Abstract

Collaborative robots and human-robot collaboration are gaining increasing interest in the industrial sector as they enable flexible and efficient production. To make the most of the abilities of both the robot and the human operator while working towards shared goals, effective communication and interaction between them are essential. In current applications of collaborative robots, these are often limited, leading to ineffective and unintuitive collaboration. Speech is a very promising communication method for human-robot collaboration as it is natural for the human operator and allows bidirectional communication. This thesis presents an implementation of a collaborative robotic system that combines speech recognition, text-to-speech, object detection and robot control. Object detection provides the robot with valuable information about its environment, while the speech-related functionalities enable both human-robot and robot-human communication. These different features are first reviewed through literature, and then their implementations are presented. Both robot control and communication between the different components utilize ROS (Robot Operating System), allowing the system to be independent of the robot brand. The implemented system is tested in various assembly-related test cases. In addition to common picking, placing and handing over of objects, more advanced functionalities such as quality inspection, dialog and robot giving instructions, are presented. These emphasize the advantages of speech as a communication method for this kind of assembly tasks. In conclusion, this thesis presents how speech can be utilized for two-way communication to enhance collaboration between human and robot. It offers a case study on this growing topic, laying the groundwork to be extended in other applications. Additionally, it paves the way and acknowledges the need for future work in this field.