Comparison of computer vision algorithms for the manipulation of deformable linear objects

Abstract

With the rise of smart factories, robots need to be able to handle increasingly complex tasks with minimal human intervention. Not only will this increase the efficiency and safety of manufacturing facilities but minimize human error in manufacturing processes. A challenging task that has yet to be fully automated is the robotic manipulation of deformable linear objects (DLOs) such as cables and hoses. This is due to the complex nature of deformation, that requires robots to be able to adapt to the objects changing shape, state, and orientation. Though the changes in DLOs can be detected with different sensors such as tactile sensors or force sensors, this thesis focuses on utilizing vision sensor data for detection. For a robot to be able to manipulate DLOs, the robot needs information such as coordinates of key points. This information can be extracted of image data through object segmentation, though traditional methods often fall short due to their limited adaptability to the dynamic changes of DLOs. The evolution of increasingly better computational power, computer vision algorithms and machine learning have made it easier to detect and segment DLOs in different states. Several computer vision algorithms have been developed to tackle this problem for different applications and use cases. The objective of this thesis is to review and analyze the current state-of-the-art computer vision algorithms for DLO detection and segmentation. This is done through a literature review and a performance analysis. Furthermore, a proof-of-concept (PoC) application is designed and developed that integrates the most suitable algorithm for the robotic manipulation of DLOs. The PoC application utilizes the segmentation results received from the DLO segmentation algorithm and calculates coordinates of grasping points that can be sent to a robot’s motion planner. The application was developed utilizing Robot Operating System (ROS) and consists of a server node and a client node. The client node acts in place of a robot’s motion planner, while the server node consists of a calibration service and coordinates service. The performance evaluation of the PoC application shows that coordinates of DLO grasping points can be accurately determined with the chosen DLO segmentation algorithm.