Visual perception with UR5 arm towards semantic manipulation

Abstract

Nowadays, the most popular impression about robotics is the factory assembly line of robotic arms working together. Although robotic manipulation has substantially grown in the last decade, creating robots capable of directly interacting with the surrounding is still a desire, where the robots could effectively and autonomously function and safely interact with the human. To achieve this, the robot must have the ability to understand its surroundings. From my perspective, the ontology is a good choice to be a robot knowledge base of the environment. With a knowledge base as the central server, the problem is how to make the system to be flexible for expanding purposes. This thesis presents an approach to tackle this problem by combining the ontology, web server, and ROS environment, which is used as a control system of robot and sensors. The target outcome is that when the system analyzes a current random scene, it can effectively detect the differences between the current scene and the target scene, i.e if there is an object that is not in the target scene, the robot needs to remove it from the current scene. As a result, the proposed system generates the equivalent actions and gives commands for the robot to act accordingly. Thanks to the above process, the robot successfully accomplishes the target tasks from random scenes.