Hybrid Controller for Robot Manipulators in Task-Space With Visual-Inertial Feedback

Abstract

This article presents a visual-inertial-based control strategy to address the task-space control problem for robot manipulators. To achieve this, a hybrid observer is introduced for visual-inertial odometry, enabling the control loop to be closed directly in Cartesian space by estimating the end-effector pose. The end-effector is equipped with an inertial measurement unit (IMU) and a stereo camera, which provide task-space feedback information to the proposed observer. A hybrid controller is then designed to utilize the estimated end-effector pose for accurate motion control. The Lyapunov stability theorem is employed to demonstrate that the resulting closed-loop system, using the proposed observer-based controller, is globally and asymptotically stable. Besides this notable merit (global asymptotic stability), the proposed method also reduces the impact of robot kinematics on the controller’s performance. Real-world experiments and computer simulations using a six-degrees-of-freedom (DoFs) long-reach hydraulic robot manipulator are conducted to evaluate the performance of the proposed observer-based control scheme.