Transfer Learning for Real-world Control of Heavy-duty Hydraulic Machines : On industrial loader cranes and wheel loaders

Abstract

Technology is evolving at a rapid pace, automating various aspects of modern life, from smart home systems and smartphones to autonomous vehicles. Breakthroughs in machine learning and reinforcement learning (RL), paired with advances in computational hardware, have achieved outcomes once deemed impossible, surpassing human experts in intricate tasks. These technological strides extend beyond specific fields and applications, and hold the potential to transform the heavy-duty machine industry, which faces many challenges including labor shortages, difficult working environments, and stricter emission standards. Traditional control systems are becoming inadequate and outdated, necessitating more advanced automation solutions to meet modern demands for productivity and efficiency.

To this end, this dissertation explores the potential of RL to enhance control systems for heavy-duty hydraulic machines. It proposes a data-driven approach for learning controllers, leveraging state-of-the-art advancements in model learning and policy optimization to achieve fast and high-quality training frameworks. The research focuses on developing accurate models for hydraulic actuators and feedforward control using real sensor data and integrates RL to create reliable, efficient, and competitive low-level and high-level automation solutions for commercial applications. Through transfer learning (TL), control systems learned on one machine are adapted to various industrial machines, ensuring flexibility and scalability. In addition, the dissertation emphasizes achieving accurate energy predictions and energy-efficient control in conventional hydraulic systems. Across four publications, this research elaborates on the methodologies for each development, demonstrating successful experiments with full-scale industrial wheel loaders and loader cranes. These findings highlight the practical applicability and effectiveness of the proposed RL/TL frameworks in advancing automation for heavy-duty hydraulic machines.