Automatic Bucket Filling for Wheel Loaders : A Machine Learning Approach

Abstract

The bucket filling task for wheel loaders is one of the most common tasks, which is normally repeated in cycles during a work shift. It is also the most difficult task for the operator; they need years of experience to do it efficiently. These facts, combined with the impending labor shortage in the industry, make this task compelling for automation. Automating the bucket filling task will lower the barrier-of-entry for new operators by making them more efficient with less training. Additionally, automating the bucket filling task has the potential to increase productivity at a worksite and reduce fuel consumption.

There are several reasons that make bucket filling a challenging automation problem. The biggest one is the interaction forces between the pile and the wheel loader’s bucket. These forces are difficult to model because of uncertainties in the materials’ properties. As a result, it is difficult to create an optimal controller based on these principles. A machine learning approach does not have to model the interaction forces explicitly; it learns them instead from data recorded by human expert operators.

The main goal of this thesis is to investigate a machine learning approach to the bucket filling task using imitation learning, transfer learning, and reinforcement learning. A secondary goal is to develop an efficient algorithm for extracting the bucket filling task from long time-series data to eliminate the burden of manual data labeling. These goals are pursued in five publications, which serve as the basis for this thesis.

Loading a single material was achieved using imitation learning with data recorded from expert operators. The synthesized controller could achieve human-level performance on the test pile.

This approach was expanded by using transfer learning to load multiple materials. Transfer learning only requires a small amount of additional data for transferring a controller to a new material. This method proved successful in loading all target materials with human-level performance. The target materials were sand, gravel, and blasted rock. Reinforcement learning was also applied as a similar strategy for multiple material loading. In contrast to the previous method, reinforcement learning loads the bucket several times and explores different loading strategies on its own to optimize its performance.

Lastly, a surrogate simulator was developed in this thesis to serve as faster than real-time simulator for optimizing a bucket filling loading technique using reinforcement learning. The optimized controller was deployed directly to the machine and evaluated against the best performing previously develop controllers.

To conclude, the conducted experiments indicated that machine learning is a feasible approach to the bucket filling task for full-sized wheel loaders and that it is possible to synthesize robust controllers with high performance for various materials.