Methods for Classifying Excavator Motions in Long-Term Activities

Abstract

This thesis describes the research work done for Novatron ltd. during the summer and fall of the year 2021. The purpose of the research was to find and compare methods for automatic motion and activity recognition for excavators using Novatron machine control system sensors and data collection tools. A new dataset was collected using a single excavator that was operated in a private testing ground by two different operators. Three different machine learning classifiers (decision tree, random forest and support vector machine) were trained and tested with the collected dataset. The data was divided into three separate data subcategories (orientation, pressure and joystick data) which were used to test how different data modalities and window sizes affect the prediction results. The training data was annotated manually using two different labeling definitions; the low level and the high level labels. Both labeling systems contained the same two idle movement and idle stationary classes. The other low level labels were swing, scooping, dumping and leveling and the high level classes were excavation, general work and detailed work. The low level labels represented temporally shorter and simpler excavator motions and the high level labels temporally longer and more complex actions.

The performance for all models was compared to each other in relation to multilabel and single label prediction accuracy, precision and recall. A 5-fold cross validation test for the multilabel accuracy score was also calculated for each classifier. For the high level of labeling an accuracy of 81\%, a precision of 90\% and recall of 82\% was achieved for multilabel prediction with the random forest classifier trained with four second orientation data windows. For the low level labeling accuracy of 79\%, precision of 88\% and recall of 82\% were gained with a random forest classifier trained with two second orientation data windows. The label-specific prediction performance was significantly better with over 90\% performance metrics for almost all individual labels. The 5-fold cross validation test results were in line with the multilabel prediction performances gained with a single train-test dataset split. The models trained with the joystick data had the smallest deviation in accuracy while the median accuracies did not differ greatly from the results gained with models trained with orientation data. Good results were achieved with all three measurement subcategories, but the models trained with the orientation and joystick datasets performed slightly better than the models trained with the pressure dataset.