Automating system error diagnosis using Bidirectional Encoder Representation from Transformers (BERT)

Abstract

Anomaly detection plays a vital role in modern industrial systems by enabling the early identification of potential failures. This thesis focuses on analyzing system and server logs from a port terminal, which record the complete container movement cycle, from offloading at the quay crane to final loading onto trucks. As terminal operations scale and become increasingly automated, the volume of log data generated within short operational periods becomes too large for manual inspection to be practical. Operational failures in container movement cycles, such as repeated picks without completion, misplacements, or interruptions caused by coordination issues, can result in significant delays and inefficiencies. These failures often lead to increased costs due to manual interventions, equipment idle time, and task rescheduling across interconnected systems. The objective of this research is to predict container movements that are likely to lead to such failures by constructing a unified representation of logs collected from multiple systems. By analyzing temporal and contextual correlations across these logs, a deep learning model is trained using a masked language modeling approach. To accurately capture the structural characteristics of port terminal logs, we introduce custom embedding layers that incorporate sequence-level information. This enhanced model, called Long Sequence BERT (LS-BERT), learns patterns from extremely long log sequences and uses them to detect anomalies in container movement cycles. By automating the identification of potential issues, this approach supports maintenance teams in efficiently pinpointing problem cases, allowing them to focus on preparing Root Cause Analyses (RCA) rather than spending time manually reviewing extensive log data.