Anomaly detection for communication network monitoring applications

Abstract

Functioning mobile telecommunication networks are taken for granted in present-day society. The network operator’s objective is to optimise the network’s capabilities in order to provide fluent connections for subscribers. Network management is based on the huge amounts of data that are recorded from all parts of the network. The data is used to monitor performance, to detect problems and also to provide novel knowledge to be used in future planning. Anomalous events in the network provide a valuable source of information for network management. This thesis presents an interpretation of anomalies and the basic theory of how to detect them when the probability distribution is known. However, since in real life applications the probability distribution is not known, the main focus is on methods that are based on distances. This thesis proposes procedures for anomaly detection and for summarising the information obtained about the anomalies. The procedures utilise clustering in both the anomaly detection and the further analysis of the anomalies. Scaling of variables affects the distances and the results of clustering. Therefore, methods to incorporate expert knowledge by application specific scaling of variables are presented. The proposed procedures are exemplified in three use cases. The cases present practical problems from distinct parts of the network; the radio interface between the mobile device and the network, the system logs from the operator’s servers, and the traffic through the cells. Each case presents unique characteristics and challenges. The problems are solved utilising the proposed procedures. Two novel anomaly detection methods developed in this thesis are applied in the second case, where anomaly detection is applied to server logs. All use cases use real data from commercial networks where the ground truth does not exist. Therefore, precise comparisons of the methods are impossible. The results have been verified with network experts and found to be informative and useful.