Predictive Autoscaling for Kubernetes Using Machine Learning Workload Forecasting

Abstract

The Kubernetes Horizontal Pod Autoscaler (HPA) is reactive: it scales only after resource thresholds are breached. Because the metric-scrape interval (15--30 s), the HPA reconciliation cycle (15--45 s), and pod cold-start time (60--120 s) execute sequentially, three to five minutes typically elapse between a traffic spike and the availability of new capacity. During this window, existing pods absorb excess load, causing tail-latency degradation and service-level objective (SLO) violations. This reactive lag is the central limitation that predictive autoscaling aims to remove. A further limitation is that the HPA bases decisions on average CPU utilization, which can mask localized overload: if one pod is saturated while others are idle, the fleet-wide average remains low and no scaling action is triggered.

This thesis proposes a proactive autoscaling system that forecasts workload demand and provisions pods before load arrives, eliminating the reactive lag. Three prediction models are compared: Facebook Prophet, which captures daily and weekly seasonality through additive decomposition; a two-layer Long Short-Term Memory (LSTM) network, which models non-linear temporal dynamics; and a multivariate Prophet-LSTM hybrid in which Prophet's forecast is fed to the LSTM as a parallel input channel alongside the raw rate. Related work on Kubernetes workload prediction reports that such hybrid models improve prediction accuracy by 65--90 % in mean squared error compared to single-model baselines.

The evaluation is split into an offline and an online stage. The offline stage trains each model on the FIFA World Cup 1998 HTTP access logs at a single fresh cutoff (July 7) and evaluates prediction quality on a three-day held-out window using MAPE, RMSE, MAE and R². A separate drift probe re-evaluates the same models 14 days past their cutoff, using the gap between training and evaluation as a proxy for staleness that does not require weeks of live cluster time. The online stage takes the best-performing model, bakes it into a custom autoscaler controller, and replays a 30-minute match-day spike window on a k3s cluster three times under identical conditions: once with the reactive HPA, once with the predictive controller using the fresh hybrid, and once with the predictive controller using the 14-day-stale hybrid. A k6 load generator is driven by a small evaluation-clock service that owns the dataset's notion of time so the controller's inference stays aligned with the workload being replayed. System-level effects (response latency percentiles, SLO violation rate, pod count over time, CPU utilisation) are recorded from the ingress controller and Kubernetes metrics. Offline, both the LSTM and the hybrid reach single-digit prediction error (4.2 % MAPE at a five-minute horizon), and the LSTM-based models tolerate 14-day staleness while Prophet alone collapses. Online, the predictive controller cuts the SLO violation rate from 53 % under the reactive HPA to 5 % and lowers median p95 latency from 619 ms to 495 ms, at the cost of a modest increase in mean replica count (17.5 to 20.1). Continuous retraining and long-term drift adaptation are deliberately out of scope and are identified as follow-up work.