Coordination-free decentralised federated learning in pervasive networks: Overcoming heterogeneity

Abstract

Fully decentralised federated learning enables collaborative model training among edge devices without relying on a central coordinator, thereby avoiding single points of failure and supporting spontaneous collaboration in pervasive environments. However, the absence of coordination introduces challenges that go beyond data heterogeneity alone. In realistic decentralised settings, devices often start from different model initializations, possess limited and non-IID local data, and interact over unstructured communication graphs, making naive parameter averaging ineffective and potentially destructive. In this paper, we address decentralised learning under combined data and initial model heterogeneity by proposing DecDiff+VT, a coordination-free decentralised learning algorithm specifically designed for such environments. DecDiff+VT integrates two complementary mechanisms: DecDiff, a disruption-aware aggregation strategy that updates local models towards their neighborhood average with a magnitude inversely proportional to model disagreement, and a lightweight virtual teacher (VT) mechanism based on soft-label regularization to improve local generalization in the absence of strong or centralized teacher models. Extensive experiments on image classification and activity recognition benchmarks (MNIST, Fashion-MNIST, EMNIST, CIFAR-10, and UCI-HAR) show that DecDiff+VT consistently outperforms or matches state-of-the-art decentralised baselines, achieving faster convergence, improved generalization, and greater robustness to overfitting, without incurring additional communication or memory overhead compared to standard decentralised averaging.