Leveraging Compute Heterogeneity in Federated Multi-Task Classification

Abstract

One of the main challenges of federated learning (FL) algorithms is resource heterogeneity, which may prevent participants with limited computing capabilities from being able to effectively participate in the learning process. The presence of such participants can significantly impede the training process and cause notable degradation in the overall system performance. In this paper, we propose a set of policies for leveraging computing heterogeneity, with the aim of accelerating the training of federated multi-task classification based on support vector machine (SVM). We evaluate the effectiveness of the proposed policies in various regimes and draw conclusions on their applicability to different scenarios. Our results indicate a significant improvement in training time and model performance, especially in cases where the computing resources are highly heterogeneous.