Multi-Task Model Personalization for Federated Supervised SVM in Heterogeneous Networks

Abstract

Federated systems enable collaborative training on highly heterogeneous, non-i.i.d. data through model personalization, which can be facilitated by employing multi-task learning. However, multi-task learning algorithms are often implemented using methods like stochastic gradient descent, which may suffer from slow convergence in a multi-task federated setting. To accelerate the training procedure, we design an efficient iterative distributed method based on the alternating direction method of multipliers (ADMM) for support vector machines (SVMs), which tackles federated classification and regression. The proposed method utilizes efficient computations and model exchange in a network of heterogeneous nodes and allows personalization of the learning model in the presence of non-i.i.d. data. To ensure data privacy, we introduce a randomization algorithm that helps avoid data inversion. Finally, we analyze the impact of the proposed privacy mechanisms and participant hardware and data heterogeneity on the system performance. Our experiments confirm the advantages of the proposed ADMM-based personalized federated multi-task learning.