Titre : Federated Learning Can Find Friends That Are Advantageous and Help with Low-Resource Machine Translation
Résumé : In Federated Learning (FL), the distributed nature and heterogeneity of client data present both opportunities and challenges. While collaboration among clients can significantly enhance the learning process, not all collaborations are beneficial; some may even be detrimental.In this talk, I will discuss our novel algorithm that assigns adaptive aggregation weights to clients participating in FL training, identifying those with data distributions most conducive to a specific learning objective.As I will explain during the talk, the proposed aggregation method converges no worse than the method that aggregates only the updates received from clients with the same data distribution.Furthermore, empirical evaluations consistently reveal that collaborations guided by the proposed algorithm outperform traditional FL approaches.Moreover, in the second part of my talk, I will explain how the proposed approach can be adjusted to the training of LLMs on Low-Resource languages
Présentation :
Conférencier : Eduard Gorbunov est chercheur au MBZUAI (où il a travaillé en tant que postdoctorant), travaillant auprès de Samuel Horváth et Martin Takáč. Auparavant, il a travaillé comme chercheur junior au MIPT et comme consultant en recherche (postdoc à distance) à Mila dans le groupe de Gauthier Gidel. Il a obtenu son doctorat au MIPT, Phystech School of Applied Mathematics and Informatics, où il a travaillé sous la supervision des professeurs Alexander Gasnikov et Peter Richtárik. Eduard s’intéresse à l’optimisation stochastique et à ses applications à l’apprentissage automatique, à l’optimisation distribuée, à l’optimisation sans dérivée et aux inégalités variationnelles. Eduard s’intéresse à des sujets comprennent l’optimisation stochastique et ses applications à l’apprentissage automatique, l’optimisation distribuée, l’optimisation sans dérivée et les inégalités variationnelles.
Bibliographie : N. Tupitsa, S. Horváth, M. Takáč, E. Gorbunov. Federated Learning Can Find Friends That Are Advantageous, arXiv preprint, https://arxiv.org/abs/2402.05050
V. Moskvoretskii, N. Tupitsa, C. Biemann, S. Horváth, E. Gorbunov, I. Nikishina. Low-Resource Machine Translation through the Lens of Personalized Federated Learning, EMNLP 2024 (Findings), https://arxiv.org/abs/2406.12564
