PhD Position F/M Privacy-Preserving Collaborative Learning of Large Language Models Across Heterogeneous Learning Paradigms

Le descriptif de l'offre ci-dessous est en Anglais

Type de contrat : CDD

Niveau de diplôme exigé : Bac + 5 ou équivalent

Fonction : Doctorant

A propos du centre ou de la direction fonctionnelle

The Inria University of Lille centre, created in 2008, employs 360 people including 305 scientists in 16 research teams. Recognised for its strong involvement in the socio-economic development of the Hauts-De-France region, the Inria University of Lille centre pursues a close relationship with large companies and SMEs. By promoting synergies between researchers and industrialists, Inria participates in the transfer of skills and expertise in digital technologies and provides access to the best European and international research for the benefit of innovation and companies, particularly in the region.

For more than 10 years, the Inria University of Lille centre has been located at the heart of Lille's university and scientific ecosystem, as well as at the heart of Frenchtech, with a technology showroom based on Avenue de Bretagne in Lille, on the EuraTechnologies site of economic excellence dedicated to information and communication technologies (ICT)

Contexte et atouts du poste

The widespread deployment of Large Language Models (LLMs) has given rise to diverse adaptation paradigms that accommodate varying computational and infrastructural constraints. In addition to traditional full fine-tuning, emerging methods such as prompt-tuning, parameter-efficient tuning like LoRA (Low-Rank Adaptation), and in-context learning for model adaptation with reduced computational resources or without access to model weights. These approaches open up new possibilities for collaborative learning in privacy-sensitive contexts, where multiple clients aim to improve LLM performance without exposing their raw data.

This PhD thesis will focus on designing privacy-preserving collaborative learning strategies for LLMs, starting in a homogeneous setting, where all participants rely on the same adaptation paradigm. This initial step will build a foundation for tackling the more ambitious and impactful goal of heterogeneous collaboration, where clients operate under different adaptation regimes due to diverse privacy, computational, or architectural constraints. A central challenge of the project is to reconcile contributions from such heterogeneous clients in a unified learning process, while ensuring rigorous privacy guarantees—most notably through differential privacy (DP), which provides strong theoretical protections against data leakage. The thesis will also address the trade-offs between model utility and privacy risk and propose novel mechanisms specifically tailored to this multi-paradigm collaborative learning scenario.

The PhD candidate will be based at Inria Lille, within the MAGNET research team, and will be co-supervised by M. Tommasi, Dr. Raouf Kerkouche (Inria Lille) and Dr. Cédric Gouy-Pailler (CEA Saclay). The research will benefit from a stimulating scientific environment, combining Inria's strong expertise in machine learning and artificial intelligence with the applied research focus of the CEA. This thesis is part of the REDEEM project, funded by the PEPR IA initiative (France It offers a highly interdisciplinary environment bridging machine learning, natural language processing, and privacy-enhancing technologies, with opportunities for national and international collaboration.

Mission confiée

The person hired will carry out original research toward a PhD on "Privacy-Preserving Collaborative Learning of Large Language Models Across Heterogeneous Learning Paradigms." The research will involve designing novel collaborative protocols, formalizing privacy guarantees, and evaluating the impact of different learning paradigms on performance and privacy.

Principales activités

The candidate will get acquainted with the state of the art on privacy-preserving collaborative learning and adaptation of Large Language Models (LLMs), perform original research in close interaction with the thesis supervisors and other collaborators, and design collaborative learning strategies across heterogeneous adaptation paradigms such as full fine-tuning, prompt-tuning, and in-context learning. A key part of the work will involve developing and analyzing privacy-preserving mechanisms—particularly those based on differential privacy, which offers strong theoretical guarantees against data leakage. The candidate will evaluate these mechanisms in terms of their utility-privacy trade-offs, write scientific articles detailing the results, and present the work at top-tier international conferences and leading peer-reviewed journals in the areas of machine learning, privacy, and natural language processing.

Compétences

• Master's degree in computer science, artificial intelligence, machine learning, natural language
  
  processing, applied mathematics, or a related field.
• Solid foundations in machine learning and natural language processing (NLP), including familiarity
  
  with training and fine-tuning large language models (LLMs), as well as an understanding of recent
  
  trends such as prompt-tuning and in-context learning.
• Experience or strong interest in collaborative learning.
• Familiarity with privacy-enhancing technologies, in particular differential privacy, is a plus.
• Strong programming skills, preferably in Python, and experience with machine learning frameworks
  
  such as PyTorch or TensorFlow.
• Excellent oral and written English proficiency.

Avantages

• Subsidized meals
• Partial reimbursement of public transport costs
• Leave: 7 weeks of annual leave + 10 extra days off due to RTT (statutory reduction in working hours) + possibility of exceptional leave (sick children, moving home, etc.)
• Possibility of teleworking and flexible organization of working hours
• Professional equipment available (videoconferencing, loan of computer equipment, etc.)
• Social, cultural and sports events and activities
• Access to vocational training
• Social security coverage

Rémunération

2 300 € monthly gross salary

Informations générales

• Thème/Domaine : Optimisation, apprentissage et méthodes statistiques

Calcul Scientifique (BAP E)
- Ville : Villeneuve d'Ascq
- Centre Inria : Centre Inria de l'Université de Lille
- Date de prise de fonction souhaitée :
- Durée de contrat : 3 ans
- Date limite pour postuler :

Attention: Les candidatures doivent être déposées en ligne sur le site Inria. Le traitement des candidatures adressées par d'autres canaux n'est pas garanti.

Consignes pour postuler

Please send your CV and cover letter

Please follow the guidelines described on for your application

Sécurité défense :

Ce poste est susceptible d'être affecté dans une zone à régime restrictif (ZRR), telle que définie dans le décret n° relatif à la protection du potentiel scientifique et technique de la nation (PPST). L'autorisation d'accès à une zone est délivrée par le chef d'établissement, après avis ministériel favorable, tel que défini dans l'arrêté du 03 juillet 2012, relatif à la PPST. Un avis ministériel défavorable pour un poste affecté dans une ZRR aurait pour conséquence l'annulation du recrutement.

Politique de recrutement :

Dans le cadre de sa politique diversité, tous les postes Inria sont accessibles aux personnes en situation de handicap.

Contacts

• Équipe Inria : MAGNET
• Directeur de thèse :

Tommasi Marc /

A propos d'Inria

Inria est l'institut national de recherche dédié aux sciences et technologies du numérique. Il emploie 2600 personnes. Ses 215 équipes-projets agiles, en général communes avec des partenaires académiques, impliquent plus de 3900 scientifiques pour relever les défis du numérique, souvent à l'interface d'autres disciplines. L'institut fait appel à de nombreux talents dans plus d'une quarantaine de métiers différents. 900 personnels d'appui à la recherche et à l'innovation contribuent à faire émerger et grandir des projets scientifiques ou entrepreneuriaux qui impactent le monde. Inria travaille avec de nombreuses entreprises et a accompagné la création de plus de 200 start-up. L'institut s'efforce ainsi de répondre aux enjeux de la transformation numérique de la science, de la société et de l'économie.

---