Stage - Training networks to be compressible: low rank, quantization, and how they combine-Saclay-H/F

As an intern at the CEA, you will have the opportunity to work in a world-renowned research environment. Our teams consist of passionate and dedicated experts, providing an environment conducive to learning and collaboration. You will have access to state-of-the-art equipment and top-tier research resources to carry out your assignments. The work performed may potentially lead to a scientific publication. Context: This internship aims to understand how training a network toward low rank changes what it costs to quantize its factors, using the team's closed-form price of rounding both to explain the effect and to act on it. It combines matrix analysis with controlled experiments on language models, to: What do we expect from you?
• Measure what low-rank preparation does to a model's compressibility, by factorization and by quantization, against an unprepared model and against quantization alone, at equal memory.
• Explain it: what the preparation changes in the factors, read through the closed-form price, and whether the price predicts the measured cost.
• Use the price to propose and test an improvement. The internship may lead to a PhD starting in October 2027. # Cea List Moyens / Méthodes / Logiciels Pyyhon
- PyTorch Profil du candidat Profile:
• Master's (M2) or engineering-school student in machine learning, applied mathematics or computer science
• Strong linear algebra; probability and optimization are a plus
• Solid Python and PyTorch
• Care in designing experiments and reading their results
• Experience with Hugging Face language models is a plus Conformément aux engagements pris par le CEA en faveur de l'intégration des personnes handicapées, cet emploi est ouvert à toutes et à tous. Le CEA propose des aménagements et/ou des possibilités d'organisation pour l'inclusion des travailleurs handicapés. Localisation du poste Site Saclay Localisation du poste France, Ile-de-France, Essonne (91) Ville Saclay Critères candidat Langues Anglais (Intermédiaire) Diplôme préparé Bac+5
- Diplôme École d'ingénieurs Formation recommandée Master 2 ou équivalent Possibilité de poursuite en thèse Oui Demandeur Disponibilité du poste 01/02/2027