Emplois actuels liés à PhD Position F/M Incremental Deep Learning for Embedded Systems - Rennes, Bretagne - INRIA
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThis PhD position is part of the Adapting project, which focuses on designing adaptive embedded hardware architectures for AI. The goal is to design new incremental machine learning algorithms that can serve as use cases in the Adapting project for other researchers who will focus on the hardware architecture...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThis PhD position is part of the Adapting project, which focuses on designing adaptive embedded hardware architectures for AI. The goal is to design new incremental machine learning algorithms that can serve as use cases in the Adapting project for other researchers who will focus on the hardware architecture...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThis PhD position is part of the Adapting project, which focuses on designing adaptive embedded hardware architectures for AI. The goal is to design new incremental machine learning algorithms that can serve as use cases in the Adapting project for other researchers who will focus on the hardware architecture...
-
PhD Position in Incremental Deep Learning for Embedded Systems
il y a 2 semaines
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThis PhD position is part of the Adapting project, which focuses on designing adaptive embedded hardware architectures for AI. The goal is to develop new incremental machine learning algorithms that can serve as use cases in the Adapting project.Key ChallengesDesigning resource-efficient incremental learning algorithms...
-
PhD Position in Incremental Deep Learning for Embedded Systems
il y a 2 semaines
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThe PhD student will investigate incremental learning of foundation models on embedded systems. The research will focus on designing new resource-efficient incremental learning algorithms that can run on embedded systems with their associated resource and privacy constraints.Key ResponsibilitiesConduct comprehensive...
-
PhD Researcher in Foundation Model Incremental Learning
il y a 4 semaines
Rennes, Bretagne, France INRIA Temps pleinAbout This Opportunity INRIA is seeking a highly motivated PhD candidate to join our research team focused on the exciting field of foundation models and their deployment on embedded systems. This position offers a unique opportunity to contribute to cutting-edge research with real-world impact. Research Focus The primary objective of this PhD is to...
-
Researcher in Incremental Machine Learning for Edge AI
il y a 2 semaines
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesThis PhD position is part of the Adapting project, which focuses on designing adaptive embedded hardware architectures for AI. The goal is to develop new incremental machine learning algorithms that can serve as use cases in the Adapting project for other researchers who will focus on the hardware architecture design.Key...
-
Rennes, Bretagne, France INRIA Temps pleinContext and ObjectivesDeep Neural Networks (DNNs) are widely used in machine learning applications, but they are prone to hardware faults that can impact their accuracy and reliability. This PhD position aims to design and develop an optimized algorithm-level fault injection framework to assess the resiliency of DNN hardware accelerators to hardware faults,...
-
PhD Researcher: Reliable Deep Neural Network Hardware
il y a 4 semaines
Rennes, Bretagne, France INRIA Temps pleinINRIA is seeking a highly motivated PhD candidate to contribute to cutting-edge research in the field of reliable deep neural network hardware accelerators. Deep learning models are revolutionizing various industries, from healthcare and robotics to finance and entertainment. However, the increasing complexity and deployment of these models on edge devices...
-
Rennes, Bretagne, France INRIA Temps pleinContext and ObjectivesDeep Neural Networks (DNNs) are widely used predictive models in machine learning, but they require significant energy and resources. To address this, custom Artificial Intelligence (AI) hardware accelerators are being designed to support DNNs. However, these accelerators are prone to hardware faults (HW faults) that can cause...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionPhD Position in Continual Learning and Low-Precision ArithmeticInria Rennes - Bretagne Atlantique Centre is seeking a highly motivated PhD researcher to work on a project focused on continual learning and low-precision arithmetic for edge devices. The successful candidate will be part of the TARAN team and contribute to the development of...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionPhD Position F/M Exploring Low-Precision Arithmetic for Continual Learning Tasks on Edge DevicesInria Rennes - Bretagne Atlantique Centre is seeking a highly motivated PhD researcher to work on a project focused on continual learning and low-precision arithmetic for edge devices. The successful candidate will be part of the TARAN team and...
-
PhD Position F/M Reliable Deep Neural Network Hardware Accelerators
il y a 1 semaine
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesWe are seeking a highly motivated PhD student to join our research team at INRIA. The goal of this PhD project is to design and develop an optimized algorithm-level fault injection framework to assess the resiliency of Deep Neural Network (DNN) hardware accelerators to hardware faults. This framework will enable the...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionPhD Position in Continual Learning and Low-Precision ArithmeticWe are seeking a highly motivated PhD researcher to join our team at INRIA Rennes to work on a project focused on continual learning and low-precision arithmetic for edge devices.About the ProjectThe goal of this project is to investigate the performance impact of using...
-
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionPhD Position in Continual Learning and Low-Precision ArithmeticWe are seeking a highly motivated PhD researcher to join our team at INRIA Rennes to work on a project focused on continual learning and low-precision arithmetic for edge devices.About the ProjectThe goal of this project is to investigate the performance impact of using...
-
Rennes, Bretagne, France INRIA Temps pleinContext and ObjectivesDeep Neural Networks (DNNs) are widely used predictive models in machine learning, but they require significant energy and resources. To address this, custom Artificial Intelligence (AI) hardware accelerators are being designed to support DNNs. However, these accelerators are prone to hardware faults (HW faults) that can cause...
-
Rennes, Bretagne, France INRIA Temps pleinContext and ObjectivesDeep Neural Networks (DNNs) are widely used predictive models in machine learning, but they require significant energy and resources. To address this, custom Artificial Intelligence (AI) hardware accelerators are being designed to support DNNs. However, these accelerators are prone to hardware faults (HW faults) that can cause...
-
PhD Position F/M Reliable Deep Neural Network Hardware Accelerators
il y a 1 semaine
Rennes, Bretagne, France INRIA Temps pleinJob DescriptionContext and ObjectivesWe are seeking a highly motivated PhD student to join our research team at INRIA. The goal of this PhD project is to design and develop an optimized algorithm-level fault injection framework to assess the resiliency of Deep Neural Network (DNN) hardware accelerators to hardware faults. The successful candidate will work...
-
Rennes, Bretagne, France INRIA Temps pleinAbout the PositionWe are seeking a highly motivated and talented PhD candidate to join our team at INRIA Rennes-Bretagne Atlantique Centre. The successful candidate will have the opportunity to work on a cutting-edge research project focused on exploring low-precision arithmetic for continual learning tasks on edge devices.Research ContextContinual learning...
-
PhD Position in Trustworthy AI Hardware Architectures
il y a 2 jours
Rennes, Bretagne, France Inria Temps pleinAbout the PhD PositionWe are seeking a highly motivated PhD student to join our research team at Inria Rennes - Bretagne Atlantique Centre. The successful candidate will work on the impact of hardware faults on AI decisions and algorithms developed to explain AI models.Research ContextWith the growing need to distribute AI applications from the cloud to edge...
PhD Position F/M Incremental Deep Learning for Embedded Systems
Il y a 3 mois
Contexte et atouts du poste
Context
This PhD will occur in the context of the project Adapting ( from
the PEPR AI ( This project focuses on designing adaptive embedded
hardware architectures for AI. In this context, our team wants to design new incremental machine learning
algorithms that could serve as use cases in the Adapting project for other researchers who will focus on the
hardware architecture design.
Continual learning , also known as lifelong learning or incremental learning, is a machine learning
paradigm that focuses on the ability of a model to learn and adapt continuously over time as it encounters
new data, tasks, or concepts, without forgetting or catastrophically overwriting what it has previously learned.
In continual/incremental learning, the learned model should retain knowledge about previous tasks or data
while incorporating new information. In this PhD, we will focus on designing new resource-efficient
incremental learning algorithms that can run on embedded systems with their associated ressource and
privacy constraints. These contraints involve limited computational power, memory, and energy efficiency.
They also involve real-time processing with low latency and often deterministic behavior. Updating
embedded models is complex due to hardware limitations and the need for efficient updates while handling
data locally to enhance privacy and security.
This PhD will focus on foundation models such as well-known LLM -Large Language Models- (e.g. GPT-3.5,
Mixtral, Llama 3,...) or multimodal ones (involving for example ViT -Vision Transformer- models such as
GPT-4o, Sora, Dall-E 3) and their ability to evolve continuously in an embedded environnement.
Application process
The position is funded for 3 years (this is the standard duration of a PhD in France). The net salary is around
$2000$ euros. The PhD student will be based in Rennes ( and will make
a few stays in Grenoble during the 3-years contract.
Applications will be processed on a first-come, first-served basis until June 15, 2024.
Application Material and Procedure
Here is the supervision team:
Applicants should send these documents to the entire supervision team :
An academic CV. An official transcript listing classes taken and grades received. A short statement (maximum of 2 pages) that explains what your research interests are and whyyou would like to work with us. A copy of your most recent English report (e.g. your master report). Any published and relevant research papers (in English, preferably PDF format). If you have several
papers, please indicate which ones you consider the three most important ones. A list of references (e.g., supervisors)
Mission confiée
Related work
Incremental learning consists in a multi-stage training in which the context (data domain, classes, or task)
evolves between each training stage. This paradigm faces the stability-plasticity dilemma. It means that the
model must continuously adapt to new contexts while avoiding catastrophic forgetting (performance on past
contexts must not deteriorate). The naive approach is the standard fine-tuning strategy in which the
parameters of the model (or part of it) are updated from one stage to another by training on the new context
only. It is known to be efficient for adaptation but it is prone to catastrophic forgetting. Several works focused
on replay strategies to tackle this challenge. It consists in managing a buffer of examples from past
contexts, which are preserved through the different training stages. Alternatives have been
proposed to avoid storing old data, for instance by learning to preserve the latent representation on the new
data through the training stage . However, current incremental foundation models do not include important
desiderata for embedded models (e.g. lack of privacy, restricted hardware resources and low energy
consumption, robustness, no need for an oracle or neurogenesis). Our current project will explore the models
[1, 2, 3,4,5,6,7] and methods having those important requirements for edge AI. Another line of research is the parameter-oriented approaches: the goal is to associate part of the model's parameters with a specific context in order to maximize the performance on both past and current contexts. This is mainly done by mapping parameters
to contexts , parameter pruning or parameter addition . Among the parameter-oriented techniques, a
recent trend focused on prompt tuning, used for transformer architectures . It consists in learning input
tokens which are prepended to the input to condition the behavior of the model whose weights are frozen.
While parameter pruning and mapping lead to saturation after many training stages, the parameter addiction
approach lead to a growing number of parameters.
A complementary approach is the knowledge-editing strategy which aims at correcting specific knowledge by locating and updating the responsible neurons.
Bibliography
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales Leonardis, GregorySlabaugh, Tinne Tuytelaars, "A continual learning survey: Defying forgetting in classification tasks",
IEEE Trans. Pattern Anal. Mach. Intell (TPAMI), 44:7, 2022. Li et al., "Learning without Forgetting", European Conference on Computer Vision (ECCV) 2016
Mallya et al., "PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning",
Conference on Computer Vision and Pattern Recognition (CPPR), 2018
Hu et al., "LoRA: Low-Rank Adaptation of Large Language Models", International Conference on
Learning Representations (ICLR), 2022 Methods in Natural language Processing (EMNLP), 2021 representations (ICLR), 2023
Principales activités
Objectives
The objective of this PhD is to investigate incremental learning of foundation models on embedded systems.
The graduate student will initially address the question: "Are foundation models prone to catastrophic
forgetting with standard fine-tuning techniques?" by conducting comprehensive studies and experiments.
Subsequently, she/he will explore how state-of-the-art (SOTA) methods can be applied to these models,
assessing the extent to which these existing techniques provide effective solutions to the problem of
incremental learning. This phase will involve adapting and optimizing SOTA methods to suit the constraints
and requirements of embedded systems (efficiency, privacy, low energy consumption, robustness, etc.).
Finally, the PhD student will develop and propose new methodologies specifically designed to enable
foundation models to learn incrementally when deployed on embedded systems, ensuring that these models
maintain performance and adaptability over time.
What we are looking for
The ideal candidate will possess the following skills and attitudes:
Compétences
(see ideal candidate before)
Avantages
Subsidized meals Partial reimbursement of public transport costRémunération
monthly gross salary amounting to 2100 euros
