Pre-phd Student

Le descriptif de l’offre ci-dessous est en Anglais_

**Type de contrat**: CDD

**Contrat renouvelable**: Oui

**Niveau de diplôme exigé**: Bac + 5 ou équivalent

**Fonction**: Chercheur contractuel

**A propos du centre ou de la direction fonctionnelle**:
The Inria centre at Université Côte d'Azur includes 42 research teams and 9 support services. The centre's staff (about 500 people) is made up of scientists of different nationalities, engineers, technicians and administrative staff. The teams are mainly located on the university campuses of Sophia Antipolis and Nice as well as Montpellier, in close collaboration with research and higher education laboratories and establishments (Université Côte d'Azur, CNRS, INRAE, INSERM...), but also with the regiona economic players.

With a presence in the fields of computational neuroscience and biology, data science and modeling, software engineering and certification, as well as collaborative robotics, the Inria Centre at Université Côte d'Azur is a major player in terms of scientific excellence through its results and collaborations at both European and international levels.

**Contexte et atouts du poste**:
**Team**

The STARS research team combines advanced theory with cutting-edge practice, with a focus on computer vision systems.

Scientific context

Feature extraction is a challenging computer vision problem that targets extracting relevant information from raw data in order to reduce dimensionality and capture meaningful patterns. When this needs to be done in a dataset and task-invariant way, it is referred to as general feature extraction. This is a crucial step in machine learning pipelines, and popular methods like VideoSwin and VIdeoMAE work well for the task of action recognition and video understanding. However, these works and also the datasets that they are tested on, like Something-Something and Kinetics, fail to capture information about interactions in daily life.

This work will be part of the ANR PROGRESS FSHD project.
The ANR FSHD project (21 June 2022 - 20 May 2026) aims to identify new Clinical outcomes Assessments (CoAs), digital and inflammatory biomarkers of disease severity and progression, and increase the knowledge of the key role of inflammation in facioscapulohumeral muscular dystrophy (FSHD) pathophysiology. This project will extend the already existing CTRN FSHD France database by implementing, at the Nice hospital, full-body motion video capturing of validated functional scores and at-home patient remote assessment, including digitalized score and full-body motion video capturing (PROGRESS FSHD app). Clinical and remotely collected data will be analyzed using classical multivariate statistical approaches, while full-body motion video will be analyzed with Deep Learning methods. An interconnection between patient data collected at the hospital and at home will be ensured, along with interconnection and interoperability with France Cohortes and RaDiCo.

So, the main question is:
How to extract general features from multimodal data with a lot of noise in the form of irrelevant information? Typical situations that we would like to monitor are daily interactions, responses, and reactions, and analyse cause and effect in behaviour.

The system we want to develop will be beneficial for all tasks requiring focus on interactions. Specifically, healthcare for psychological disorders - general feature extraction will allow deep learning models to assist in various subtasks involved in the diagnosis process.

**Mission confiée**:
In this work, we would like to go beyond existing computer vision deep learning models and introduce ways to extend them to utilise information from new modalities. Also, to identify ways to focus on relevant information for interactions in the input. The system should also take into account the long temporal duration of videos in the datasets in this domain. These have to be done in a flexible way, so that there is mínimal change to the original model and hence the original model’s trained weights are useful too.

Existing methods have mostly focused on modelling the variation of visual cues pertinent to the classes provided for video classification tasks. Though they perform these tasks well, changes in the recording setting or addition of noise in the form of irrelevant background information makes it hard for these models to perform well. So, for obtaining a general feature extractor, the models have to be modified to accommodate for these shortcomings.

**Principales activités**:
The Inria STARS team is seeking a pre-PhD student or an engineer with a strong background in computer vision, deep learning, and video understanding.

**Compétences**:
With theoretical knowledge in Computer Vision, OpenCV, Mathematics, Deep Learning (PyTorch, TensorFlow), and technical background in C++ and Python programming, and Linux.

**Avantages**:

- Subsidized meals
- Partial reimbursement of public transport costs
- Leave: 7 week