Confidence-based Safety Properties in Caisar

Description du poste

**Domaine**:

- Mathématiques, information scientifique, logiciel

**Contrat**:

- Stage

**Intitulé de l'offre**:

- Confidence-based safety properties in CAISAR H/F

**Sujet de stage**:

- The main objective of this internship will be to investigate, design, and implement a
support for confidence-based safety properties in the CAISAR, the AISER’s open-source
platform for characterizing Artificial Intelligence systems’ safety and robustness.

**Durée du contrat (en mois)**:

- 5 - 6 mois

**Description de l'offre**:

- Through the recent developments of AI, the use of models produced by machine learning has become widespread, even in industrial settings. However, studies are flourishing
showing the dangers that such models can bring, in terms of safety, privacy or even
fairness. We can for example cite the adversarial attacks, small perturbations invisible
to naked eyes which can drastically change the output of our AI. To face these dangers
and improve trust in AI, research works are constantly emerging, resulting into several
tools like ERAN (Gehr et al. 2018) or Marabou (Katz et al. 2019).
In the past couple of years, the AISER team has developed CAISAR (Alberti et al. 2022), an open-source platform that focuses on the specification and verification of AI systems’
robustness and safety. In particular, CAISAR provides a language for formally specifying
AI properties, and integrates various state-of-the-art tools for establishing their validity
in a guaranteed way.The concept of a confidence-based safety property has been recently introduced by
Athavale et al. (Athavale et al. 2024) to recast robustness and fairness properties in
terms of the confidence score with which a neural network generates its outcomes.
The main objective of this internship will be to investigate, design, and implement a
support for confidence-based safety properties in the CAISAR, the AISER’s open-source
platform for characterizing AI systems’ safety and robustness.The broad internship goals are:

- familiarization with the state-of-the-art on formal approaches to properties for AI

safety (Casadio et al. 2022)
- familiarization with the work on confidence-based safety properties (Athavale et

al. 2024)
- getting started with the CAISAR platform
- design and implementation of confidence-based safety properties in CAISAR
- identification and evaluation against benchmarks

**Moyens / Méthodes / Logiciels**:

- CAISARinclusive and enjoyable workplace. We are aware of discriminations based on gender
(especially prevalent on our fields), race or disability, we are doing our best to fight them.
One of our team member is formally trained against psychological harassment and sexual abuse.Minimal requirements- master student or equivalent (2nd/3rd engineering school year) in computer science
- some knowledge of the OCaml programming language (or any other functional
language)
- some knowledge of the Python programming language
- ability to work in a team

Preferred requirements- notions in formal methods (i.e. abstract interpretation, SAT/SMT solving, etc.)
- notions in machine learning and, in particular, neural networks

Localisation du poste

**Site**:

- Saclay

**Localisation du poste**:

- France, Ile-de-France, Essonne (91)

**Ville**:

- Saclay

**Langues**:

- Anglais (Courant)
- Français (Courant)

**Diplôme préparé**:

- Bac+5 - Master 2

**Formation recommandée**:

- Computer science

**Possibilité de poursuite en thèse**:

- Oui

Demandeur

**Disponibilité du poste**:

- 03/03/2025

Informations générales

**Entité de rattachement**:
Le CEA est un acteur majeur de la recherche, au service des citoyens, de l'économie et de l'Etat.

Il apporte des solutions concrètes à leurs besoins dans quatre domaines principaux : transition énergétique, transition numérique, technologies pour la médecine du futur, défense et sécurité sur un socle de recherche fondamentale. Le CEA s'engage depuis plus de 75 ans au service de la souveraineté scientifique, technologique et industrielle de la France et de l'Europe pour un présent et un avenir mieux maîtrisés et plus sûrs.

Implanté au cœur des territoires équipés de très grandes infrastructures de recherche, le CEA dispose d'un large éventail de partenaires académiques et industriels en France, en Europe et à l'international.

Les 20 000 collaboratrices et collaborateurs du CEA partagent trois valeurs fondamentales:

- La conscience des responsabilités
- La coopération
- La curiosité

**Référence **:2024-32918**Description de l'unité**:

- The French Alternative Energies and Atomic Energy Commission (CEA) is a key player in
research, development, and innovation. Drawing on the widely acknowledged expertise
gained by its 16,000 staff spanned over 9 research centers with a budget of 4.1 billion
Euros, CEA actively participates in more than 400 European collaborative projects with
numerous academic (notably as a member of Pa