Internship M2

**Internship M2 - Prompt Engineering for Visual Language Foundation Models**:

- Réf **ABG-126445**
- Stage master 2 / Ingénieur- Durée 6 mois- Salaire net mensuel 600- 24/10/2024- LaTIM, UMR 1101- Lieu de travail- Brest Bretagne France- Champs scientifiques- Biotechnologie
- Informatique
- Mots clés- LLM, Vision Language Models, foundation models, medical data**Établissement recruteur**:
The internship will take place in the Laboratory of Medical Information Processing (LaTIM- INSERM

**Description**:
The joint collaboration between LaTIM and Evolucare Technologies resulted in an automated algorithm that screens for ocular anomalies such as diabetic retinopathy, glaucoma and age-related macular degeneration based on fundus photographs. The automated algorithm, whose performance reaches that of a retinal expert, is currently commercialized by OphtAI, which was created by Evolucare Technologies (Villers Bretonneux, France) and ADCIS. It has been deployed in several clinical centers around the world, through the Evolucare Technologies cloud. The success of this solution is partly thanks to the large amount of annotated data collected from the OPHDIAT screening network in Île-De-France, namely 760,000 images from 100,000 diabetic patients. The goal of LaTIM and Evolucare Technologies is to expand screening to all pathologies affecting the eye, or visible through the eye (cardiovascular pathologies, neurodegenerative diseases, etc.). To this end, developing a foundation model based on fundus photographs is highly relevant, especially in the context of enhancing model generalizability [1].
- Literature review of LLMs used in foundation models
- Optimizing the hyperparameters for LLMs in prompt engineering.
- Benchmarking the perfomances of several LLMs using the OPHDIAT dataset.
- Participating in progress meetings

[1] Yukun Zhou et al. “A foundation model for generalizable disease detection from retinal images”. In: Nature 622.7981 (2023), pp. 156-163.
[2] Julio Silva-Rodriguez et al. “A foundation language-image model of the retina (flair): Encoding expert knowledge in text supervision”. In: Medical Image Analysis (2024), p. 103357.
[3] Meng Wang et al. “Common and Rare Fundus Diseases Identification Using Vision-Language Foundation Model with Knowledge of Over 400 Diseases”. In: arXiv preprint arXiv:2406.09317 (2024).

**Profil**:

- Master 2 student
- Programming skills: Python, PyTorch.
- Knowledge in LLM is a plus

**Prise de fonction**:

- 01/01/2025