Intership Agentic Multimodal AI for Understanding and Reasoning over Complex Technical Documents-H/F

As an intern at the CEA, you will have the opportunity to work within a world-renowned research environment. Our teams consist of passionate and dedicated experts, providing an ideal setting for learning and collaboration. You will have access to state-of-the-art equipment and top-tier research resources to carry out your assignments. The work conducted may potentially lead to a scientific publication. Context : Complex technical documents combine heterogeneous information such as text, tables, diagrams, figures and equations, often distributed across multiple pages. They include technical manuals, maintenance procedures, user guides, datasheets, product specifications, engineering documents, and technical or scientific reports. Understanding such documents raises several challenges: retrieving relevant information across long documents, jointly interpreting heterogeneous modalities, linking information distributed over multiple pages, and performing multi-step reasoning. This internship aims to develop multimodal agentic AI capable of planning and orchestrating specialized models and tools for complex document understanding and reasoning. What do we expect from you? To address these challenges, the internship will investigate the following research directions:
• Review existing benchmarks and, if necessary, develop a benchmark covering a fine-grained taxonomy of technical documents and multimodal reasoning tasks.
• Evaluate recent Vision-Language Models (VLMs) for technical document understanding.
• Design an agentic architecture able to decompose complex queries and plan multi-step reasoning.
• Dynamically retrieve relevant pages and select appropriate tools or models.
• Combine and reason over information from text, tables, figures, diagrams and equations across multiple pages.
• Evaluate the proposed approach against direct VLM and multimodal RAG baselines. # Cea-List Profil du candidat Profile :
• Students in their 4th or 5th year of studies (M1, M2 or gap year)
• Strong interest for Multimodal Generative AI
• Knowledge of LLMs, Vision-Language Models (VLMs), Transformers, RAG and agentic systems
• Computer vision skills
• Machine learning skills (deep learning, perception models, generative AI…)
• Python proficiency in a deep learning framework (especially TensorFlow or PyTorch) Localisation du poste Site Saclay Localisation du poste France, Ile-de-France, Essonne (91) Ville Saclay Critères candidat Langues
• Français (Courant)
• Anglais (Courant) Diplôme préparé Bac+5
- Diplôme École d'ingénieurs Formation recommandée Apprentissage automatique et IA Possibilité de poursuite en thèse Oui