Paris, Île-de-France
Doctolib (France)
Temps plein
120 000 € - 180 000 € Contrat
Gratuit avec email ou Google
Enregistrez cette offre et organisez votre recherche
Créez un compte gratuit pour enregistrer des offres d'emploi, créer des alertes et revenir à cette liste depuis votre tableau de bord.
Gratuit avec email ou Google
En continuant, vous acceptez nos Conditions d’utilisation & Politique de confidentialité.
# Staff AI Engineer
- Agent Evaluation Platform (x/f/m)Unlimited ContractAIParis, FranceRetour aux résultats##
À propos
de DoctolibNous réinventons la santé pour les soignants et les patients.
Plus de 3 000 Doctolibers en France, en Allemagne, en Italie, au Royaume-Uni et aux Pays-Bas conçoivent une nouvelle génération de technologies et de services basés sur l’IA, auxquels font confiance chaque jour 520 000 soignants et plus de 90 millions de patients.
Chez Doctolib, la maîtrise de l’IA est une compétence clé pour chaque membre de l’équipe : nous cultivons une culture qui s’appuie sur l’IA pour gagner en efficacité, renforcer la qualité et démultiplier notre impact à grande échelle.## Fiche de posteJoin a team of passionate and hardworking entrepreneurs to transform healthcareWorking in the tech team at Doctolib involves building innovative products and features to improve the daily lives of care teams and patients. We work in feature teams in an agile environment, while collaborating with product, design, and business teams.##
What you'll do
We are looking for a Staff AI Software Engineer to join the AI Evaluation team (EvalX) at Doctolib and help every AI product team measure, understand and improve the quality, reliability and safety of their systems.
Our AI products span patient-facing conversational agents, AI-powered phone assistants and clinical assistants supporting healthcare professionals. EvalX builds the platform, tools and methods that let these teams evaluate continuously, and our ambition is to make evaluation a core engineering practice.
In this role, you will work at the intersection of platform engineering, applied AI and product teams: you build shared evaluation capabilities, and you work directly with the teams that use them so they actually get adopted.
In this role, you will:
* Design, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations) used by AI product teams across Doctolib
* Work hands-on with product teams to assess how they evaluate today, define improvement plans, and help them own their evaluation quality
* Design evaluation strategies for agentic systems, combining deterministic checks, LLM-as-a-judge, simulation-based testing, and human review
* Build intelligent capabilities for production sampling, dataset representativeness monitoring, distribution-shift detection, and test case generation
* Help teams define meaningful quality metrics, representative datasets, and reliable quality gates throughout the development lifecycle
* Turn recurring product needs into reusable, self-service platform capabilities, and measure success through adoption, team autonomy, and evaluation coverage
* Contribute to cross-functional initiatives that shape Doctolib's AI platform and best practices for building and evaluating agentic systems## Who you areBefore you read on, if you don't have the exact profile described below, but you feel this
job description
matches your skill set, we still encourage you to apply You could be our next team mate if you:
* Have 6+ years of experience as a software engineer, with significant experience using Python; or as a machine learning engineer
* Have built or operated evaluation for LLM-powered or agentic systems: datasets, LLM-as-a-judge, human review, production-driven evaluation
* Understand and promote software development best practices
* Are product-oriented and enjoy working directly with other teams, challenging assumptions, and turning complex needs into concrete solutions
* Can influence engineering practices and drive change without hierarchical authority: your job is not done until your solution is adopted
* Are comfortable speaking and writing in English in your day-to-day with the team*Now, it would be fantastic if you: Have experience with applied ML techniques such as embeddings, clustering, statistical sampling, and distribution monitoring
* Have experience building developer platforms or internal tooling, or in a Forward Deployed Engineering role
* Have experience in the medical domain or another regulated environment##
What we offer
**
* Free comprehensive health insurance (basic package) for you and your children
* 25 days of paid vacation per year, plus up to 14 days of RTT
* Free mental health and coaching services through our partner Moka.care
* Work from abroad for up to 10 days per year thanks to our flexibility days policy
* Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib
* A subsidy from the work council to refund part of the membership to a sport club or a creative class
* 50% reimbursement of your public transport subscription
* Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth
* ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib
* For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off f
- Agent Evaluation Platform (x/f/m)Unlimited ContractAIParis, FranceRetour aux résultats##
À propos
de DoctolibNous réinventons la santé pour les soignants et les patients.
Plus de 3 000 Doctolibers en France, en Allemagne, en Italie, au Royaume-Uni et aux Pays-Bas conçoivent une nouvelle génération de technologies et de services basés sur l’IA, auxquels font confiance chaque jour 520 000 soignants et plus de 90 millions de patients.
Chez Doctolib, la maîtrise de l’IA est une compétence clé pour chaque membre de l’équipe : nous cultivons une culture qui s’appuie sur l’IA pour gagner en efficacité, renforcer la qualité et démultiplier notre impact à grande échelle.## Fiche de posteJoin a team of passionate and hardworking entrepreneurs to transform healthcareWorking in the tech team at Doctolib involves building innovative products and features to improve the daily lives of care teams and patients. We work in feature teams in an agile environment, while collaborating with product, design, and business teams.##
What you'll do
We are looking for a Staff AI Software Engineer to join the AI Evaluation team (EvalX) at Doctolib and help every AI product team measure, understand and improve the quality, reliability and safety of their systems.
Our AI products span patient-facing conversational agents, AI-powered phone assistants and clinical assistants supporting healthcare professionals. EvalX builds the platform, tools and methods that let these teams evaluate continuously, and our ambition is to make evaluation a core engineering practice.
In this role, you will work at the intersection of platform engineering, applied AI and product teams: you build shared evaluation capabilities, and you work directly with the teams that use them so they actually get adopted.
In this role, you will:
* Design, ship, and operate production-grade evaluation tooling (APIs, services, CI/CD integrations) used by AI product teams across Doctolib
* Work hands-on with product teams to assess how they evaluate today, define improvement plans, and help them own their evaluation quality
* Design evaluation strategies for agentic systems, combining deterministic checks, LLM-as-a-judge, simulation-based testing, and human review
* Build intelligent capabilities for production sampling, dataset representativeness monitoring, distribution-shift detection, and test case generation
* Help teams define meaningful quality metrics, representative datasets, and reliable quality gates throughout the development lifecycle
* Turn recurring product needs into reusable, self-service platform capabilities, and measure success through adoption, team autonomy, and evaluation coverage
* Contribute to cross-functional initiatives that shape Doctolib's AI platform and best practices for building and evaluating agentic systems## Who you areBefore you read on, if you don't have the exact profile described below, but you feel this
job description
matches your skill set, we still encourage you to apply You could be our next team mate if you:
* Have 6+ years of experience as a software engineer, with significant experience using Python; or as a machine learning engineer
* Have built or operated evaluation for LLM-powered or agentic systems: datasets, LLM-as-a-judge, human review, production-driven evaluation
* Understand and promote software development best practices
* Are product-oriented and enjoy working directly with other teams, challenging assumptions, and turning complex needs into concrete solutions
* Can influence engineering practices and drive change without hierarchical authority: your job is not done until your solution is adopted
* Are comfortable speaking and writing in English in your day-to-day with the team*Now, it would be fantastic if you: Have experience with applied ML techniques such as embeddings, clustering, statistical sampling, and distribution monitoring
* Have experience building developer platforms or internal tooling, or in a Forward Deployed Engineering role
* Have experience in the medical domain or another regulated environment##
What we offer
**
* Free comprehensive health insurance (basic package) for you and your children
* 25 days of paid vacation per year, plus up to 14 days of RTT
* Free mental health and coaching services through our partner Moka.care
* Work from abroad for up to 10 days per year thanks to our flexibility days policy
* Lunch vouchers (Swile card) worth €8.50 per working day, with €4.50 covered by Doctolib
* A subsidy from the work council to refund part of the membership to a sport club or a creative class
* 50% reimbursement of your public transport subscription
* Enrollment in Doctolib's long-term employee value sharing plan called DoctoGrowth
* ParentCare Program: Enjoy full salary coverage (100%) during your first month of birth leave, and 75% during the second, covered by Doctolib
* For caregivers and workers with disabilities, a package including an adaptation of the remote policy, extra days off f