Lead LLM Engineer
Il y a 1 jour
Paris, Ile-de-France
Leonar
Temps plein
Gratuit avec email ou Google
Enregistrez cette offre et organisez votre recherche
Créez un compte gratuit pour enregistrer des offres d'emploi, créer des alertes et revenir à cette liste depuis votre tableau de bord.
Gratuit avec email ou Google
En continuant, vous acceptez nos Conditions d’utilisation & Politique de confidentialité.
Licorne Society a été missionné par une startup IA en pleine croissance pour les aider à trouver leur Lead LLM Engineer.
What You Will Own
You will be responsible for one thing:
Make our AI outputs reliable, fast, and indispensable in real workflows.
Concretely
• Design and evolve our LLM / agent architecture
• Own output quality across key use cases (emails, document analysis, etc.)
• Build evaluation systems (datasets, metrics, regression detection)
• Drive fast iteration loops from production data
• Improve retrieval, reasoning, and tool usage
• Ensure production reliability (latency, failure modes, fallback)
• Work directly with product + founders on what to build and why What This Role Is Really About Most teams fail because:
• they don’t know what “good output” means
• they don’t have evals
• they iterate randomly
• they overuse agents Your job is to fix that. You Will Turn
• vague user problems
• → into structured AI systems
• → with measurable performance
• → that improve every week What You Need To Be Excellent At
• Shipping real LLM systems
• You’ve built systems used in production (not demos)
• You understand RAG, tools, agents, structured outputs
• You can design full pipelines, not just prompts
• Evaluation-driven development
• You know how to define quality metrics
• You build datasets from real usage
• You run continuous evals to prevent regressions
• Debugging complex failures
• You can trace issues across:
• retrieval
• prompts
• model behavior
• You don’t guess — you isolate and fix
• Speed of iteration
• You move from problem → improvement in hours or days, not weeks
• You use logs, traces, and data — not intuition alone
• Strong judgment
• You know when to:
• use an agent vs a pipeline
• add complexity vs simplify
• You optimize for reliability and user value, not novelty What We Don’t Care About
• Number of years of experience
• Whether you’ve used a specific framework
• Fancy research credentials If you can build, debug, and improve real systems, you’re a fit. What Success Looks Like (first 90 Days)
• Clear eval framework for core use cases
• Measurable improvement in output quality
• Faster iteration cycles across the team
• Reduced hallucinations / failures
• Stronger system architecture decisions Stack (context, Not Requirements)
• Python (FastAPI)
• Postgres
• Google Cloud
• LangGraph / LangChain (evolving)
• PostHog (product analytics)
• Langfuse (LLM traces)
• LLM APIs (Azure OpenAI)
• Design and evolve our LLM / agent architecture
• Own output quality across key use cases (emails, document analysis, etc.)
• Build evaluation systems (datasets, metrics, regression detection)
• Drive fast iteration loops from production data
• Improve retrieval, reasoning, and tool usage
• Ensure production reliability (latency, failure modes, fallback)
• Work directly with product + founders on what to build and why What This Role Is Really About Most teams fail because:
• they don’t know what “good output” means
• they don’t have evals
• they iterate randomly
• they overuse agents Your job is to fix that. You Will Turn
• vague user problems
• → into structured AI systems
• → with measurable performance
• → that improve every week What You Need To Be Excellent At
• Shipping real LLM systems
• You’ve built systems used in production (not demos)
• You understand RAG, tools, agents, structured outputs
• You can design full pipelines, not just prompts
• Evaluation-driven development
• You know how to define quality metrics
• You build datasets from real usage
• You run continuous evals to prevent regressions
• Debugging complex failures
• You can trace issues across:
• retrieval
• prompts
• model behavior
• You don’t guess — you isolate and fix
• Speed of iteration
• You move from problem → improvement in hours or days, not weeks
• You use logs, traces, and data — not intuition alone
• Strong judgment
• You know when to:
• use an agent vs a pipeline
• add complexity vs simplify
• You optimize for reliability and user value, not novelty What We Don’t Care About
• Number of years of experience
• Whether you’ve used a specific framework
• Fancy research credentials If you can build, debug, and improve real systems, you’re a fit. What Success Looks Like (first 90 Days)
• Clear eval framework for core use cases
• Measurable improvement in output quality
• Faster iteration cycles across the team
• Reduced hallucinations / failures
• Stronger system architecture decisions Stack (context, Not Requirements)
• Python (FastAPI)
• Postgres
• Google Cloud
• LangGraph / LangChain (evolving)
• PostHog (product analytics)
• Langfuse (LLM traces)
• LLM APIs (Azure OpenAI)