Senior Coding Annotator

il y a 1 jour

Paris, Île-de-France Braintrust Temps plein

Job Description
*This is a contracting engagement - initially 6 months - with potential for long term engagement.
Location: Paris-based preferred; alternatively Europe remote for strong candidates
We are building and evaluating state-of-the-art large language models (LLMs) and are looking for experienced software engineers to join our evaluation and annotation team. This role sits at the intersection of
real-world software engineering, model evaluation, and applied AI*
, and is critical to improving model reliability, reasoning, and code quality.

You will design challenging coding tasks, evaluate model outputs against rigorous benchmarks, identify failure modes, and contribute to reinforcement learning and model improvement workflows.

This is
*not*
a junior annotation role. We are looking for practitioners with deep hands-on coding experience who can think like both an engineer and an evaluator.

What You'll Do

  • Create high-quality coding prompts and reference answers (benchmark-style, e.g. SWE-Bench-like problems).
  • Evaluate LLM outputs for code generation, refactoring, debugging, and implementation tasks.
  • Identify and document model failures, edge cases, and reasoning gaps.
  • Perform head-to-head evaluations between private LLMs (Mistral-based) and leading external models.
  • Build or configure coding environments to support evaluation and reinforcement learning (RL).
  • Follow detailed annotation and evaluation guidelines with high consistency.

What We're Looking For

  • 5+ years of professional software development experience.
  • Strong Python skills (required).
  • Knowledge of at least one additional programming language (bonus).
  • 1+ year of coding annotation and/or LLM evaluation experience (part-time OK) for a major frontier AI lab or AI infrastructure company.
  • Prior code reviewer experience is a plus.
  • Proven ability to apply structured evaluation criteria and write clear technical feedback.
  • Fluent in English (written and spoken).
  • Team lead or mentoring experience is a strong plus.

Why This Role

  • Work hands-on with cutting-edge LLMs.
  • Apply real-world engineering judgment to model evaluation and improvement.
  • High-impact, technical work with a focused, senior team.

  • Data Quality Specialist, AI Tutor

    il y a 6 jours

    Paris, Île-de-France Mistral AI Temps plein

    About Mistral At Mistral AI, we believe in the power of AI to simplify tasks, save time, and enhance learning and creativity. Our technology is designed to integrate seamlessly into daily working life. We democratize AI through high-performance, optimized, open-source and cutting-edge models, products and solutions. Our comprehensive AI platform is...

  • Senior Quantitative Lead pricing

    il y a 1 jour

    Paris, Île-de-France BreJa Partners Temps plein

    We are currently supporting a leading financial technology player in the search for a hands-on Quantitative Lead to join its Paris-based quantitative team.You will lead a small team of senior/expert quants while remaining deeply involved in model research, development, and production delivery.Your Role* Lead and mentor a team of 3 senior/expert quants*...

  • Ai Senior Software Engineer

    il y a 3 jours

    Paris, Île-de-France UpSlide Temps plein

    Résumé du posteDéveloppeur·euse senior IA enhybride à Paris, au sein de l'équipeFA & AId'UpSlide, une scale-up SaaS qui améliore la productivité dans Microsoft 365 via des fonctionnalités innovantes, notamment IA.Objectif principalConcevoir et implémenterfonctionnalités IA à fort impactpour les produits PowerPoint, Excel et Word, tout en guidant...

  • Senior Data Engineer

    il y a 1 semaine

    Paris, Île-de-France Swile Temps plein

    At Swile, we believe that good products can help reduce friction in daily professional life and boost employee satisfaction. Today, we provide innovative solutions in various areas such as Fintech, Travel, HR, and Employee Benefits to more than 5.5 million users in 85,000 companies in France and Brazil.As aSenior Data Engineer, your primary role is to design...

  • Senior Software Engineer

    il y a 1 jour

    Paris, Île-de-France Jobgether Temps plein

    This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Senior Software Engineer (Security) in France. As a Senior Software Engineer specializing in security, you will be instrumental in designing and implementing robust security features across large-scale platforms. You will work closely with product and...

  • Senior Full Stack Developer

    il y a 1 semaine

    Paris, Île-de-France Relai Temps plein

    At Relai, we are on a mission to make Bitcoin the go-to savings technology—simple, accessible, and secure. As Europe's leading Bitcoin startup, we're pioneering the future of Bitcoin investing by making it effortless for individuals and institutions alike to buy, sell, and hold Bitcoin without complexity or middlemen. With $12M in Series A funding and...

  • Senior Data Platform Engineer

    il y a 1 semaine

    Paris, Île-de-France Swile Temps plein

    At Swile, we believe that good products can help reduce friction in daily professional life and boost employee satisfaction. Today, we provide innovative solutions in various areas such as Fintech, Travel, HR, and Employee Benefits to more than 5.5 million users in 85,000 companies in France and Brazil. As a Senior Data Platform Engineer, your primary role...

  • ERP Developer senior

    il y a 6 jours

    Paris, Île-de-France Blackfluo Temps plein

    Job Description:Location: Fully remote, Central Europe Time ZoneStart date: To be definedLanguages: English is mandatoryThe ERP Developer is responsible for designing, developing, customizing, and maintaining ERP applications to meet business requirements. They work closely with functional analysts, business users, and IT teams to ensure ERP systems...

  • Senior Software Engineer

    il y a 2 semaines

    Paris, Île-de-France Back Market Temps plein

    Hi, we're Back Market.We're here to helpmake tech reliable, affordable, and better than new. We're a global marketplace for refurbished devices, helping lower our collective environmental impact by providing trustworthy, affordable tech with 92% less carbon emissions than new.Yep, you read that right. Turns out refurbished tech is way better for the planet...

  • Senior and/or Staff Backend Software Engineer

    il y a 1 semaine

    Paris, Île-de-France Criteo Temps plein

    What You'll Do:We're building technology that operates at global scale, and we're doing it together.By joining our R&D organization of more than 1,000 engineers, you'll be part of a diverse, international community that shares knowledge, challenges ideas, and continuously raises the bar for engineering excellence. At the same time, you'll work day to day...