Evaluation Scenario Writer

il y a 8 heures

Paris, France Mindrift Temps plein

Overview Please submit your CV in English and indicate your level of English proficiency. Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment. What This Opportunity Involves While each project involves unique tasks, contributors may: Create structured test cases that simulate complex human workflows Define gold-standard behavior and scoring logic to evaluate agent actions Analyze agent logs, failure modes, and decision paths Work with code repositories and test frameworks to validate your scenarios Iterate on prompts, instructions, and test cases to improve clarity and difficulty Ensure that scenarios are production-ready, easy to run, and reusable What We Look For This opportunity is a good fit for software engineers, open to part-time, non-permanent projects. Ideally, contributors will have: 3+ years of software development experience with strong Python focus Experience with Git and code repositories Comfortable with structured formats like JSON/YAML for scenario description Understanding core LLM limitations (hallucinations, bias, context limits) and how these affect evaluation design Familiarity with Docker English proficiency - B2 How It Works Apply → Pass qualification(s) → Join a project → Complete tasks → Get paid Project time expectations Tasks for this project are estimated to take 6-10 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted. Payment Paid contributions, with rates up to $50/hour* Fixed project rate or individual rates, depending on the project Some projects include incentive payments Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project #J-18808-Ljbffr

  • Evaluation Scenario Writer

    il y a 2 semaines

    Paris, Île-de-France Mindrift Temps plein

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI. What we...

  • AI Agent Evaluation Analyst

    il y a 9 heures

    Paris, France Mindrift Temps plein

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI....

  • Freelance Agent Evaluation Analyst

    il y a 2 jours

    Paris, France Mindrift Temps plein

    4 days ago Be among the first 25 applicants Get AI-powered advice on this job and more exclusive features. This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift, innovation meets...

  • Freelance Agent Evaluation Analyst

    il y a 2 semaines

    Paris, Île-de-France Mindrift Temps plein

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency.At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of...

  • UX Writer

    il y a 2 semaines

    Paris, France Nexton Consulting FR Temps plein

    Description de l'offre NEXTON recrute un UX WRITER H/F, en CDI, à Paris ! Qui sommes-nous ? NEXTON c’est avant tout une entreprise qui accompagne ses clients dans leur transformation digitale. Tous les jours, nous travaillons avec des grands comptes et des pures players (SNCF, Orange, BNP PARIBAS ). Nous sommes experts du digital aussi bien sur de...

  • Principal Medical Writer

    il y a 20 heures

    Paris, Île-de-France ICON plc Temps plein

    Principal Medical WriterICON plc is a world-leading healthcare intelligence and clinical research organization. We're proud to foster an inclusive environment driving innovation and excellence, and we welcome you to join us on our mission to shape the future of clinical developmentWe are currently seeking a Principal Medical Writer to join our diverse and...

  • Lead Technical Writer

    il y a 6 jours

    Paris, France Tinubu Temps plein

    **ENTREPRISE** - ** Description**: Éditeur de solution SaaS BtoB - ** Année de création**: 2000 - ** Cœur business **:Assurtech - ** Taille **:200 personnes - ** Localisation**: Issy-les-Moulineaux **MISSIONS** Intégré au sein du département IT et en lien direct avec le Head of Software Engineering, vous êtes responsable d’assurer une...

  • Medical Writer

    il y a 2 semaines

    Paris, France Excelya Temps plein

    Created in 2014, Excelya is a people-centered Contract Research Organization (CRO) that excels with care. We offer a personal and authentic experience within a young, ambitious health company on the path to becoming the clinical research leader in Europe thanks to our 800 Excelyates. Our unique one-stop provider service model - leveraging full-service,...

  • AI Evaluation Engineer

    il y a 2 semaines

    Greater Paris Metropolitan Region, France Braintrust Temps plein

    Job DescriptionThis is a contracting engagement - initially 6 months - with potential for long term engagement.Location: Paris-based preferred; alternatively Europe remote for strong candidatesWe are building and evaluating state-of-the-art large language models (LLMs) and are looking for experienced software engineers to join our evaluation and annotation...

  • MCP & Tools Python Developer - Agent Evaluation Infrastructure

    il y a 2 jours

    Paris, France Mindrift Temps plein

    This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English proficiency. At Mindrift, innovation meets opportunity. We believe in using the power of collective human intelligence to ethically shape the future of AI....