Bilingual LLM Assessment Specialist Expert

Il y a 23 minutes

SaintMartinLacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

About The Job
Mercor
connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
Benchmark
,
General Catalyst
,
Peter Thiel
,
Adam D'Angelo
,
Larry Summers
, and
Jack Dorsey
.

Position:
AI Model Evaluator

Type:
Full-time or Part-time Contract Work
Compensation:
$36/hour
Location:
Geography restricted to Europe, Canada (Quebec), USA
Role Responsibilities

  • Evaluate LLM-generated responses on their ability to effectively answer user queries.
  • Conduct fact-checking using trusted public sources and external tools.
  • Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies.
  • Assess reasoning quality, clarity, tone, and completeness of responses.
  • Ensure model responses align with expected conversational behavior and system guidelines.
  • Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines.

Qualifications
Must-Have

  • Bachelor's degree
  • Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in French
  • Significant experience using large language models (LLMs)
  • Excellent writing skills
  • Strong attention to detail
  • Adaptable and comfortable moving across topics
  • Background or experience in domains requiring structured analytical thinking
  • Excellent college-level mathematics skills

Preferred

  • Prior experience with RLHF, model evaluation, or data annotation work
  • Experience writing or editing high-quality written content
  • Experience comparing multiple outputs and making fine-grained qualitative judgments
  • Familiarity with evaluation rubrics, benchmarks, or quality scoring systems

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
,

  • Bilingual LLM Evaluator

    Il y a 7 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:AI Model EvaluatorType:Full-time or Part-time Contract WorkCompensation:$40/hourLocation:Geography restricted to Europe,...

  • Conversational AI Quality Evaluator

    Il y a 11 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:AI Model Evaluation SpecialistType:Full-time or Part-time Contract WorkCompensation:$40/hourLocation:Geography...

  • Translator | Upto 40/hr Hourly

    Il y a 27 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Bilingual Translation Quality SpecialistType:Project-basedCompensation:$35–$40/hourLocation:RemoteRole...

  • Translator | Up to 30/hr Rate

    Il y a 50 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Bilingual Translation Quality SpecialistType:ContractCompensation:$20–$30/hourLocation:RemoteDuration:1+...

  • Translator

    Il y a 16 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Bilingual English–Italian Language ExpertType:ContractCompensation:$35–$40/hourLocation:RemoteDuration:1+...

  • Clinical Pharmacology Specialist

    Il y a 5 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Translational Pharmacology ExpertType:ContractCompensation:$175–$250/hourRole ResponsibilitiesDevelop golden FIH dose...

  • Localization Project Lead Coordinator

    Il y a 47 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Bilingual English–Italian Language ExpertType:Independent...

  • Physics Specialist Fully Remote

    Il y a 36 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Physics ExpertType:ContractCompensation:$60–$80/hourLocation:RemoteDuration:~2 months, with extension...

  • Freelance Language Translator Specialist 35/hr Max

    Il y a 10 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Linguistic Expert – ItalianType:Part-timeCompensation:$20–$35/hourLocation:RemoteCommitment:10 hours/weekRole...

  • Lead PK/PD Modeling Specialist

    Il y a 38 minutes

    Saint-Martin-Lacaussade, Nouvelle-Aquitaine, France Mercor Temps plein

    About The JobMercorconnects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors includeBenchmark,General Catalyst,Peter Thiel,Adam D'Angelo,Larry Summers, andJack Dorsey.Position:Translational/Clinical Pharmacology Decision-MakerType:Contract /...