Researcher, Evaluations and Benchmarks

Written by the T&S Network from Alice's posting.

Owns a benchmark cadence for measuring frontier AI safety and security risks, shipping a new benchmark every two to three weeks, some public and some shared only with labs. Sits in the CTO office alongside the research lead who sets the public research agenda, with around 150 researchers working on harms available to pull onto subjects.

• AI misuse and jailbreaks

What you'd do

• Ship a benchmark every two to three weeks, sized by subject and sensitivity for public or lab-only release.
• Own the taxonomy, harness, quality bar and release for each benchmark.
• Read evals directly, screening with a model but checking items against the taxonomy and pushing back on researchers.
• Direct in-house researchers and two or three freelance SMEs, holding the plan and calendar.
• Sit monthly with the CTO and research leads to revise the quarterly release plan.
• Spend about 20% of time on the research ecosystem, including lab contacts, papers and conferences.

What it asks for

• Three or more years running safety or security evaluations of language models in production settings.
• 5+ research publications in AI safety and security, lead author on at least 2.
• Hands-on engineering: building eval harnesses, distributed inference, working in vLLM, reading and fixing codebases.
• Strong written and spoken English for cross-time-zone communication.
• Post-training experience with SFT, DPO or GRPO (preferred).
• Agentic evaluation experience covering tool use, orchestration, permissions and prompt injection (preferred).

Terms in this role

Red teaming.

Testing a system, often an AI model, by deliberately trying to make it fail or misbehave, so problems are found and fixed first.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Alicenorth_east before you apply. If we got something wrong, report this listing.