Anthropic Fellows Program, AI Safety & Security

Written by the T&S Network from Anthropic's posting.

A four-month, full-time fellowship doing empirical AI safety and AI security research on external infrastructure such as open-source models and public APIs, aiming for a public output like a paper. Two workstreams cover alignment research topics and offensive security work on LLM systems, with mentorship from Anthropic researchers.

• AI misuse and jailbreaks

What you'd do

• Run an empirical research project aligned with Anthropic's research priorities, using open-source models and public APIs
• Work with an assigned mentor on project selection and matching to a research area
• Pursue research areas such as scalable oversight, adversarial robustness, model organisms, interpretability, or AI welfare
• Pursue AI security work such as vulnerability research and exploit discovery in smart contracts or LLM systems
• Build and test ideas quickly, and communicate results clearly within a shared research workspace
• Aim to produce a public output such as a paper submission

What it asks for

• Fluent in Python programming
• Available to work full-time on the program for four months
• Work authorization in the US, UK, or Canada, and located there during the program
• Degree-level training in a quantitative field such as computer science, math, or physics
• Experience with empirical ML research projects (preferred)
• Experience working with large language models (preferred)
• Hands-on experience finding security flaws in software through attack-style testing (preferred)

Worth knowing

• Fixed four-month fellowship with possible extension
• Weekly stipend and compute funding; no guaranteed full-time offer

Terms in this role

Red teaming.

Testing a system, often an AI model, by deliberately trying to make it fail or misbehave, so problems are found and fixed first.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Anthropicnorth_east before you apply. If we got something wrong, report this listing.