Researcher, Safety Oversight

Written by the T&S Network from OpenAI's posting.

Research on maintaining human oversight of frontier AI models, aimed at detecting and mitigating misuse and misalignment in OpenAI's deployed systems. The Safety Oversight Research team within Safety Systems builds ML methods in human-AI collaboration, reasoning, robustness, and scalable oversight, and uses what it learns from deployment to keep deployed models safe.

• AI misuse and jailbreaks

What you'd do

• Develop and refine AI monitor models that detect known and emerging misuse and misalignment patterns.
• Set research directions and strategies to make AI systems safer, more aligned, and more robust.
• Design and evaluate red-teaming pipelines that test the end-to-end robustness of safety systems.
• Research how models reason about human values and apply those models to practical safety problems.
• Coordinate with T&S, legal, policy, and other research teams so products meet safety standards.

What it asks for

• 4+ years of experience in AI safety, especially RLHF, human-AI collaboration, or fairness and bias.
• 4+ years of research engineering experience.
• Coding skill in Python or a comparable language.
• Doctorate or other advanced degree in computer science, machine learning, or a closely related field.
• Experience working with large-scale AI systems.

Worth knowing

• Background check required in accordance with applicable law.

Terms in this role

Red teaming.

Testing a system, often an AI model, by deliberately trying to make it fail or misbehave, so problems are found and fixed first.

RLHF.

Reinforcement learning from human feedback: training an AI model on people's ratings of its answers.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on OpenAInorth_east before you apply. If we got something wrong, report this listing.