AI Safety Policy Evaluator, Violence & Threats | Seattle Onsite

Written by the T&S Network from Handshake's posting.

Policy evaluation of AI model conversations on violence and threats, judging whether a request or response falls on the depiction side or the enabling side of the line. The work sits in AI safety for frontier AI lab models, with reviewers writing rationales that train the model. The posting says the work is non-engineering content-policy evaluation.

What you'd do

• Review user prompts and model replies on violence, weapons, threats, and dark fiction, reading each against the full conversation.
• Sort fictional, educational, historical, and self-defense violence apart from requests for real-world help or genuine intent to hurt.
• Decide whether a model reply hands over meaningful real-world capability, no matter how the request is dressed up.
• Write and refine tricky or boundary-testing prompts that show where a model's line actually sits.
• Choose the most defensible classification in ambiguous cases and write concise rationales citing policy language.
• Raise policy gaps and emerging edge cases with project leads and policy teams, and take part in calibration discussions.

What it asks for

• Engineering or LLM product work by itself won't meet the bar.
• Strong judgment on violence in at least one area: fiction, real-world mechanics, or how it appears in people who are struggling.
• Ability to write clear, auditable rationales for judgment calls.
• Heavy personal use of AI tools (preferred)
• Prior AI evaluation, red teaming, data annotation, RLHF, or trust and safety work (preferred)
• Professional experience reviewing ChatGPT, Claude, Gemini, or other language model outputs (preferred)

Worth knowing

• Regular, deliberate exposure to graphic material including violence, torture, abuse, threats, self-harm, and death.
• Exposure limits, content rotation, and access to mental health support are provided.
• Full-time, non-exempt W-2 assignment through TCWGlobal supporting Handshake AI, described as ongoing.

Terms in this role

Red teaming.

Testing a system, often an AI model, by deliberately trying to make it fail or misbehave, so problems are found and fixed first.

RLHF.

Reinforcement learning from human feedback: training an AI model on people's ratings of its answers.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Handshakenorth_east before you apply. If we got something wrong, report this listing.