AI Model Policy Trainer, Image Evaluation - Seattle Onsite

Written by the T&S Network from Handshake's posting.

Image policy evaluation for a generative image model: judging generated images against their prompts for adherence, technical quality and realism, and classifying them against customer content policies covering sexual content, violence, hate symbols, real-person likeness, intellectual property and minors. Work is part of Handshake AI's effort with frontier AI lab researchers to train models to produce better and safer images.

• Adult and sexual content
• Hate speech

What you'd do

• Judge how closely each generated image follows its prompt, and check its layout, realism, style consistency and visible flaws.
• Compare two images side by side and select the stronger one with an evidence-based rationale.
• Classify images against customer content policies, choosing the most defensible category when a case is ambiguous.
• Write concise rationales citing rubric language and specific visual details so another evaluator can audit them.
• Keep separate judgments on personal taste, prompt failure and policy violation, applying the customer's standard consistently.
• Craft new prompts that expose where a model's output quality or safety behavior fails.
• Flag rubric gaps, contradictions and new edge cases to project leads and policy teams.

What it asks for

• Trained visual eye from photography, illustration, design, art direction, retouching, VFX or similar, with the ability to say why one image is better.
• Heavy hands-on use of generative image tools such as Midjourney, Stable Diffusion, ComfyUI, Flux, DALL-E or Ideogram, including their failure modes.
• Experience applying a policy taxonomy to visual content, ideally at scale and under time pressure (preferred).
• Clear written reasoning that another person can audit.
• Ability to hold a rubric steady across long sessions of near-identical images.
• Working knowledge of copyright, trademark and right-of-publicity basics (preferred).
• Prior work in AI evaluation, RLHF, image labeling or data annotation (preferred).

Worth knowing

• Regular exposure to sensitive imagery including sexual content, nudity, graphic violence, hate symbols, self-harm and depictions of minors.
• Exposure limits, content rotation, mandatory reporting for illegal material and access to mental health support are offered.
• Full-time, 40 hours per week, Monday to Friday, 8 a.m. to 5 p.m. PT.
• W-2 assignment through TCWGlobal supporting Handshake AI.

Terms in this role

RLHF.

Reinforcement learning from human feedback: training an AI model on people's ratings of its answers.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Handshakenorth_east before you apply. If we got something wrong, report this listing.