Safeguards Enforcement Analyst, Child Safety

Written by the T&S Network from Anthropic's posting.

Operational child safety enforcement for AI products, covering detection and response to CSAM and CSEM generated or facilitated through Anthropic's products. The Child Safety team's work is to scale and improve the workflows and systems that find and act on this material, and to manage the people who review it.

• Child sexual abuse material
• Child safety and grooming

What you'd do

• Keep daily child safety review running, handling task assignment, queue backlogs, escalations, and tracking against response deadlines.
• Serve as the lead point of contact for outside review vendors, handling their setup, instruction, quality checks, and continuing management.
• Design enforcement workflows that grow with volume while keeping review decisions accurate and consistent.
• Work with Engineering and Data Science on detection models and automated enforcement for CSAM, CSEM, and related violations.
• Review novel or ambiguous flagged content and pass policy gaps to the Safeguards policy design team.
• Draft and upkeep internal reference material, decision flowcharts, and reviewer rules.
• Report misuse trends to Policy, Legal, and Trust & Safety leadership.

What it asks for

• Background running or coordinating content review teams, including quality checks and workflow oversight.
• Track record of building out policy enforcement or content review processes from the ground up.
• Working knowledge of SQL or other data tools to monitor queues and spot enforcement patterns.
• Experience raising emerging risks with Product, Policy, Engineering, and Legal stakeholders.
• Understanding of the difficulties in applying product policies at scale in content moderation.
• Deep subject matter expertise in child safety, CSEA, or online child protection, including CSAM/CSEM classification standards like COPINE or SAM scale (preferred).

Worth knowing

• Regular exposure to explicit sexual content involving minors, plus violent or psychologically disturbing content.
• Wellness resources and support offered for team members in this area.

Terms in this role

COPPA.

The Children's Online Privacy Protection Act, a US law. A service aimed at children, or that knows a user is under 13, needs verifiable parental consent before collecting their personal data.

CSAM.

Child sexual abuse material: images or video of the sexual abuse of children. Illegal to make, share or possess in nearly every country.

CSEA.

Child sexual exploitation and abuse. The wider harm area around CSAM, including grooming, sextortion and trafficking.

NCMEC.

The National Center for Missing & Exploited Children, a US nonprofit. It runs the CyberTipline, where US-based online services must report apparent child sexual abuse material they find.

Hash matching.

Comparing a digital fingerprint of an image or video against lists of known violating content. PhotoDNA, from Microsoft, is the best-known tool for it.

Age assurance.

Ways of checking or estimating a user's age, from asking them to facial age estimation and ID checks.

Our best summary of the posting, and it can be wrong. The employer's posting is what counts: read it in full on Anthropicnorth_east before you apply. If we got something wrong, report this listing.