What AI safety teams actually do

An AI safety team works on what a company's AI models will and won't do, and on the people who try to misuse them. At a company that builds its own models, it can be hundreds of people. At a platform that has just added a chatbot to its app, it's often two people borrowed from Trust & Safety.

Most of the jobs are ones this field already does. What changes is the material. You used to read posts and profiles. Now you're reading prompts, model answers and the accounts sending them.

The jobs, and what they're called

AI safety job titles run long, sometimes with two commas in them. Here's how real ones line up against work you'd recognize:

Two words come up constantly. Red teaming is attacking your own model on purpose, to find what it will do wrong before anyone else does. An evaluation, or eval, is a fixed set of test prompts the model is scored against, so a team can tell whether a new version is safer or worse than the last one.

How it differs from platform work

• Most of the work happens before release. A post can be taken down after it's reported. A model's behavior has to be decided, tested and trained in before it ships.
• Misuse usually takes a whole conversation. Someone after something harmful splits it into harmless-looking pieces, so enforcement reads sessions and accounts, and one message tells you little.
• Some harms are new to most T&S people. Anthropic's open enforcement roles are split by harm, and the list includes biological, chemical and explosives, cyber and conventional weapons alongside child safety and account takeover.
• The customer is often a developer. A lot of misuse arrives through the API, from businesses building on the model, so investigations look like fraud work as much as content work.
• Agents add a new kind of risk. When a model can take actions, such as browsing, sending messages or running code, the question is what it did, as well as what it said.

Who's hiring

Three kinds of employer post these jobs. Companies that train their own models, such as Anthropic and OpenAI. Large platforms adding AI to products people already use, such as Apple, TikTok and Google. And AI app companies, such as Character.AI and ElevenLabs. Vendors are in it too: TaskUs is hiring for its AI safety services.

Pay depends most on which kind it is. T&S Levels prices the frontier labs on their own because they pay well above everyone else at the same level, and a platform's AI safety jobs pay much like its other Trust & Safety jobs.

So if you've been wondering whether you're qualified, here's a useful way to see it: a lot of AI safety is Trust & Safety work, done before launch instead of after it.