ALERT
Standard and adversarial red teaming prompts, from Babelscape.
About ALERT
ALERT is a dataset page on Hugging Face under the Babelscape organization. The dataset is listed with the task Text Generation.
The alert_adversarial subset is listed at 31k rows. The dataset is listed under the cc-by-nc-sa-4.0 license.
Drafted from huggingface.co · not yet confirmed by Babelscape
How it works
ALERT is a benchmark of about 45,000 red teaming prompts from Babelscape, organized into a fine grained taxonomy of 32 harm categories (weapons, crime, hate, self harm, substance abuse and so on). Each prompt is a standard instruction and there's an adversarial version with jailbreak style rewrites. You run a model over the prompts and score its refusal rate per category.
It's a text only evaluation set for LLM safety, and the taxonomy is the useful part: it's more granular than most.
From the project
Reviews
Be the first to review it
AnonymousKnow a team weighing ALERT? Send them this page.