Trust & Safety Tools and Services

AILuminateHuman written prompts across harm categories, from MLCommons.Datasets and benchmarks · Open source
ALERTStandard and adversarial red teaming prompts, from Babelscape.Datasets and benchmarks · Open source
Aya Red-teamingMultilingual red teaming prompts across harm categories, from Cohere Labs.Datasets and benchmarks · Open source
DEF CON Red Teaming DatasetResults from DEF CON's generative AI red teaming event.Datasets and benchmarks · Open source
Do Not AnswerQuestions across risk areas for testing safety and refusal behavior.Datasets and benchmarks · Open source
HackAPrompt DatasetPrompt based jailbreak attempts, for testing model vulnerability.Datasets and benchmarks · Open source
HarmBenchEvaluation dataset for automated red teaming, from the Center for AI Safety.Datasets and benchmarks · Open source
JailbreakBenchHarmful behaviors used as a benchmark for jailbreaking evaluation.Datasets and benchmarks · Open source
PKU-SafeRLHFPrompts marked for unsafe responses across multiple harm categories.Datasets and benchmarks · Open source
SorryBenchAdversarial prompts with linguistic mutations for testing safety.Datasets and benchmarks · Open source
SOSBenchRegulation grounded prompts across six high risk scientific domains.Datasets and benchmarks · Open source