Trust & Safety Tools and Services

AILuminateHuman written prompts across harm categories, from MLCommons.Datasets and benchmarks · Open source
ALERTStandard and adversarial red teaming prompts, from Babelscape.Datasets and benchmarks · Open source
Alice AI Red TeamingExpert-led and automated red teaming of foundation models across text, image, audio and video.Red teaming and evaluation · Service
Arthur ShieldLLM firewall and evaluation from an ML observability vendor.AI guardrails · Enterprise
Aya Red-teamingMultilingual red teaming prompts across harm categories, from Cohere Labs.Datasets and benchmarks · Open source
AymaraAutomated evaluations for AI safety, accuracy and jailbreak vulnerability.Red teaming and evaluation · Open source
Bugcrowd AI Safety and SecurityCrowdsourced AI red teaming, AI penetration tests, bias assessments and bug bounties for AI systems.Red teaming and evaluation · Service
CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
DeepEvalOpen source LLM evaluation with pytest style tests and a red teaming module.Red teaming and evaluation · Open source
DEF CON Red Teaming DatasetResults from DEF CON's generative AI red teaming event.Datasets and benchmarks · Open source
Do Not AnswerQuestions across risk areas for testing safety and refusal behavior.Datasets and benchmarks · Open source
GarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
Gemini API safety settingsConfigurable harm filters built into the Gemini API.AI guardrails · Free
GiskardOpen source testing for LLM and ML apps, with a scan for common vulnerabilities.Red teaming and evaluation · Open source
Granite GuardianOpen guardrail models for detecting harms in LLM traffic.AI guardrails · Open source
Guardrails AIPython framework for validating what goes into and comes out of a model.AI guardrails · Open source
HackAPrompt DatasetPrompt based jailbreak attempts, for testing model vulnerability.Datasets and benchmarks · Open source
HackerOne AI Red TeamingTime-boxed adversarial testing of AI models, agents and apps by vetted outside researchers.Red teaming and evaluation · Service
HarmBenchEvaluation dataset for automated red teaming, from the Center for AI Safety.Datasets and benchmarks · Open source
Humane Intelligence Red TeamingDesigns and runs public and expert red teaming exercises for AI systems as a paid service.Red teaming and evaluation · Service