Trust & Safety Tools and Services

AILuminateHuman written prompts across harm categories, from MLCommons.Datasets and benchmarks · Open source
ALERTStandard and adversarial red teaming prompts, from Babelscape.Datasets and benchmarks · Open source
Arthur ShieldLLM firewall and evaluation from an ML observability vendor.AI guardrails · Enterprise
Aya Red-teamingMultilingual red teaming prompts across harm categories, from Cohere Labs.Datasets and benchmarks · Open source
AymaraAutomated evaluations for AI safety, accuracy and jailbreak vulnerability.Red teaming and evaluation · Open source
CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
DeepEvalOpen source LLM evaluation with pytest style tests and a red teaming module.Red teaming and evaluation · Open source
DEF CON Red Teaming DatasetResults from DEF CON's generative AI red teaming event.Datasets and benchmarks · Open source
Do Not AnswerQuestions across risk areas for testing safety and refusal behavior.Datasets and benchmarks · Open source
GarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
Gemini API safety settingsConfigurable harm filters built into the Gemini API.AI guardrails · Free
GiskardOpen source testing for LLM and ML apps, with a scan for common vulnerabilities.Red teaming and evaluation · Open source
Granite GuardianOpen guardrail models for detecting harms in LLM traffic.AI guardrails · Open source
Guardrails AIPython framework for validating what goes into and comes out of a model.AI guardrails · Open source
HackAPrompt DatasetPrompt based jailbreak attempts, for testing model vulnerability.Datasets and benchmarks · Open source
HarmBenchEvaluation dataset for automated red teaming, from the Center for AI Safety.Datasets and benchmarks · Open source
InspectOpen evaluation framework for LLMs from the UK AI Security Institute, widely adopted.Red teaming and evaluation · Open source
JailbreakBenchHarmful behaviors used as a benchmark for jailbreaking evaluation.Datasets and benchmarks · Open source
Lakera GuardHosted detection of prompt injection, jailbreaks and data leakage for LLM apps.AI guardrails · Usage based
LangSmith EvaluationEvaluation and tracing for LLM apps built with or without LangChain.Red teaming and evaluation · Freemium