Trust & Safety Tools and Services

AymaraAutomated evaluations for AI safety, accuracy and jailbreak vulnerability.Red teaming and evaluation · Open source
CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
DeepEvalOpen source LLM evaluation with pytest style tests and a red teaming module.Red teaming and evaluation · Open source
GarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
GiskardOpen source testing for LLM and ML apps, with a scan for common vulnerabilities.Red teaming and evaluation · Open source
InspectOpen evaluation framework for LLMs from the UK AI Security Institute, widely adopted.Red teaming and evaluation · Open source
LangSmith EvaluationEvaluation and tracing for LLM apps built with or without LangChain.Red teaming and evaluation · Freemium
LLM CanaryEvaluates models for security vulnerabilities and adversarial robustness.Red teaming and evaluation · Open source
OWASP Top 10 for LLM ApplicationsThe reference list of security risks in LLM applications.Red teaming and evaluation · Free
Prompt FuzzerTests prompt injection weaknesses in systems built on language models.Red teaming and evaluation · Open source
PromptfooAutomated model evaluations with ready made attack strategies and reports.Red teaming and evaluation · Open source
PyRITMicrosoft's Python toolkit for AI red teaming and security testing.Red teaming and evaluation · Open source