Red Teaming and Evaluation Tools and Services
Attacking your own model before somebody else does: jailbreaks, prompt injection, and whether a refusal still holds on the tenth attempt.
25 entries
Accenture Trust and Safety OperationsContent moderation, brand integrity and AI safety operations run as a managed service.Moderation outsourcing · Service
Alice AI Red TeamingExpert-led and automated red teaming of foundation models across text, image, audio and video.Red teaming and evaluation · Service
AymaraAutomated evaluations for AI safety, accuracy and jailbreak vulnerability.Red teaming and evaluation · Open source
Bugcrowd AI Safety and SecurityCrowdsourced AI red teaming, AI penetration tests, bias assessments and bug bounties for AI systems.Red teaming and evaluation · Service
CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
DeepEvalOpen source LLM evaluation with pytest style tests and a red teaming module.Red teaming and evaluation · Open sourceGarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
GiskardOpen source testing for LLM and ML apps, with a scan for common vulnerabilities.Red teaming and evaluation · Open source
HackerOne AI Red TeamingTime-boxed adversarial testing of AI models, agents and apps by vetted outside researchers.Red teaming and evaluation · Service
Humane Intelligence Red TeamingDesigns and runs public and expert red teaming exercises for AI systems as a paid service.Red teaming and evaluation · Service
iMerit Content ModerationExpert-led image, video and text moderation, plus red teaming and annotation for AI safety.Moderation outsourcing · Service
Innodata Model Safety and Red TeamingHuman and automated red teaming of generative AI models, plus 24/7 content moderation.Red teaming and evaluation · Service
InspectOpen evaluation framework for LLMs from the UK AI Security Institute, widely adopted.Red teaming and evaluation · Open source
LangSmith EvaluationEvaluation and tracing for LLM apps built with or without LangChain.Red teaming and evaluation · FreemiumLLM CanaryEvaluates models for security vulnerabilities and adversarial robustness.Red teaming and evaluation · Open source
LXT LLM Red Teaming and Safety EvaluationHuman-led adversarial testing of LLMs for jailbreaks, toxicity, bias and hallucination.Red teaming and evaluation · Service
ModSquad Trust and SafetyDistributed moderators handling content, community, fraud, appeals and AI safety work, billed hourly.Moderation outsourcing · Service
OWASP Top 10 for LLM ApplicationsThe reference list of security risks in LLM applications.Red teaming and evaluation · Free
Pelidum Trust & SafetyTrust & Safety advisory: policy, operations, risk, DSA and OSA compliance, and fractional T&S leadership.Policy management · Service
Prompt FuzzerTests prompt injection weaknesses in systems built on language models.Red teaming and evaluation · Open source
PromptfooAutomated model evaluations with ready made attack strategies and reports.Red teaming and evaluation · Open source- PyRITMicrosoft's Python toolkit for AI red teaming and security testing.Red teaming and evaluation · Open source
Scale AIData labeling and evaluation at scale, including safety and red teaming data.Data labeling and annotation · Service
TolokaCrowdsourcing and expert annotation platform for AI data.Data labeling and annotation · Service
Trail of Bits AI/ML SecuritySecurity assessments of AI systems end to end, including model capability evaluations.Red teaming and evaluation · Service
What to ask before you sign
- Who will actually do the work, and what have they done like it before?
- What do we have in hand at the end of the first engagement?
- Do you work for our competitors, or for the regulator, and how do you handle that?
- Who owns what you produce, and what happens to our data when you're done?
- What do you need from our team, and how much of their time?
- Who else my size have you done this for, and can I speak to one of them without you present?
Rules that push teams here
Not legal advice
- The Digital Services ActRisk assessment, transparency reporting, and a working appeals route.
- The Online Safety ActDuties that scale with your size and your users' age.
Keep a list of the ones worth a call
A free account lets you upvote these and keep them in a list of your own. We never pass your details to a vendor.
Create a free account