Trust & Safety Tools and Services

Content CredentialsOpen standard for recording where an image or video came from.Deepfake detection and provenance · Open source
Content Review FiltersReact components for putting blurs and filters into a review tool.Case management · Open source
CoPESmall open model that classifies content against a policy you define.AI content classifiers · Open source
CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
DeepEvalOpen source LLM evaluation with pytest style tests and a red teaming module.Red teaming and evaluation · Open source
DetoxifyOpen models scoring text for toxicity, threats and identity attacks.Text and language analysis · Open source
FaceTec3D face liveness detection and matching, licensed as an SDK inside other identity platforms.Identity verification · Enterprise
GarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
GiskardOpen source testing for LLM and ML apps, with a scan for common vulnerabilities.Red teaming and evaluation · Open source
gpt-oss-safeguardOpen weight reasoning model that classifies text against a policy you write.AI content classifiers · Open source
Granite GuardianOpen guardrail models for detecting harms in LLM traffic.AI guardrails · Open source
Guardrails AIPython framework for validating what goes into and comes out of a model.AI guardrails · Open source
HateBERTBERT retrained on banned Reddit communities for abusive language detection.Text and language analysis · Open source
HoleheChecks whether an email address is registered on many sites without alerting the owner.Investigations and OSINT · Open source
InspectOpen evaluation framework for LLMs from the UK AI Security Institute, widely adopted.Red teaming and evaluation · Open source
Llama GuardOpen weight model that checks prompts and responses against a written policy.AI guardrails · Open source
LLM CanaryEvaluates models for security vulnerabilities and adversarial robustness.Red teaming and evaluation · Open source
MaigretUsername OSINT that also pulls profile details and links, built on Sherlock's idea.Investigations and OSINT · Open source
NCMEC ReportingOpen Ruby client for filing CyberTipline reports.Case management · Open source
NeMo GuardrailsToolkit for programmable rails around LLM apps: topic, safety, jailbreak and fact checks.AI guardrails · Open source