Trust & Safety Tools and Services

Aegis Content SafetyNVIDIA's dataset for content moderation and toxicity detection.Datasets and benchmarks · Open source
Content Review FiltersReact components for putting blurs and filters into a review tool.Case management · Open source
CoPESmall open model that classifies content against a policy you define.AI content classifiers · Open source
gpt-oss-safeguardOpen weight reasoning model that classifies text against a policy you write.AI content classifiers · Open source
Granite GuardianOpen guardrail models for detecting harms in LLM traffic.AI guardrails · Open source
Llama GuardOpen weight model that checks prompts and responses against a written policy.AI guardrails · Open source
Mastodon moderationReports, account actions, filters and domain blocks built into Mastodon.Community management · Open source
NSFW ModelOpen Keras model for detecting explicit imagery.Image and video analysis · Open source
OzoneOpen source labeling and moderation console for the AT Protocol.Case management · Open source
PDQOpen perceptual hash for images, with a matcher to go with it.Hash matching and databases · Open source
Private DetectorOpen model for identifying unsolicited lewd images.Image and video analysis · Open source
ShieldGemmaOpen weight safety classifiers for model inputs and outputs.AI guardrails · Open source
WildGuardOpen moderation model for prompts and responses, with a benchmark of real and adversarial cases.AI guardrails · Open source