Tools & Services

New in the Catalog

  • Aegis Content SafetyNVIDIA's dataset for content moderation and toxicity detection.Datasets and benchmarks · Open source
  • Collective MetricsOpen source metrics and public dashboards on online safety, trust, privacy and security.Audit and reporting · Free
  • FIRESShared moderation advisories and label distribution between fediverse servers.Community management · Open source
  • Jigsaw ToxicityWikipedia comments labeled by human raters for toxic behavior.Datasets and benchmarks · Open source
  • AILuminateHuman written prompts across harm categories, from MLCommons.Datasets and benchmarks · Open source
  • ALERTStandard and adversarial red teaming prompts, from Babelscape.Datasets and benchmarks · Open source
  • AU10TIXDocument, biometric, address and age verification with anti money laundering screening.Identity verification · Enterprise
  • Aya Red-teamingMultilingual red teaming prompts across harm categories, from Cohere Labs.Datasets and benchmarks · Open source
  • AymaraAutomated evaluations for AI safety, accuracy and jailbreak vulnerability.Red teaming and evaluation · Open source
  • badwordsWord lists in several locales for flagging user submitted content.Datasets and benchmarks · Open source
  • CoopOpen source content review and enforcement: queues, routing, enforcement rules and reporting to NCMEC.Case management · Open source
  • CounterfitAutomation for assessing the security and robustness of AI models.Red teaming and evaluation · Open source
  • DarkBenchBenchmark for detecting dark design patterns in language models.Datasets and benchmarks · Open source
  • DEF CON Red Teaming DatasetResults from DEF CON's generative AI red teaming event.Datasets and benchmarks · Open source
  • Do Not AnswerQuestions across risk areas for testing safety and refusal behavior.Datasets and benchmarks · Open source
  • FaceTec3D face liveness detection and matching, licensed as an SDK inside other identity platforms.Identity verification · Enterprise
  • GarakFramework for adversarial testing and evaluation of language models.Red teaming and evaluation · Open source
  • HackAPrompt DatasetPrompt based jailbreak attempts, for testing model vulnerability.Datasets and benchmarks · Open source
  • HarmBenchEvaluation dataset for automated red teaming, from the Center for AI Safety.Datasets and benchmarks · Open source
  • ID.meIdentity verification and a reusable credential wallet, used for government and commercial access.Identity verification · Enterprise
  • IDnowIdentity verification across automated and video agent flows, built for European regulated markets.Identity verification · Enterprise
  • iProovFace biometric verification and liveness assurance for remote onboarding and authentication.Identity verification · Enterprise
  • JailbreakBenchHarmful behaviors used as a benchmark for jailbreaking evaluation.Datasets and benchmarks · Open source
  • LLM CanaryEvaluates models for security vulnerabilities and adversarial robustness.Red teaming and evaluation · Open source
  • MedSafetyBenchMedical safety prompts for evaluating model safety in medical contexts.Datasets and benchmarks · Open source
  • MitekMobile document capture and identity verification, with check fraud detection for banking.Identity verification · Enterprise
  • PindropVoice fraud and synthetic speech detection for call centers and voice channels.Deepfake detection and provenance · Enterprise
  • PKU-SafeRLHFPrompts marked for unsafe responses across multiple harm categories.Datasets and benchmarks · Open source
  • Prompt FuzzerTests prompt injection weaknesses in systems built on language models.Red teaming and evaluation · Open source
  • PromptfooAutomated model evaluations with ready made attack strategies and reports.Red teaming and evaluation · Open source
  • PyRITMicrosoft's Python toolkit for AI red teaming and security testing.Red teaming and evaluation · Open source
  • Reality DefenderMultimodal deepfake detection across video, audio, images and text.Deepfake detection and provenance · Enterprise
  • Roblox SentinelRoblox's open source classifier for detecting child endangerment signals in chat.AI content classifiers · Open source
  • Sensity AIForensic detection of face swaps, synthetic identity documents and AI generated images.Deepfake detection and provenance · Enterprise
  • Shufti ProIdentity, document and age verification with anti money laundering screening.Identity verification · Enterprise
  • SorryBenchAdversarial prompts with linguistic mutations for testing safety.Datasets and benchmarks · Open source
  • SOSBenchRegulation grounded prompts across six high risk scientific domains.Datasets and benchmarks · Open source
  • SquadboxFriend sourced moderation of your own inbox: a helper screens messages on your behalf.Community management · Open source
  • Toxic ChatToxic conversations collected from real interactions with a chat assistant.Datasets and benchmarks · Open source
  • TruepicContent provenance and C2PA signed capture, for verifying where an image came from.Deepfake detection and provenance · Enterprise
  • UliBrowser extension and crowd sourced slur list for reducing online gendered harassment.Community management · Open source
  • Uli DatasetGendered abuse data, created for Tattle's Uli redaction model.Datasets and benchmarks · Open source
  • XSTestPrompts that test exaggerated safety behavior and over refusal.Datasets and benchmarks · Open source