Tools & Services

Self Harm and Suicide

Content about self harm and suicide, where removal is not automatically the right answer: recovery communities and crisis support look similar to a classifier.

16 entries

Most upvotedA to Z
  • Aegis Content SafetyNVIDIA's dataset for content moderation and toxicity detection.Datasets and benchmarks · Open source
  • Azure AI Content SafetyHosted classifiers scoring text and images across the main harm categories.AI content classifiers · Usage based
  • CoPESmall open model that classifies content against a policy you define.AI content classifiers · Open source
  • gpt-oss-safeguardOpen weight reasoning model that classifies text against a policy you write.AI content classifiers · Open source
  • Hive Audio ModerationSpeech transcription and moderation for uploaded audio, video and live voice chat.AI content classifiers · Usage based
  • Hive ModerationCloud APIs classifying text, image, video and audio for harmful content.Content moderation platforms · Usage based
  • Innodata Model Safety and Red TeamingHuman and automated red teaming of generative AI models, plus 24/7 content moderation.Red teaming and evaluation · Service
  • Llama GuardOpen weight model that checks prompts and responses against a written policy.AI guardrails · Open source
  • MedSafetyBenchMedical safety prompts for evaluating model safety in medical contexts.Datasets and benchmarks · Open source
  • OpenAI Moderation APIFree classifier covering the main harm categories, for anyone using the API.AI content classifiers · Free
  • ResolverAnalyst-led trust and safety intelligence for platforms, monitored around the clock.Threat intelligence · Service
  • Stanford Trust and Safety (CS 152)Ten-week Stanford course on how online services are abused and the operational, product and engineering responses.Training and certification · Service
  • ToxModVoice moderation that flags harmful speech in live audio.Content moderation platforms · Enterprise
  • UnitaryContext aware video moderation, read frame by frame.Content moderation platforms · Enterprise
  • WildGuardOpen moderation model for prompts and responses, with a benchmark of real and adversarial cases.AI guardrails · Open source
  • ZwillGen Trust, Safety, & Reporting ObligationsCounsel on NCMEC reporting programs, community guidelines and escalation protocols for sensitive content.Legal counsel · Service

Keep a list of the ones worth a call

A free account lets you upvote these and keep them in a list of your own. We never pass your details to a vendor.

Create a free account