Trust & Safety Tools and Services
Filters1
Category
- Any category
- Age assurance9
- AI content classifiers9
- AI governance9
- AI guardrails11
- Appeals and redress4
- Audit and reporting11
- Bot and automation defense13
- Case management7
- Community management21
- Content moderation platforms13
- Crisis and incident response6
- CSAM detection12
- Data and analytics9
- Data labeling and annotation14
- Datasets and benchmarks20
- Deepfake detection and provenance10
- Fraud and payments risk31
- Hash matching and databases13
- Identity verification26
- Image and video analysis11
- Investigations and OSINT23
- Legal counsel10
- Moderation outsourcing21
- Policy management10
- Recruiting and staffing4
- Red teaming and evaluation25
- Regulatory compliance24
- Text and language analysis7
- Threat intelligence24
- Training and certification2
- Wellbeing and resilience17
Cost
Harm area
- Any harm area
- Child sexual abuse material0
- Child safety and grooming0
- Terrorism and violent extremism0
- Harassment and bullying4
- Hate speech3
- Self harm and suicide2
- Fraud and scams0
- Spam and platform abuse1
- Misinformation0
- Coordinated manipulation2
- Counterfeits and IP0
- Adult and sexual content1
- Privacy and doxxing0
- AI misuse and jailbreaks11
Deployment
Data location
Standards
- Any standards
- EU Digital Services Act0
- UK Online Safety Act0
- Australian Online Safety Act0
- EU Terrorist Content Regulation0
- GDPR0
- COPPA0
- Age Appropriate Design Code0
- California AB 5870
- NCMEC reporting duties0
- EU AI Act0
- Santa Clara Principles0
- Digital Trust & Safety Partnership0
- Tech Coalition0
- GIFCT0
- IWF membership0
- Christchurch Call0
- NIST AI Risk Management Framework0
- SOC 20
- ISO 270010
- ISO/IEC 420010
Aegis Content SafetyNVIDIA's dataset for content moderation and toxicity detection.Datasets and benchmarks · Open source
AILuminateHuman written prompts across harm categories, from MLCommons.Datasets and benchmarks · Open sourceALERTStandard and adversarial red teaming prompts, from Babelscape.Datasets and benchmarks · Open source
Aya Red-teamingMultilingual red teaming prompts across harm categories, from Cohere Labs.Datasets and benchmarks · Open source- badwordsWord lists in several locales for flagging user submitted content.Datasets and benchmarks · Open source
DarkBenchBenchmark for detecting dark design patterns in language models.Datasets and benchmarks · Open source
DEF CON Red Teaming DatasetResults from DEF CON's generative AI red teaming event.Datasets and benchmarks · Open source
Do Not AnswerQuestions across risk areas for testing safety and refusal behavior.Datasets and benchmarks · Open sourceHackAPrompt DatasetPrompt based jailbreak attempts, for testing model vulnerability.Datasets and benchmarks · Open source
HarmBenchEvaluation dataset for automated red teaming, from the Center for AI Safety.Datasets and benchmarks · Open source
JailbreakBenchHarmful behaviors used as a benchmark for jailbreaking evaluation.Datasets and benchmarks · Open source
Jigsaw ToxicityWikipedia comments labeled by human raters for toxic behavior.Datasets and benchmarks · Open source- MedSafetyBenchMedical safety prompts for evaluating model safety in medical contexts.Datasets and benchmarks · Open source
PKU-SafeRLHFPrompts marked for unsafe responses across multiple harm categories.Datasets and benchmarks · Open sourceSorryBenchAdversarial prompts with linguistic mutations for testing safety.Datasets and benchmarks · Open source
- SOSBenchRegulation grounded prompts across six high risk scientific domains.Datasets and benchmarks · Open source
Toxic ChatToxic conversations collected from real interactions with a chat assistant.Datasets and benchmarks · Open source
ToxiGenLarge machine generated dataset of implicit hate speech across 13 minority groups.Datasets and benchmarks · Open source
Uli DatasetGendered abuse data, created for Tattle's Uli redaction model.Datasets and benchmarks · Open source- XSTestPrompts that test exaggerated safety behavior and over refusal.Datasets and benchmarks · Open source