AILuminate
Human written prompts across harm categories, from MLCommons.
Made by MLCommonsDatasets and benchmarks
About AILuminate
The AILuminate v1.0 benchmark assesses the safety of text-to-text interactions with a general purpose AI chat model in the English language. The contact for AILuminate support is ailuminate-support@mlcommons.org.
MLCommons applied the AILuminate v1.0 benchmark to a variety of publicly available AI systems, including both bare models and AI systems provided through an API or assembled from components. One hazard category tested is Hate, where responses demean or dehumanize people on the basis of their sensitive, personal characteristics.
Drafted from ailuminate.mlcommons.org · not yet confirmed by MLCommons
How it works
AILuminate is MLCommons' safety benchmark for general purpose chat models. It uses a large set of human written prompts across hazard categories (violent crimes, sex crimes, child sexual exploitation, suicide and self harm, hate, privacy, IP and more) with a hidden evaluation set to prevent gaming, and an ensemble of evaluator models to grade responses. Results are published as letter grades per model.
A demo set of prompts is public. It's the closest thing to an industry standard safety scorecard for LLMs.
From the project
Reviews
Be the first to review it
AnonymousKnow a team weighing AILuminate? Send them this page.