WildGuard
Open moderation model for prompts and responses, with a benchmark of real and adversarial cases.
About WildGuard
WildGuard is a safety classification model for user-model chat exchanges. The repository describes its tools as open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs.
The repository lists support for both VLLM and HuggingFace backends. The repository also provides an example of using WildGuard as a safety filter to guard another model's inference. The work will appear at NeurIPS 2024 Datasets & Benchmarks.
Drafted from github.com · not yet confirmed by Allen Institute for AI
How it works
WildGuard is a 7B open model from AI2 that classifies whether a prompt is harmful, whether a response is harmful, and whether the model refused. It ships with WildGuardMix, a training and test set of about 90,000 examples including adversarial jailbreaks, and it outperforms Llama Guard on refusal detection in AI2's evaluations. Apache licensed, easy to run.
Reviews
Be the first to review it
AnonymousKnow a team weighing WildGuard? Send them this page.