PKU-SafeRLHF
Prompts marked for unsafe responses across multiple harm categories.
About PKU-SafeRLHF
The PKU-SafeRLHF dataset is listed on Hugging Face as PKU-Alignment/PKU-SafeRLHF. Its listed task is Text Generation.
The dataset is associated with arXiv paper arxiv: 2406.15513.
Drafted from huggingface.co · not yet confirmed by PKU-Alignment
How it works
PKU-SafeRLHF is a large preference dataset from Peking University: prompts, pairs of model responses, and human labels for which response is safer and which is more helpful, with harm categories and severity levels. It's built for training reward models and running safe RLHF, and it's one of the few open sets that separates helpfulness from harmlessness.
Use it if you're aligning your own model rather than just evaluating one.
From the project
Reviews
Be the first to review it
AnonymousKnow a team weighing PKU-SafeRLHF? Send them this page.