Pareto Improvement
COPSD moves up and right.
Abstract
Why COPSD?
Augmented OPSD within Safety Boundaries eliminates geometric leakage to prevent response and entropy collapse.
Comprehensive 12-Benchmark Evaluation enables our 4B student model to surpass the 235B Oracle's safety performance.
Native VERL & vLLM Implementation provides a high-throughput, cluster-ready production pipeline with accelerated token rollouts.
Anti-Over-Refusal & Context-Aware Data mitigates hyper-conservatism through open-sourced multimodal splits that master subtle real-world environments.
Method
Two Stages
Cross-SFT first calibrates the teacher. Then constitution-conditioned OPSD distills safer behavior without collapsing expressiveness.
Stage 1. Cross-SFT cold-start.
Stage 2. Constitution-conditioned on-policy distillation.
Results
Less Text, More Signal
Student Beats Teacher
Student_C surpasses Teacher_C and OPD.
Ablation
Teacher tuning matters.
Safety Table
Main numbers.
Citation
BibTeX
@article{wen2026copsd,
title = {Constitutional On-Policy Safe Distillation},
author = {Wen, Ming and Liu, Yuxuan and Yang, Kun and Feng, Yunhao and Xu, Zhuoer and Sun, Yuhao and Cui, Shiwen and Zheng, Xiang and Wang, Guoyu and Ma, Xingjun and Jiang, Yu-Gang},
journal = {arXiv preprint arXiv:2606.03089},
year = {2026}
}