๐Ÿ“–
OPDArena: Understanding Learning and Forgetting in On-Policy Distillation
Yuanze Xu, Tianzhu Ye, Hui Sun, Wenhui Wang, Li Dong, Furu Wei

OPDArena: Understanding Learning and Forgetting in On-Policy Distillation

Science & Mathematics Published 2026 Verification Pending

Overview & Synopsis

On-policy distillation (OPD) is increasingly used for language model post-training, yet how teacher supervision shapes learning and forgetting remains unclear. We present OPDArena, a framework comparing supervised fine-tuning (SFT), reinforcement learning with verifiable rewards (RLVR), and OPD across five pretrained models and diverse teacher sources. It jointly evaluates mathematical learning, general capability retention, and predictive distribution changes. Teacher choice strongly shapes retention: students of RLVR-trained teachers improve on the target task while retaining prior capabilities, whereas students of SFT-trained teachers incur greater capability loss. Across post-training methods, out-of-domain (OOD) distribution change shows a more consistent positive association with forgetting than in-domain change. To understand how teacher choice affects these changes, we examine the additional information teachers provide relative to the student's pretrained model. Controlled teacher comparisons show that the task-specific preferences teachers acquire through post-training make student updates more concentrated in-domain.

Indexed from repository: arXiv (arxiv.org)
Loading document from verified source...
Keyboard Shortcuts: ← / → for pages • + / - for zoom • 0 to fit width Powered by Mozilla PDF.js v3.11

Document Information

Document Title OPDArena: Understanding Learning and Forgetting in On-Policy Distillation
Author / Organization Yuanze Xu, Tianzhu Ye, Hui Sun, Wenhui Wang, Li Dong, Furu Wei
Subject & Discipline cs.LG
Categories Science & Mathematics
Publication Year 2026
License / Rights arXiv Open Access / Creative Commons
Resource Verification Listed PDF (Verification Pending)
Original Host Website arXiv (arxiv.org)
File Format application/pdf (Portable Document Format)
Language EN
Catalog ID DOC-274759
Original Source Link Visit Source Page on arxiv.org »