$ cat wiki/papers/2026/2608.16647-on-policy-distillation-generalization.md
Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy Distillation of Large Language Models (arXiv:2608.16647)
TL;DR
A controlled study that varies one generalization factor at a time finds on-policy distillation transfers the teacher's reasoning behaviour, not its answers to particular problems: training difficulty barely matters, and problems the teacher never solves are still useful. How far it reaches depends on the origin relationship — same-origin teacher/student pairs transfer across languages, reasoning horizons and even other domains, cross-origin pairs mostly fit the trained distribution. The reach is the same property in both directions: because a teacher's influence cannot be confined by routing prompts to domain experts, combining teachers produces a mixture-dependent seesaw among their capabilities (source).
Authors & Org
Not obtainable. arxiv.org is EGRESS_BLOCKED from this run's sandbox; the
paper was not read. Listed on HuggingFace Daily Papers, 2026-08-25, 14 upvotes
(source).
Method
The gap it names. On-policy distillation (OPD) supervises trajectories sampled from the student's own policy, but its generalization behaviour is "poorly understood, as most studies evaluate OPD on a single domain and on benchmarks close to the training data".
The design is the contribution: vary one factor at a time, across in-domain distribution shift → cross-domain transfer → the multi-teacher setting. That is a controlled study rather than a method paper, which is why its findings are stated as properties of OPD rather than as a system's scores.
Results
Reported as findings; the abstract carries no numbers at all.
- What transfers is behaviour, not answers. Training difficulty barely matters, and problems the teacher never solves are useful — which only makes sense if the signal being copied is how the teacher reasons, not what it concludes.
- Origin relationship dominates. Same-origin teacher/student pairs bring the student close to the teacher "across languages, reasoning horizons, and even other domains". Cross-origin pairs "mostly fit the trained distribution".
- The reach cuts both ways. Since routing prompts to domain experts cannot confine each teacher's influence, combining teachers yields a mixture-dependent seesaw among their capabilities — gaining one is losing another, and which one depends on the mixture.
Significance
"Same-origin transfers, cross-origin does not" is a claim with commercial consequences, and this wiki already holds the dispute it lands in. Alibaba / Qwen AI Lab carries the accusation of a 25K-account, 28.8M-interaction distillation campaign against Claude. This paper does not speak to that case and must not be read as evidence about it — but it does say something about the general shape: a distillation from a cross-origin teacher, on this study's finding, mostly buys the trained distribution, while the broad capability transfer people worry about is the same-origin case.
The multi-teacher seesaw is the practically load-bearing result. The obvious way to build a strong student is to distil several strong teachers and route by domain. This says routing does not partition the influence — so multi-teacher OPD is not additive, and the result depends on the mixture rather than on the teachers' individual strengths.
It sits beside Post-Training Scaling as a limit on the same lever. That page records post-training as the axis capability is currently arriving on — GLM-5.2 → GLM-5.3 at an unchanged base, measured by a third party at 53 → 60. Distillation is one of the cheapest instruments on that axis, and this is the first result here that says its yield is set by a relationship (teacher/student origin) rather than by teacher quality or data volume.
Open Questions
- What is "origin"? Same pretraining corpus, same family, same tokenizer, same lab — the abstract does not define it, and the whole result rests on it.
- No numbers anywhere. Every finding is directional; effect sizes are unknown.
- Which models, which domains, which languages? None named.
- Is the seesaw avoidable? The paper diagnoses it and offers "a useful perspective for diagnosing multi-teacher OPD" — not a remedy.
- Does it hold for off-policy distillation? The controlled comparison is within OPD; the contrast that would matter to practitioners is against plain SFT on teacher outputs.
Cite
Every Coin Has Two Sides: On the Dual Nature of Generalization in On-Policy
Distillation of Large Language Models (2026). arXiv:2608.16647.