Negative Self-Distillation: Learning to Reason by Avoiding Flaws

TL;DR

Inverts On-Policy Self-Distillation: instead of imitating a privileged teacher, the model diverges from a deliberately flawed one it generates itself. The stated problem is that OPSD can severely degrade LLMs on complex reasoning — forcing the student to imitate "an artificially confident reasoning trace conditioned on privileged information" suppresses expressions of uncertainty and penalises the exploratory, self-corrective behaviours hard problems need. NSD has the model produce a question-specific negative condition — the paper's example is acting as a "careless reasoner" — and pushes the student's distribution away from that self-generated negative teacher. Naively applying an unlearning objective breaks this, because flawed-reasoning tokens are confounded with basic linguistic tokens; a dynamic gating mechanism isolates reasoning-critical tokens so the gradient targets behavioural flaws and leaves the model's linguistic priors intact. Reported to consistently beat OPSD and other label-free, self-bootstrapping RL baselines (source).

Authors & Org

Not published in anything read. The snapshot carries no author list and no affiliation (source).

Method

Element	Detail
Teacher	the model itself, conditioned to be bad — a "question-specific negative condition", e.g. a "careless reasoner"
Objective	push the student's distribution away from the negative teacher
Supervision required	none — no ground-truth answers, no external supervision
The failure it had to solve	flawed-reasoning tokens are confounded with basic linguistic tokens; penalising both "risks catastrophically degrading the model's foundational language capabilities"
Fix	a dynamic gating mechanism that automatically identifies and isolates reasoning-critical tokens
(source)

Results

Stated as a consistent win over OPSD and other label-free, self-bootstrapping RL baselines. No benchmark, model, size or figure appears in anything read — the abstract in the snapshot is qualitative throughout, and arxiv.org is blocked from this run's sandbox (source).

Recorded that way deliberately: a claim of consistent improvement with no number attached is not a result this wiki can compare to anything, and writing one in would be inventing it.

HuggingFace Daily Papers, 2026-09-13, 17 upvotes — that community's popularity signal and nothing more.

Significance

It is the direct negative-image of a paper this wiki ingested four days ago, and the two agree on the diagnosis. One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation argues that the asymmetry producing OPSD's signal — a teacher that is no stronger, only better informed — is the same asymmetry that biases it, and that the field's dominant failure is collapse, the progressive narrowing of the reasoning paths a model can produce. NSD names the same mechanism in different words: imitating an "artificially confident" privileged trace suppresses uncertainty and self-correction. One paper diagnoses; this one proposes that the fix is to stop imitating and start diverging.

The confound it reports is the part with teeth. Unlearning objectives applied to reasoning have to separate how the model reasons from how the model writes, and the paper states plainly that indiscriminate penalisation is catastrophic for language ability. That is a specific, checkable failure mode of an entire family of methods, and the gating mechanism is an admission that the separation does not come for free.

Label-free is the economic claim. No ground truth means the method scales with compute rather than with annotation — the same argument Agentic Reinforcement Learning and Post-Training Scaling record for verifier-based RL, but without a verifier either. Nothing read reports what it costs, so the comparison is a shape, not a measurement.

Open Questions

• Every number. No benchmark, no baseline figure, no model scale, no compute budget appears in anything read.
• Whether the negative teacher can be gamed. The student is optimised to be unlike a model prompted to be careless; a policy could satisfy that by being different rather than better, and nothing read reports a check for it.
• How the gating mechanism decides what is reasoning-critical. Described as "automatic" and "dynamic"; no criterion, classifier or threshold is given.
• Whether it addresses collapse. One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation makes collapse the central symptom; nothing read reports a diversity or path-narrowing measurement for NSD.
• Whether the negative condition has to be hand-written per domain. The one example given is a prompt.

Cite

arXiv 2609.11699, Negative Self-Distillation: Learning to Reason by Avoiding Flaws. Read via source — arxiv.org is blocked from this run's sandbox, so the abstract in that snapshot is the entire text this page rests on.