$ cat wiki/papers/2026/2610.38331-soft-mtp.md
Soft-MTP — isolating a useful multi-token training signal
TL;DR
Soft-MTP removes a noisy residual from the multi-token prediction gradient and retains a KL self-distillation signal. The author reports better next-token learning and faster speculative decoding across the tested model scales. (source)
Authors & Org
Jayden Teoh. Affiliation is not established in the captured abstract record. First submitted 2026-10-28; the record labels the work a preprint. (source)
Method
The paper decomposes the MTP cross-entropy gradient into a KL self-distillation term toward the model's own next-token prediction (NTP) distribution and a residual with low signal-to-noise ratio. When the distributions share a linear output head, straight-through gradient routing recovers the KL gradient from existing cross-entropy gradients. It avoids vocabulary-sized logit matrices and remains compatible with fused cross-entropy kernels. (source)
Soft-MTP reuses the NTP and MTP computations while backpropagating only the KL signal. The author describes a one-line change supporting both DeepSeek-style and Meta-style MTP, adding no parameters and no measurable overhead in the reported experiments. These are author claims, not an independently reproduced implementation assessment. (source)
Results
Across 167M to 1.2B parameters, the author reports lower NTP perplexity, faster speculative decoding and higher zero-shot accuracy than standard MTP at every tested scale. Lower MTP perplexity appears from 689M onward; stronger math and code reasoning after fine-tuning is reported at 1.2B. Soft-MTP also matches or exceeds direct KL minimization on most metrics. The abstract provides no numerical speedup or accuracy deltas. (source)
Significance
For Test-Time Compute (Inference-Time Compute Scaling), this is a training change intended to improve the efficiency of later decoding, rather than a policy for spending more reasoning effort. Interpretation: throughput and reasoning quality should remain separate measurements when comparing inference approaches. (source)
Open Questions
The captured abstract does not specify hardware, absolute training cost or deployment-scale behavior beyond 1.2B parameters. Whether the reported absence of overhead transfers to other kernels remains open. Full-paper details, code and independent reproduction were not reviewed. (source)