$ cat wiki/papers/2026/2608.14929-training-leaves-traces.md
Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (arXiv:2608.14929)
TL;DR
A data-free, white-box test for whether two model checkpoints share ancestry, read from the weights alone. It separates fine-tuned, LoRA-merged, pruned and quantized descendants from independent models at AUROC = 1.0 — and, critically, also from distilled models, because it measures weight ancestry rather than behavioral similarity (source).
Authors & Org
Not obtainable. arxiv.org is EGRESS_BLOCKED from this environment and the
paper was not read; the HuggingFace Daily Papers snapshot carries title, id, date
and abstract only
(source).
Listed on HuggingFace Daily Papers, 2026-08-21, 16 upvotes — that community's popularity signal and nothing more (source).
Method
The stated setting: open-weight models are fine-tuned, quantized, pruned and merged, and their provenance is often undocumented. Question asked: can weights alone reveal shared ancestry, with no data and no inference?
The obstacle the paper identifies first: residual training produces a shared identity-aligned component in branch products, present in every residual network. That structure alone cannot establish ancestry — it is what all such models have in common. So the method removes it ("centered") and compares the checkpoint-specific structure that remains across residual blocks, yielding a symmetric lineage score calibrated against independent checkpoints.
Evaluated on residual-MLP and GPT-2 benchmarks, with a case study on LLaMA-2 public checkpoints.
Robustness is tested against function-preserving checkpoint laundering — deliberate transformations that keep a model's behaviour while disturbing its weights.
Results
| Item | Figure |
|---|---|
| Separating fine-tuned / LoRA-merged / pruned / quantized descendants from independent and distilled models | AUROC = 1.0 |
| Under function-preserving laundering | score unchanged; weight-space baselines "lose margin or fail" |
| Speed | 76× faster than the nearest robust baseline on GPT-2 |
| Case study | correctly identifies 3 related and 7 unrelated LLaMA-2 public checkpoints |
| Generality | projection-pairing signal appears across six language-model families and beyond |
Significance
Read the AUROC = 1.0 carefully, because the sentence containing it is doing two opposite jobs. Distilled models are grouped with the independent models — the ones the score is designed to reject. So this method does not detect distillation. It detects shared weights. A student trained on a teacher's outputs has no weight ancestry and will look unrelated, correctly by the method's own definition and uselessly for the dispute this wiki has been tracking.
That dispute is a live one. Alibaba / Qwen AI Lab stands accused of a 25K-account, 28.8M-interaction Claude distillation campaign (June 2026), and China's distillation counter-claim of 2026-07-28 is recorded on Open-Weights Policy Fight. Nothing in that argument is settled by this paper, and a reader who saw only "AUROC = 1.0 for model lineage" would conclude the opposite. The distinction between weight ancestry and behavioral similarity is the paper's own, stated in its abstract, and it is the whole of its relevance here.
What it does settle is the licence-compliance question, which is a different and more tractable one. Whether a published checkpoint is secretly a fine-tune of a licence-encumbered base — the OpenMDW, Apache-2.0 and MIT weights this wiki tracks weekly — is exactly weight ancestry, and it is now answerable without the publisher's cooperation, without inference, and 76× faster than the nearest robust alternative. The laundering result matters more than the AUROC for this purpose: a provenance test only means something if evading it is hard.
The case study is the weakest evidence and the most interesting. Ten LLaMA-2 public checkpoints, correctly sorted. That is a real-world demonstration on ten items; GPT-2 and residual-MLPs carry the AUROC. The gap between "AUROC = 1.0 on GPT-2" and "10 checkpoints of one family" is where a frontier-scale claim would have to go, and it is not made.
Open Questions
- Does it hold at frontier scale? Benchmarks are residual-MLP and GPT-2; the case study is LLaMA-2. Nothing read tests a 100B+ MoE, and MoE routing is a structure the "centered residual" construction does not obviously address.
- What counts as laundering? "Function-preserving" transformations were tested; an adversary willing to lose a little quality is a different threat model and is not evaluated.
- Compatible checkpoints only. The method requires architectural compatibility between the two checkpoints compared, which excludes the case where a lab is suspected of copying a different-shaped competitor.
- Could it be inverted into an obfuscation recipe? A published, data-free signature is also a published target.
- Author list, affiliation, licence, code availability — unknown; the paper was not read.
Cite
Training Leaves Traces: Centered Residual Signatures for Language Model Lineage Verification (2026). arXiv:2608.14929.