Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation

TL;DR

Updating a model on its own generated text can worsen prediction on independent human text. The authors trace a feedback loop between the changing generator and the updates retained in its weights, and test an independent-evidence gate before committing updates. (source)

Authors & Org

Cheng Luo, Bing Li and Bernard Ghanem. Affiliations are not established by the abstract page. Submitted 2026-10-04; read scope is the abstract and bibliographic record. (source)

Method

Across 128K-token streams, the study tests TTT-E2E configurations labeled 125M, 760M and 3B, plus Adam updates to Qwen3-4B's existing weights. Fixed Generation freezes the model producing training chunks; Recorded Replay separates degraded-input effects from damage retained through updates. A paired one-update comparison tests source-text fit against new real text. (source)

Results

Fixed Generation removes over 98% of the damage at 125M and 760M. Updates can fit their own source better while predicting new real text worse; the same mechanisms can improve on real text, so updating itself is not the failure. (source)

Settlement checks a candidate state against independent real text before committing it. It leaves mean endpoint gaps of 0.07 and -0.02 nats at 125M and 760M, while retaining real-text adaptation. These results are not reported for every tested model in the abstract. (source)

Significance

This qualifies a neighboring mechanism to Test-Time Compute (Inference-Time Compute Scaling): test-time training changes weights, whereas the concept page's core definition keeps them fixed. Interpretation: spending more inference-time compute and retaining self-generated updates are different interventions and need different controls. (source)

Open Questions

The abstract does not establish the cost of Settlement or performance at frontier scale. It does not show that all synthetic-data training fails, and the reported 98% mitigation is limited to the two named configurations. (source)

Cite

Cheng Luo, Bing Li, Bernard Ghanem. “Self-Generated Feedback Destabilizes Test-Time Training: A Causal Decomposition of Long-Horizon Adaptation.” arXiv:2610.05076, 2026. arXiv.