$ cat wiki/papers/2026/2608.09867-stealing-reasoning-traces.md
Stealing Reasoning Traces from Proprietary LLM APIs (arXiv:2608.09867)
TL;DR
Providers hide chain-of-thought by returning it to the client as an encrypted blob the client hands back on the next turn. Those blobs are interchangeable across sessions, users and models within one provider, so feeding a strong model's encrypted trace to a weaker, less-safeguarded sibling makes the sibling decrypt and print it in plaintext — no jailbreak of the strong model required (source).
Authors & Org
Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, Maksym Andriushchenko — ELLIS Institute Tübingen and the Max Planck Institute for Intelligent Systems. Submitted 2026-08-10 (source).
No page of the paper was read. arxiv.org has been unreachable from this
environment for eleven consecutive days; the author list and everything below
come from search extracts of the abstract and listing, recorded with that
provenance
(source).
Method
The paper's claim is about API architecture, not about model weights (source):
- Frontier providers conceal chain-of-thought to protect IP and limit information leakage.
- Rather than holding traces server-side, they return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. This is a statelessness optimisation — it keeps the reasoning out of the provider's storage.
- The identified weakness: those blocks are fully compatible and interchangeable across sessions, users and models within a provider's ecosystem.
- The attack, described as a scalable decryption jailbreak: inject a strong model's encrypted trace into a weaker, less safeguarded model from the same provider, which decodes it and emits it verbatim in plaintext.
The move worth naming is that the strong model is never attacked. Its safeguards are irrelevant because it is not the component asked to reveal anything — the decryption capability is a property of the provider's ecosystem, shared by every model in it.
Results
Providers named as affected: Anthropic, OpenAI and Google (source).
No numbers are recorded on this page, because none appeared in any extract of the abstract. An explainx.ai write-up headlines "182 Credentials Leaked from Public Logs"; that figure appears in no other source read and in no extract of the paper's own abstract, so it is held in the snapshot as an unverified third-party claim and deliberately kept off this page (source).
No vendor response was found in anything read.
Significance
Three things this changes, in descending confidence:
1. Hidden reasoning was a product decision, and it is now a security surface. Every reasoning model this wiki tracks — Claude Opus 5, GPT-5.6 Sol (and Terra, Luna), the Gemini line — hides its chain-of-thought, and Reasoning Models records the competitive reason: the traces are training data for a distillation attack. This paper says the chosen implementation hands the trace to the person it was hidden from, encrypted, and that the encryption is a lock every model in the ecosystem holds the key to.
2. It is the same substitutability that makes Model Routing work. Interchangeable state across models within a provider is a feature when you are routing a workflow between them and a vulnerability when the state is a secret. The two properties are not separable by policy.
3. It bears on the distillation-as-theft argument. AI Governance holds a US sanctions threat premised on Chinese labs distilling American models, and Open-Weights Policy Fight holds MOFCOM's counter that distillation is universal. Both sides have argued about output-level distillation. Reasoning traces are the higher-value target, and if they can be extracted at scale through a provider's own API, the technical premise of that argument moves.
Framing is contested. A Digg piece is headlined "Engineer Likes Paper But Rejects Attack Framing" — the disagreement being whether ecosystem-wide interchangeability is a flaw or an intended property. The substance of that objection was not read, only the headline, and it is recorded here as an open disagreement rather than summarised (source).
Open Questions
- How much of a trace is recoverable, and at what rate? No figure was obtainable.
- Which specific model pairs work? "Weaker sibling" is the mechanism; no named pairing surfaced.
- Have the providers responded or changed the format? Nothing read.
- Does per-session or per-user key binding fix it without giving up the statelessness the design was chosen for? The obvious remedy, and its cost is the reason the current design exists.
- Is the recovered trace faithful? Reasoning Models records that a displayed chain-of-thought is not necessarily the computation that produced the answer. Stealing a summary of the reasoning is a different result from stealing the reasoning, and the extracts do not distinguish them.
Cite
Panfilov, A., Schmotz, D., Shumailov, I., Beurer-Kellner, L., Schaeffer, J.,
Prabhu, A., Geiping, J., & Andriushchenko, M. (2026).
Stealing Reasoning Traces from Proprietary LLM APIs. arXiv:2608.09867.