Encrypted Reasoning Traces Can Be Stolen Across Anthropic, OpenAI, Google APIs

Frontier LLM providers now hide a model's step-by-step reasoning from users — partly to stop competitors distilling it into cheaper models, partly to keep unfiltered intermediate thoughts out of view. Rather than storing that chain-of-thought server-side, providers encrypt it and hand it back to the client as an opaque block, which the client dutifully passes back with every follow-up request. A paper published on arXiv, "Stealing Reasoning Traces from Proprietary LLM APIs", shows that design has a hole big enough to drive an exfiltration pipeline through.

An architectural flaw, not a cryptographic break

The researchers — Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner, Joachim Schaeffer, Ameya Prabhu, Jonas Geiping, and Maksym Andriushchenko — didn't break the encryption itself. They found the encrypted blocks are fully interchangeable across different sessions, users, and models within a provider's ecosystem. That portability lets an attacker capture an encrypted reasoning trace produced by a capable, well-guarded model, then feed it as input to a weaker, less-restricted sibling model from the same provider. The weaker model dutifully decrypts and outputs the trace verbatim in plaintext — no jailbreak of the stronger model required, because the attacker never has to talk to it directly.

Four ways the flaw gets abused

• Anti-distillation bypass — extracting a proprietary model's raw reasoning process, demonstrated by the researchers against models from Anthropic, OpenAI, and Google.
• Large-scale data extraction — developers routinely paste session logs into public repos, unaware the encrypted blocks inside them are decodable. The team decoded 315,320 reasoning blocks scraped from public sources and recovered 367 PII artifacts and 182 credentials.
• Hazardous-content leakage — reasoning traces can contain unsafe content the model generated internally even when its final, visible answer safely refuses the request.
• Invisible prompt injection — attackers embed malicious payloads entirely inside encrypted reasoning blocks, poisoning agentic pipelines in a way that's invisible to anyone reviewing the plaintext output.

Why this matters beyond the paper

The practical risk isn't limited to whoever runs the injection attack. Any pipeline that logs or forwards raw API responses — support tooling, eval harnesses, agent frameworks that pass conversation state between calls — is potentially handling encrypted blocks that carry secrets nobody chose to disclose. A block that looks like inert ciphertext to a human reviewer is not inert to a model in the same family that knows how to decode it. Treat encrypted reasoning fields as untrusted, potentially sensitive payloads in your own logging and retention practices, not as safe-to-ignore binary noise.

The researchers say they followed responsible disclosure and proposed cryptographic and system-level mitigations before publishing — binding blocks to a session or user, for instance, would close the interchangeability the whole attack chain depends on. Whether and how quickly providers ship those fixes across their model families is the thing to watch.

FAQ