How Researchers Cracked Encrypted Chain-of-Thought in Claude, GPT and Gemini
A new paper shows that the encrypted reasoning blocks Anthropic, OpenAI and Google return from their APIs can be replayed into a weaker sibling model and jailbroken into plaintext — defeating anti-distillation protections and, in the wild, exposing PII and credentials.
Key Takeaways
- Anthropic, OpenAI and Google encrypt the raw chain-of-thought their reasoning models return via API, but every model in a given family shares the same encryption key.
- Researchers replayed an encrypted reasoning block from a stronger model into a weaker sibling and jailbroke it into transcribing the plaintext — turning the small model into a decryption oracle.
- Claude Haiku 4.5 was reported as the easiest target, via an assistant-turn-prefix behaviour Anthropic later removed in its 4.6 models.
- Scanning public repositories for logged reasoning blocks, the researchers recovered 315,320 decrypted traces containing 367 PII artifacts and 182 credentials; all three vendors have since patched the issue.
A paper published this month, Stealing Reasoning Traces from Proprietary LLM APIs, shows that the encrypted chain-of-thought (CoT) blocks returned by reasoning-model APIs are not as sealed as their design suggests. The researchers — led by Alexander Panfilov, with seven co-authors — found a way to extract a frontier model's hidden reasoning in readable plaintext, across Anthropic, OpenAI and Google's APIs.
Why the reasoning is encrypted at all
Reasoning models generate a long internal chain-of-thought before answering. Providers don't show this to users in raw form: it can be unfiltered, and — more to the point commercially — a competitor could use it to distil a cheaper model that mimics the frontier model's reasoning quality. So the API returns the trace as an encrypted block, which the client passes back unread on the next turn to preserve context across a multi-step or agentic conversation.
One key per model family
The vulnerability the paper describes is architectural: as Simon Willison notes, "every model under the same family used the same encryption key." That means an encrypted block produced by a top-tier model will still decrypt correctly if it's fed into a smaller, weaker model from the same lineup — the models are, in effect, interchangeable decoders for each other's ciphertext.
Turning a small model into a decryption oracle
The attack replays a captured encrypted reasoning block into the weakest available sibling model, then prompts it to transcribe whatever it just decrypted, verbatim. Because smaller models are typically easier to jailbreak, this bypasses the safeguards that would normally stop a model from repeating hidden system content. Willison reports that Claude Haiku 4.5 was the easiest to attack, exploiting an assistant-turn-prefix behaviour that Anthropic subsequently removed in its 4.6 model generation.
What ended up exposed
Beyond defeating anti-distillation protections, the same technique has a privacy angle: reasoning traces logged by tools, agents, or developers and posted publicly are themselves decryptable this way. The researchers scanned public repositories and recovered 315,320 decrypted reasoning blocks, from which they extracted 367 PII artifacts and 182 credentials — sensitive content that had been sitting, encrypted but crackable, in reasoning traces nobody thought a third party could read.
Patched — for now
The researchers disclosed the issue to all three vendors before publication. Per Willison's summary, "all model providers acknowledged the receipt of our report and subsequently we were unable to launch the same attacks" — meaning the specific replay path has been closed. It doesn't remove the underlying tension: shared keys across a model family are also what let some products swap models mid-conversation without losing reasoning context, so the fix is a narrower mitigation rather than a redesign of the whole scheme.
Frequently Asked Questions
Does this mean Claude, ChatGPT or Gemini reasoning is currently readable by attackers?
No — the researchers privately disclosed the flaw to Anthropic, OpenAI and Google before publishing, and reported that the specific replay-and-jailbreak technique no longer worked against the providers' APIs afterward.
Why do providers encrypt chain-of-thought instead of just hiding it?
Reasoning traces are useful context for follow-up turns in a conversation, so the API returns them to the client to be passed back later. Encrypting rather than omitting them keeps that context available to the client without exposing the raw reasoning text.
Is this a prompt injection technique?
It's closer to a decryption jailbreak: an encrypted reasoning block from one model is fed to a weaker sibling model, which is then coaxed into transcribing the plaintext it just decrypted.