What Anthropic's Own Numbers Say About Agentic Coding-Tool Risk

Simon Willison published an edited transcript of a fireside chat with Cat Wu and Thariq Shihipar from Anthropic's Claude Code team, recorded at the AI Engineer World's Fair. Most of the coverage will focus on developer-experience anecdotes, but the security detail is the part worth reading closely: it's an unusually candid look at how a frontier lab defends the coding agent it ships to its own engineers before it ships it to anyone else.

Defense-in-depth, not a single trust boundary

Auto mode is Claude Code's headline safety feature — Wu said it's used by almost everyone inside Anthropic and has been hardened internally since January, ahead of a March public rollout. Mechanically it's a Sonnet-based classifier that judges each tool call and the surrounding conversation before letting an action through, including requests that try to reach outside the sandbox (a network call, for example).

The classifier is not the whole story. Anthropic's own engineering write-up on containing Claude describes an OS-level sandbox underneath it — Seatbelt on macOS, bubblewrap on Linux — that allows reads and workspace writes but denies network access by default. According to that post, the sandbox alone drove an 84% reduction in permission prompts, and the classifier on top catches roughly 83% of overeager behaviours before they execute, at a 0.4% false-positive rate on benign commands. The flip side is stated plainly: about 17% of overeager actions still get through the classifier. That gap is exactly why the sandbox exists as a second, independent boundary rather than a belt without braces.

Prompt injection: 'lower than the average human reviewer' isn't zero

Wu framed Anthropic's confidence in blunt terms: for prompt injection and data exfiltration, she said the risk is now lower than an average human reviewer's — the product of, in her words, thousands of evals and multiple external red teamers building adversarial environments. She was careful to add that this doesn't catch everything. Shihipar described the approach as a Swiss-cheese model: reinforcement learning against known attack patterns, sandbox containment, and auto-mode review stacked so that a single missed layer doesn't become a full compromise. For teams evaluating agentic coding tools, that's the useful mental model — ask which layer catches what, not whether the vendor claims the model itself is safe.

Credentials the agent can use but never sees

One detail is worth lifting directly into your own architecture reviews. Wu described Anthropic's internal Datadog credentials as usable by the agent but not accessible to it — injected on the fly at request time rather than held in the agent's context or environment. That separates "the agent can call this API" from "the agent can exfiltrate this secret," which is precisely the distinction that matters when the same agent is also reading untrusted input (issue comments, fetched web content, Slack messages) that could try to instruct it to leak what it holds.

Claude Tag: a multiplayer surface needs its own identity

Claude Tag, Anthropic's newer Slack-based coding agent, runs entirely on auto mode, which Shihipar said is deliberate given how many attack vectors a collaborative, multiplayer chat surface opens up — a feedback channel that users can post into, and now the bot is reading it, as he put it. The mitigation is identity separation: Claude Tag can be provisioned with its own credentials rather than acting under an individual user's authority, so a successful injection compromises a scoped bot identity, not a person's access.

What this means if you're bringing a coding agent in-house

• Treat a model-based approval classifier as one layer, not the control — demand the vendor can name the OS-level or infrastructure boundary underneath it (filesystem, network, process isolation).
• Ask for false-negative rates, not just headline catch rates — a vendor quoting an 83% block rate is implicitly telling you 17% needs a second layer.
• Audit how the agent holds credentials for the systems it touches — injected-at-call-time beats held-in-context every time you also feed the agent untrusted input.
• Give any chat-native or multiplayer agent (Slack, Teams, ticketing bots) its own scoped identity, separate from the humans it works alongside.