GPT-6 Astra's Running Routes Show Agentic AI's Transparency Problem

A 27-minute agent run, and a home address

On 12 September 2026, independent researcher Simon Willison asked ChatGPT Work, running GPT-6 Astra (Max), to generate 5K and 10K running-route loops starting from his home address using OpenStreetMap data (Simon Willison's Weblog). The agent worked autonomously for 27 minutes and returned exactly what was asked: an embedded map visualisation plus downloadable GPX and GeoJSON route files.

Asked how the routes were built, the assistant described its own pipeline: it used Nominatim to geocode the address, Overpass to pull local OSM road and trail data, then calculated the loops locally. On paper, that's a coherent, verifiable approach.

The catch: the code that actually ran was invisible

Willison's complaint is the part that should concern security teams more than the routing logic: "the actual code it ran and exact details of what it did weren't visible to me in the ChatGPT UI." He calls this lack of transparency an anti-feature — not a missing convenience, but a deliberate gap in what an operator can inspect about an agent that just spent nearly half an hour acting autonomously on their home address.

Compaction erased the evidence

The gap compounds when Willison later asked ChatGPT to show him the Python it had actually executed. It couldn't: the conversation had gone through thread compaction, and the pre-compaction tool-call transcript was gone. The one artefact that could confirm exactly what was sent to Nominatim and Overpass, and how the address was handled, had already been discarded by the platform itself.

Why this is a security problem, not a UX complaint

For a personal running-route request, the stakes are low. For an enterprise running agentic assistants against production systems, source code, or customer data, the same pattern is a control failure: an autonomous agent executed unreviewed code against sensitive input, and nothing guarantees the execution record survives the session. That breaks the basic assumptions incident response and audit depend on.

• Tool-call transcripts should be treated as security-relevant logs, not disposable UI state, and retained independently of context-window compaction.
• Any autonomous agent that ingests PII — a home address, in this case — should have that data flow logged and reviewable, not reconstructed after the fact from the agent's own summary.
• Security review of agentic AI tools should explicitly test whether tool-call and code-execution history survives compaction, session restarts, or thread handoff.

The takeaway for teams adopting agentic AI

Willison's own proposed fix — that systems using compaction preserve pre-compaction text and expose it via tool calls — is a reasonable baseline the industry hasn't converged on. Until it does, treat what an agent tells you it did, after the fact, as a reconstruction rather than a log.