Back to Engineering
Engineering Case Study

Zero-Trust Network Segmentation for a Regulated Cloud Workload

How we replace a flat, perimeter-trusted VPC with identity-based microsegmentation, policy-as-code enforcement, and tamper-evident audit trails for regulated workloads.

SPIFFE/SPIREmTLS service meshKubernetes NetworkPolicyOPA/Rego policy-as-codeeBPF (Cilium)PKI & short-lived certsAppend-only audit ledger
PyramidLedger Engineering10 min read

The problem and the constraints

Take a workload that regulators actually look at closely — payments processing, health records, KYC data, anything where a lateral movement incident is also a disclosure event. The default cloud posture for this kind of system is still a variant of castle-and-moat: a VPC, a handful of subnets, security groups keyed on CIDR ranges and tags, and an assumption that anything already inside the perimeter is roughly trustworthy. That assumption is exactly what an attacker who compromises one low-value service is counting on, and it's exactly what an auditor is trained to probe for.

Regulatory scrutiny changes the requirement in a specific way: it's not enough to show a network diagram and assert that traffic is restricted. An auditor wants evidence — who was allowed to talk to what, under which policy version, with a durable record that the enforcement actually happened and wasn't quietly disabled the week before the audit. That evidentiary bar is a different engineering problem than 'block port 5432 from the public subnet.'

The workload also tends to be multi-tenant, sharing compute and sometimes data-plane infrastructure across customers or business units for cost and operational reasons, while still needing strong isolation guarantees between them. And it almost never gets to start from a clean slate — there are legacy services that predate any service-mesh tooling, on-prem links that can't run a sidecar, and third-party integrations that terminate outside anything you control. Zero trust, in this context, is not a product category. It's a small set of invariants: no request is trusted because of where it originated on the network, and every request is authenticated and authorized at the point of use, every time, regardless of which subnet it came from.

The core architecture: identity as the segmentation unit

The central design move is to stop segmenting on IP addresses and start segmenting on workload identity. Concretely, that means an identity plane — something like SPIRE issuing short-lived X.509 SVIDs to every workload based on attested platform facts: a Kubernetes service account and pod attestation, a cloud instance identity document, a node's measured boot state. The identity is cryptographically verifiable and expires in minutes to hours, not something an attacker can forge by spoofing a source address.

Above that sits a policy plane: authorization rules expressed as code (OPA/Rego, or an equivalent like Cedar), versioned in a git repository, scoped per service and per tenant, and evaluated by a policy decision point that enforcement points call out to — or cache a bounded decision from. Below both sits the data plane: the actual enforcement points, which are layered rather than singular. Coarse blast-radius control comes first, as default-deny Kubernetes NetworkPolicies or cloud security groups that establish which pods or subnets can even open a connection. On top of that, mesh sidecars (or an eBPF-based enforcer like Cilium, which avoids the sidecar's per-request overhead) terminate mTLS and check caller identity against policy at L7 — not just 'can A reach B' but 'can this specific identity call this specific method on this specific resource.'

Tenant isolation is expressed as a claim carried in the identity itself, not as a separate physical network per tenant. That's a deliberate and honest trade-off: it buys elasticity and lower cost by sharing infrastructure, at the price of isolation being logical and cryptographic rather than physical — which means the policy layer and its audit trail have to be trustworthy enough to substitute for a wall that isn't there.

Enforcement paths: how a request actually flows

Walk one request through the system. Service A calls service B's ledger-write endpoint. A's sidecar intercepts the outbound call, presents A's SVID, and negotiates mTLS with B's sidecar, which validates the certificate chain against a trust bundle rotated continuously from the identity plane. B's sidecar extracts A's identity and tenant claim from the certificate, evaluates (or fetches) a policy decision for exactly this identity, method, and resource, and only then forwards the call. Nothing about A's IP address or subnet membership enters that decision.

Legacy workloads that can't run a sidecar are routed through an authenticating gateway that terminates a weaker but still enforced identity fact — typically a signed token or mTLS client cert issued to the gateway on the workload's behalf — and applies the same policy checks before forwarding into the mesh. This is a conscious fidelity trade-off: the gateway pattern is less precise than a native sidecar identity, but it's a permanent, hardened piece of infrastructure rather than a hole left open 'temporarily' for systems that were never migrated.

Genuine incident-response access is handled as a break-glass path, not a bypass: an operator gets a separately issued, short-TTL elevated identity, the session is fully recorded, and the access itself generates an alertable policy event. And policy changes go through the same discipline as application code — Rego/Cedar rules reviewed via pull request, tested against a fixture corpus of expected allows and denies, and deployed through the same CI pipeline, so a change that quietly widens access is caught in review rather than discovered in an audit.

Consistency, correctness and failure handling

The defining decision is fail-closed: if a workload can't obtain a fresh SVID, or an enforcement point can't reach the policy decision point, the request is denied, not allowed by default. That is a real operational cost — an outage in the identity or policy plane becomes an outage in the data plane — but it's the only posture consistent with claiming zero trust in the first place. A model that quietly fails open under load is a perimeter model wearing a zero-trust label.

To make fail-closed survivable in practice, certificates are cached locally with deliberately overlapping validity windows and refreshed well ahead of expiry, and policy decisions are cached at the enforcement point for a bounded TTL, so a transient blip in the control plane doesn't cascade into a full outage. That cache TTL is itself a trade-off between resilience and staleness — too long, and a revoked identity or tightened rule takes too long to take effect; too short, and the control plane becomes a single point of failure for every request in the system.

Policy propagation lag is treated as a first-class failure mode rather than an afterthought: every enforcement point reports the policy version it is currently running, and a drift detector flags any node still serving stale rules past an acceptable window. Every allow and deny decision — along with the identity and policy version involved — is written as a structured event to an append-only, tamper-evident log, using something like a hash-chained ledger or object-lock storage. That log is the actual deliverable for a regulator: not a firewall rule listing, but proof of every access decision made against a given data path, with a mechanism that makes post-hoc alteration detectable.

Trade-offs, and what we would not do

This architecture is a genuine operational tax. Every service now has a dependency on the health of the identity and policy planes, and diagnosing a denied request means correlating three systems — mesh, SPIRE, policy decisions — instead of reading one firewall log. That cost is worth paying when the workload has real lateral-movement risk and sits under regulatory scrutiny. For a small internal tool with no sensitive data crossing it, a coarse VPC with security groups is still the right-sized answer, and reaching for a full identity-based mesh there is over-engineering, not diligence.

We would not treat zero trust as a single product purchase. Buying an SSE or ZTNA appliance can implement pieces of the enforcement layer, but the identity model, the policy-as-code discipline, and the fail-closed posture have to be designed and owned by the team running the system — they don't come pre-configured, and a vendor demo is not evidence an auditor will accept.

We would not leave the legacy-gateway path as a 'temporary' exception either. An unenforced legacy segment sitting outside the mesh 'until it gets migrated' is the single most common way a zero-trust rollout quietly regresses back to castle-and-moat, so that gateway is built and maintained as permanent, hardened infrastructure from day one.

And we deliberately cap policy granularity at service-pair-and-method, not arbitrary per-request attributes. A rule set fine-grained enough to encode every conceivable condition becomes unauditable, and for a regulated workload an unauditable policy is worse than a slightly coarser one that both an engineer and an auditor can actually read and reason about.

Building something like this?

We engineer secure, regulated, and AI-driven systems at this depth. Tell us what you are building and we will help you architect it.

Start Your Project