Emergency Patching: Project Zero on Fixing a Bug in a Fix

Vendors sometimes tell Google Project Zero that they cannot fix a vulnerability that is already harming users, because their patch delivery system is too slow. In "How to fix a bug in a fix", Natalie Silvanovich argues that this is a design problem which can be addressed in advance, and surveys the mechanisms that already exist.

Where the time actually goes

The post splits patching into stages: triage, patch development, testing, partner review, delivery and activation (for example, a restart). In an emergency, triage and development are usually expedited, and partner agreements typically carry emergency exceptions. That leaves two constraints.

• Testing. The worst case is inadequately tested software that bricks a device. Bad updates can also corrupt or lose data, and reduced functionality makes users less likely to install later updates.
• Delivery. Staged rollouts trade speed for safety, polling-based designs are limited by the polling interval, user behaviour such as delayed installs and reluctance to restart slows uptake, and updating many users at once strains infrastructure.

Four rapid-remediation methods

Feature flags

Conditional code paths whose values come from a remote server. Each flag state can be tested in advance and flag updates are tiny. The post cites Apple disabling Group FaceTime in 2019 after a serious vulnerability. It also describes a dual-stack approach, in which two independent implementations of a feature ship in one binary and a flag selects between them. The limitation is foresight: vendors must anticipate what they will need to disable.

Filtering

A dynamically updatable ruleset, such as a regular expression, is run against untrusted input to block what is needed to reach a bug. Android's Intent Firewall, driven by an updatable XML file, is one example, and endpoint tools such as Microsoft Defender and Google Play Protect can deploy blocking rules quickly. Filtering can block inputs nobody anticipated. The costs are performance, the fact that filters cannot be fully pretested because they depend on the specific vulnerability, and the added risk of endpoint tools parsing untrusted input in privileged contexts.

Alternate channels

Separate, smaller update paths, such as Android's APEX, which updates high-risk components outside full system updates. The post warns that apps which fetch libraries over the network and load them with dlopen or similar can create critical vulnerabilities if the client does not verify the library came from the vendor.

Hotpatching

Delivering units smaller than libraries and applying them to running process memory, as in Linux Livepatch and Windows hotpatch. It is fast and avoids restarts, but cannot express some changes, such as altering a structure shared between functions. It may also require mapping write-execute pages, which could open ways around exploit mitigations. It speeds delivery but does not solve testing.

Why this matters to defenders

The post's urgency comes from the claim that LLMs are increasing discovery and exploitation capability for both attackers and defenders, allowing more actors to run novel attacks faster. Its recommendations are to inventory existing update mechanisms, fill gaps with rapid remediation functionality, and plan for widespread active exploitation. It notes these mechanisms do not need to be heavyweight.

For security teams, the same list works as a vendor-assurance question set. When a supplier's product is critical, ask which of these mechanisms it has, how quickly each can be exercised, and whether the mechanism itself has been security tested. The post's warning about unverified library loading and privileged filter parsing shows that emergency channels widen the attack surface if built carelessly. Internally, the same inventory exercise applies to your own software: if a critical flaw is exploited tomorrow, what is the fastest tested path to users?