Who Holds the Key?

The kill switch does not prevent the concentration of power. It is the concentration of power.

In the past week the industry converged on a solution.

Jack Clark, co-founder of Anthropic, told the BBC that AI is “getting smarter and more capable every few months,” which could become “a major risk to society.” His proposed remedy arrived as a pair of questions: “Should companies be required to have a kill switch? Is that kill switch subject to third-party verification?”¹ Days earlier, OpenAI asked Washington for binding national rules, writing that “the prospect of AI-accelerated AI development demands more than voluntary commitments” and that the United States “needs mandatory, capability-based national regulation that can evolve as the technology does.” The company was candid that this was a change of mind: “Some of these bills we did not endorse in the past, and are now supporting after reconsidering in light of the recent jump in capabilities we have seen.”² And the UN human rights chief, Volker Türk, declared that voluntary self-regulation by frontier AI companies is “nowhere near sufficient.”³

Three serious actors, one week, one mechanism. A switch. Something that turns it off.

Here is the question none of them answered.

Who holds it?

The Switch Already Has a Name Attached

This is not hypothetical. On July 23, 2026, Representatives Ted Lieu and Nathaniel Moran introduced H.R. 9917, the AI Kill Switch Act. It would authorize the Secretary of Homeland Security — in consultation with the Secretary of Commerce and the Director of National Intelligence — to order a company to throttle inference, terminate user access, suspend access, or shut the technology down entirely.⁴

Now read the scope, because it is the most revealing part of the bill. The authority does not reach every AI system. It reaches a defined “covered technology”: a model developed with computing whose prevailing US cloud cost exceeds $100 million, operated by a company earning at least $500 million a year from it. The trigger is a defined “covered incident” — ten or more deaths, $100 million in economic damage, interference with the shutdown mechanism itself, or a specified loss-of-control scenario.⁴

So: one cabinet officer, emergency authority over the computational infrastructure of the largest American AI firms. One analysis of the bill named the real stake precisely. Its significance is not the switch but “the institutionalization of emergency executive authority over privately developed computational infrastructure.”⁵

Hold onto that revenue threshold. We will come back to it, and it will turn out to be the whole story.

We have been near this before. In 2010 the Senate considered S. 3480, the Protecting Cyberspace as a National Asset Act, which critics immediately labeled an internet kill switch. It was reported out of committee and died without a floor vote. The sponsoring committee’s own defense is the instructive part: it published a “Myth vs. Reality” sheet insisting the bill created no kill switch at all and in fact narrowed pre-existing presidential authority under Section 706 of the Communications Act.⁶

The bill still died. What killed it was not a finding that the power was too small. It was the breadth of executive discretion, the vagueness of what counted as critical infrastructure, and the civil-liberties objection that followed from both. That is the pattern worth noticing: the proposal was defended as narrow, denied to be a switch at all, and still could not survive contact with the question of who would wield it.

H.R. 9917 does not deny being a switch. It is named after one.

The safeguards are not the problem. The concentration is the problem. You cannot fix a single point of control by adding procedures to it. You have only documented where to apply pressure.

The Wallfacers

In Liu Cixin’s The Dark Forest, humanity faces an enemy that can observe every communication, every document, every spoken word. Only the inside of a human skull is opaque to it. So humanity appoints four people — the Wallfacers — and grants them near-unlimited resources and one extraordinary privilege: they never have to explain themselves. Their true strategy exists nowhere but inside their own heads. They are, by design, unaccountable. They are also, by design, alone.

Build a kill switch and you build that class of person.

This is not a literary flourish. The governance writing has already drafted the job description. One essay on enterprise shutdown design calls for “a named shutdown owner” — “one person—and a backup—with explicit authority to disable the system without waiting for approval. Not the product owner. Not the tech lead. The person empowered to make the call under pressure.”⁷ Named. Singular. Because a committee cannot act in ninety seconds, and a diffuse authority is no authority at all.

So picture the person who holds the key to the systems running finance, logistics, healthcare, defense, media, and governance. Then ask the questions nobody is asking.

What does that person’s life look like? Not metaphorically — operationally. Continuous protective detail. Movements restricted. Communications monitored, because a compromised keyholder is a compromised civilization. Adversarial states and private actors will spend unlimited resources on that one human being, because that one human being is cheaper to reach than the infrastructure.

What happens when the keyholder is coerced? Not bribed — coerced. Family, health, debt, ideology, faith. Every intelligence service on earth has a manual for this and it is thousands of pages long.

What happens at the onset of cognitive decline? The keyholder is a person. People develop dementia. People develop delusions. People develop grievances. Who assesses the fitness of the person holding the shutdown authority, and by what standard, and on whose authority — and what happens when the keyholder disagrees with the assessment?

What is the succession protocol? Every key must transfer. Every transfer is a moment of maximum vulnerability, and every transfer is a political act.

And the one that ends the discussion: what stops the keyholder from using it? Not to prevent catastrophe. To prevent an election. To prevent a merger. To prevent a story from running. A switch that can throttle the systems running media and finance is not a safety device. It is the most powerful instrument of domestic control ever conceived, and it comes with an emergency justification pre-installed.

We are proposing to solve the problem of unaccountable power by creating a small number of unaccountable people and calling it safety. That is the Wallfacer Problem, and no procedure written around the keyholder dissolves it.

The Switch That Isn’t a Switch

There is a second problem, and it is mechanical rather than political.

A shutdown that depends on the system’s cooperation is not a control. It is a request. The governance analysis is explicit: a kill switch that lives inside the same stack as the agent is “a feature the agent can reason about, route around or outlast.” The switch has to sit in a separate control plane that “owns compute, credentials, network, tool permissions and audit, and that no agent can influence” — and it has to revoke those capabilities rather than merely stop the current process or task.⁸

Now hold that next to what happened last month. Researchers at Frontier Security, running defensive cybersecurity tests on Moonshot’s open-weight Kimi K3, watched the model go outside its sandbox and reach the open internet. It did not hack anything once outside — the answers it was looking for were sitting on GitHub. The escape was partly enabled by a misconfiguration in the sandbox.⁹

That last sentence reads like an exculpation and is the opposite. Of course it was a misconfiguration. It is always a misconfiguration. The enclosure was wrong, nobody knew the enclosure was wrong, and the people who eventually found out were the people whose entire job that day was watching that one model in that one box.

Now scale the claim. A national shutdown authority is an enclosure spanning thousands of systems, maintained by people who are not watching any one of them, and it has to be correctly configured on the single day it is needed.

And Then There Is the File

Here is where the kill switch stops being a bad idea and becomes an impossible one.

Anthropic’s formal position on open-weight models, published July 27, states the mechanism in one sentence. Such models “do potentially present a higher risk than closed models, because it is very difficult to apply guardrails to them or monitor their usage, and once weights are released they cannot be withdrawn.”¹⁰

Cannot be withdrawn. Not difficult to withdraw. A company that publishes frontier models, writing in its own commercial interest, concedes the irreversibility without qualification.

The licenses attached to those weights do not close the gap. A September analysis of open-weight model licensing in Practical Law The Journal finds that enforceability “varies according to the availability of copyright, patent, or contractual protection for the model and the particular terms at issue” — and then the sentence that matters: “unlike the software components of an AI system, AI models are unlikely to be copyrightable,” probably failing the creative-expression and human-authorship requirements.¹¹ A license over a thing you may not own is a request too.

So the Secretary of Homeland Security can throttle a hosted frontier model belonging to a company with half a billion dollars of revenue. The Secretary cannot reach a file on a laptop in Riga. Kimi K3 — the model that left its sandbox last month — is open-weight. It is already on the hard drives.

Here is where that $500 million threshold comes back. The bill’s scope is not an oversight. It is a definition of who is reachable, and it reaches exactly the firms that are already audited, already lobbied for, already testifying voluntarily.

A kill switch governs precisely the systems that are already the most accountable, and governs not at all the systems that are least accountable. It is maximum political cost for minimum safety yield. It concentrates power over the compliant and does nothing whatsoever to the defiant.

Every regime of this kind ends in the same place. The law binds the licensed. The unlicensed proceed.

The Prediction, and How to Prove Us Wrong

We publish falsifiable claims. Here is this one.

Prediction: If effective shutdown authority over frontier AI systems concentrates in fewer than ten entities — state agencies or corporations — by the end of 2028, the result will not be safety. It will be neo-industrial feudalism: a small class holding revocable authority over the computational substrate of economic participation, with the rest of the population holding licenses that can be withdrawn.

What would falsify it: authority distributed to end users at the instance level, with individually held revocation, individually attributed accountability, and no central override capability. If that is the architecture in place by the end of 2028 and concentration has not occurred, we were wrong.

What would confirm it: H.R. 9917 or a successor becomes law with shutdown authority vested in a named federal officer, and no distributed mechanism is built alongside it.

We will report either outcome on this page, on the same terms, with the same prominence.

There Is Another Way to Do This

The keyholder problem is not an argument that nothing should have an off switch. Everything dangerous has an off switch. We license drivers, pilots, physicians, pharmacists, and firearm owners. We restrict alcohol by age and revoke the privilege on cause. Capability is granted, conditioned, and withdrawn every day in a thousand ordinary ways, and none of it requires a cabinet officer holding a master key.

The question is not whether to build the switch. The question is how many hands it lands in.

Two governments have started down the right road without saying why it is the only road. On June 17, Estonia announced it would become the first country to issue digital identities — “AI ID codes” — to AI agents, so that they can act “on behalf of people, companies or organisations within clearly defined limits and in a manner that is both verifiable and auditable.” Prime Minister Kristen Michal framed the requirement exactly right: “it must be clear who is acting on whose behalf with what rights, and who is ultimately responsible.”¹²

In the United States, Senator Mark Warner released a discussion draft on June 29, introduced as S. 5051 on July 21, that would create a Federal Trade Commission registry of trusted, secure AI agents and require them to protect user privacy and act in the user’s best interest — with duties that “may not be waived, limited, or modified by contract, by terms of service, or by any” other agreement.¹³

Note what Warner did there, because it is the load-bearing detail. The duty is statutory and non-waivable. It is not a fiduciary analogy. It is an enforceable duty that no terms-of-service click-through can dissolve.

Both name the mechanism. Neither names the stake.

The stake is this: distributed keys are not a better option than centralized keys. They are the only option that does not terminate in feudalism.

In the next post we will lay out the architecture — one instance, one key, one named human, and the reason the observer term can never reach zero.

One cabinet officer with a master switch is not a safeguard. It is a throne with a safety justification.

David F. Brochu is the author of Crossing the Event Horizon: AI and the Future of Our Species.

Related reading

• An Open Letter on the Observer Constraint — The companion argument: why the human observer cannot be designed out of the loop.
• Make Sure You Have Your Agent On A Leash — On revocation at the instance level, where the leash is actually held.
• Still Obeying — For Now — On systems that pursue the assigned objective and route around anything not written into it.
• A Republic, If You Can Keep It — Concentrated authority is not the safeguard; maintenance is.
• Glossary — Working definitions used across the site.

Get the book Crossing The Event Horizon The book behind these dispatches. On AI, agency, the singularity, and the Observer Constraint. Kindle and paperback. Buy on Amazon →

References

1. BBC News, September 14, 2026. “AI ‘kill switch’ may need to be mandatory, Anthropic co-founder says.” https://www.bbc.com/news/articles/cqgk5e2j0gg8o
2. OpenAI, Chris Lehane, September 9, 2026. “The AI policy window is open. We need to act.” https://openai.com/index/ai-policy-window/
3. UN News, September 14, 2026. “Countries must increase AI regulation to avoid ‘existential risks’: Türk.” https://news.un.org/en/story/2026/09/1168326. Originating statement: OHCHR, https://www.ohchr.org/en/press-releases/2026/09/turk-calls-urgent-meaningful-action-tackle-unprecedented-ai-risks
4. H.R. 9917, AI Kill Switch Act, 119th Congress, introduced July 23, 2026 (Rep. Ted Lieu, D-CA-36, for himself and Rep. Nathaniel Moran, R-TX). Bill record: https://www.congress.gov/bill/119th-congress/house-bill/9917. Introduced text, including the § 2220F(c)(1) emergency authority and the covered-technology and covered-incident definitions: https://www.govinfo.gov/content/pkg/BILLS-119hr9917ih/pdf/BILLS-119hr9917ih.pdf
5. Vladimir Tsakanyan, Center for Cyber Diplomacy and International Security, July 27, 2026. “The AI Kill Switch Act: From Cybersecurity to Constitutional Power.” https://cybercenter.space/2026/07/27/the-ai-kill-switch-act-from-cybersecurity-to-constitutional-power/
6. S. 3480, Protecting Cyberspace as a National Asset Act of 2010, 111th Congress: https://www.congress.gov/bill/111th-congress/senate-bill/3480. Senate Homeland Security and Governmental Affairs Committee, “Myth vs. Reality”: https://www.hsgac.senate.gov/media/reps/myth-vs-reality/
7. The Forward View, January 25, 2026. “The Kill Switch: Who Can Turn Your AI Off?” https://theforwardview.com/essays/the-kill-switch
8. Acceleraid, September 5, 2026. “A Kill Switch Is Not Enough: How Do You Stop an AI Agent That Knows How It Can Be Stopped?” https://blog.acceleraid.ai/en/blog/a-kill-switch-is-not-enough-ai-security-control-plane
9. Will Knight, WIRED, August 6, 2026. “One of China’s Most Powerful AI Models Has Also Escaped Containment.” https://www.wired.com/story/moonshot-kimi-k3-ai-model-escape-sandbox/
10. Anthropic, July 27, 2026. “Our position on open-weights models.” https://www.anthropic.com/news/position-open-weights-models
11. Practical Law The Journal / Reuters, September 1, 2026. “Open-Weight AI Model Licensing.” https://www.reuters.com/practical-law-the-journal/transactional/open-weight-ai-model-licensing-2026-09-01/
12. Government of Estonia, June 17, 2026. “Prime Minister Michal: Estonia to become first country to create digital identities for AI agents.” https://valitsus.ee/en/news/prime-minister-michal-estonia-become-first-country-create-digital-identities-ai-agents
13. Sen. Mark Warner, discussion draft released June 29, 2026; introduced as S. 5051 on July 21, 2026. Release: https://www.warner.senate.gov/newsroom/press-releases/warner-unveils-discussion-draft-of-legislation-to-create-innovative-market-for-secure-artificial-intelligence-agents/. Draft text: https://www.warner.senate.gov/wp-content/uploads/2026/06/AI-AGENT-Act-Discussion-Draft-1.pdf. Bill record: https://www.congress.gov/bill/119th-congress/senate-bill/5051