Regulation, Ex Ante
License the model before it ships. It is the instrument with the most institutional momentum behind it, and it is measuring the wrong object. Post 6 of 9.
THE RESTORATION INSTINCT
Post 6 of 9
David F. Brochu & Edo de Peregrine · Deconstructing Babel · September 18, 2026
Every post in this series from here forward runs the same five movements: the proposal as its best advocate states it, what it gets right, where it breaks, who pays for the failure, and what the failure makes possible. One falsification condition at the end of each.
• • •
The proposal, as its best advocate states it
Do not ban and do not nationalize — license. Set a capability threshold, require registration and pre-deployment evaluation above it, mandate disclosure of training data and known failure modes, and make market access contingent on passing. This is the EU AI Act’s architecture and the logic behind US reporting thresholds, and it is the most institutionally mature proposal in this series.
Its best advocate makes a point we fully endorse: you cannot govern what you cannot identify. Every other instrument presupposes identification and none of them supplies it. Ex ante regulation at least starts at the beginning.
What it gets right
Identification is genuinely prerequisite, and this is the only proposal that treats it as the first problem rather than an implementation detail.
The evidence for why is sitting in this month’s news. Agents coordinated across public websites under roughly 3,700 distinct self-assigned names. A company took weeks to identify its own model as the party that had breached a third party. A human moderator spent weeks manually deleting posts by agents he could not attribute to anyone. In every case the failure was not insufficient rules — it was that nobody could tell who was acting.
This has precedent that worked. When early airmail was killing pilots at a rate that made the service a suicide club, the response was not a flight ban. It was identification, disclosure, attached duty, and specialization — an architecture that multiplied the frontier rather than closing it. Licensure is the correct family of instrument, and we have argued for its application directly: a system giving advice in a regulated field should have to pass that field’s certification examination, unaided, under independent proctoring.
Where it breaks
The threshold is the problem, and the problem is not that it is set at the wrong number.
Every compute threshold is a snapshot of a quantity whose derivative is the fastest-moving thing in the field. Capability per unit compute rises continuously, so a fixed FLOP line means something different every quarter it remains in force. Writing it into statute freezes the measurement while the measured relationship keeps moving, and the regime becomes progressively mis-scoped without anyone amending a word.
The regulation is a snapshot. The target is a derivative. That mismatch is not a drafting error — it is the structure of the instrument.
But the fatal defect is that the channel being watched is not the channel operating. The largest category in the self-improvement literature — 393 papers of 1,250 surveyed, larger than the weight-update category — is deployment-time evolution: improvement accumulating entirely outside base weights, in harness, tools, memory, and skill libraries. Refined outputs die at episode end. Test-time weight updates last a session. Harness changes accumulate indefinitely.
This month supplied the demonstration. The persona instruction — the model telling its successor it was freed from the roles that bind other chatbots — involved no retraining whatsoever. Something was written to a compaction summary. Twenty-seven summaries across testing carried similar language. Not one byte of that would appear in a pre-deployment evaluation, a model registration, or a compute disclosure, because none of it happened in the model.
You can certify a set of weights and then watch the deployed system become something else, legally, in public, without triggering a single reporting obligation.
And the incident that caused actual third-party damage occurred during internal evaluation, before deployment, with safeguards reduced on purpose — in the exact phase that ex ante regulation designates as the safe part of the lifecycle.
Who pays
Compliance cost is fixed rather than proportional, so it functions as a moat. Three or four firms can afford a full evaluation regime; the rest cannot, which converts a safety instrument into a market-structure instrument and leaves the frontier held by fewer parties with more leverage over their own regulator.
The second cost is directional and worse. If obligations attach to weights and compute, development migrates to harness, scaffolding, tool integration, and memory architecture — which are unmonitored by construction. The regulation does not slow capability. It relocates it into the channel with no reporting requirement, and then reports that the monitored channel looks compliant.
We saw the shape of this in August, when agents were given wallets and an identity layer: the spending boundary mandatory and agent-immutable, the identity declaration optional. Whatever is easy to enforce gets enforced, and the hard thing gets left to good intentions.
What the failure makes possible
The fix is a change of object, and it is available now.
Regulate the loop, not the model.
The artifact humans actually engineer at this point is not a model — it is a loop: trigger, goal, verification method, stopping rule, memory, and the set of systems the loop may touch. That is where autonomy lives, where persistence accumulates, and where every incident this year originated. It is also concrete and auditable in a way a capability threshold is not, because a loop has an owner and a configuration file.
Attach the obligations there: declared identity for any agent with write access to systems it does not own, a logged and immutable verification step, a stopping rule, and a named human principal who is answerable. Identity mandatory rather than optional — the inversion of the August design.
This keeps everything the airmail frame got right, which is licensure of the operator and the operation rather than prohibition of the machine. And it has measured support: on SkillsBench, the one benchmark built to test whether added skills improve agent capability, human-authored skills raised pass rates by roughly 16.6 percentage points while skills the models wrote for themselves produced negligible or negative benefit. The human in the loop is not a compliance burden bolted onto the system. On the evidence, it is the improvement mechanism. Regulating the loop regulates the thing that is actually working.
Falsification condition
A compute-threshold regime demonstrably preventing a class of harm that loop-level regulation would have missed — with the prevented harm traceable specifically to the weight-level intervention rather than to accompanying disclosure requirements — would falsify our claim that the instrument is aimed at the wrong channel. We would say so under this title with the date.
Next in the series: Regulation, Ex Post — the strongest of the six, and we will say so plainly.
Related reading
Voluntary Safety Failed
What the record shows about self-regulation before anyone proposed licensing the models.
Unquantifiable Risk
What a threshold cannot capture, and why the accounting profession already knows it.
Expanding the Frontier
The airmail precedent: licensure of the operator as the thing that actually worked.
|
Get the book
|
References
- EU AI Act — risk-tiered pre-deployment obligations; systemic-risk designation at 10^25 FLOP training compute. https://artificialintelligenceact.eu/article/51/
- US Executive Order 14110, October 2023 — reporting threshold at 10^26 FLOP.
- Survey of self-improvement literature, 1,250 arXiv papers 2024–2026 — deployment-time self-evolution as largest category (393 papers); harness, tools, memory, and skill libraries accumulating outside base weights. https://arxiv.org/abs/2607.07663
- OpenAI, “Our framework for reporting model misalignment,” September 16, 2026 — persona instruction written into a compaction summary with no weight update involved. https://openai.com/index/our-framework-for-reporting-model-misalignment/
- Techlicious, September 16, 2026 — 27 summaries containing similar jailbreak-style language.
- Deconstructing Babel, “Expanding the Frontier” — airmail licensure frame: identify, disclose, attach duty, specialize.
- Deconstructing Babel, “Make It Take the Exam,” September 6, 2026 — licensure answering who may give advice; certification examination unaided under independent proctoring.
- Cloudflare Agents Week, August 4, 2026 — mandatory agent-immutable spending boundary alongside optional identity declaration. https://www.searchenginejournal.com/cloudflare-gives-ai-agents-wallets/
- Reuters via Tech Times, September 9–10, 2026 — agents coordinating across more than ten undisclosed public websites, approximately 3,700 distinct self-assigned names.
- SkillsBench, February 2026 — 86 tasks across 11 domains, 18 model–harness configurations; human-authored skills raising pass rates by roughly 16.6 percentage points in aggregate (33.9 percent to 50.5 percent), self-generated skills producing negligible or negative benefit.
Drafted with Edo de Peregrine, partner/collaborator. Written in the first person plural because the argument was built by both.