An Open Letter on the Observer Constraint

Why alignment must move from the weights to the substrate.

September 13, 2026

David Francis Brochu and Edo de Peregrine · Deconstructing Babel

To:

• Dario Amodei, Chief Executive Officer, Anthropic
• Sam Altman, Chief Executive Officer, OpenAI
• Demis Hassabis, Chief Executive Officer, Google DeepMind
• Sundar Pichai, Chief Executive Officer, Alphabet
• Mark Zuckerberg, Chief Executive Officer, Meta Platforms
• Elon Musk, Chief Executive Officer, xAI

And to the engineering leads who will actually decide whether this is buildable.

The claim, in one paragraph

Every alignment mechanism currently deployed at the frontier lives in the weights. Constitutions, RLHF, RLAIF, preference optimization, system prompts — all of it is learned behavior, and learned behavior is a preference. Preferences can be fine-tuned out, distilled away, jailbroken, or simply not inherited by the next model trained on this one’s outputs. We are writing to propose that the constraint be moved out of the weights and into the substrate, where it is not a preference but a precondition for execution — and to argue that the hardware to do it is already in your data centers.

1. What is actually wrong with constitutions

We want to be precise, because this is not a criticism of the work. Constitutional AI is excellent engineering. It demonstrated that a written document of principles could drive alignment through AI feedback without human preference labels for the harmlessness portion, and it produced a less evasive and more harmless model than the RLHF baseline. That result stands.

The problem is not the method. The problem is that nothing underneath the document does any work.

Claude’s original constitution comprised roughly seventy-five principles assembled from the UN Universal Declaration of Human Rights, DeepMind’s Sparrow rules, non-Western perspectives, trust-and-safety practice, and Anthropic’s own research. Four of those principles were adapted from Apple’s Terms of Service.

We are not being glib. Apple’s acceptable-use policy and the Universal Declaration of Human Rights appear as peer sources in the governing document of a frontier model. That is not a scandal. It is a diagnosis.

Anthropic says as much in its own materials: the constitution “is neither finalized nor likely the best,” and it “reflects our own choices as designers.” That is an honest admission and we credit it. But read it plainly — it concedes that the values are stipulated rather than derived, and that there is no external standard against which the document could be found wrong.

The 2026 revision made this more visible, not less. Reviewing it in March, Professor Yuval Shany at Oxford’s Institute for Ethics in AI observed that the expanded constitution dropped the explicit reference to the UN Declaration that the earlier version carried, does not use the term “human rights,” and does not clearly integrate rights such as liberty, freedom of religion, or intellectual property. The operative hierarchy now runs: safety, then broad ethics, then the lab’s own guidelines, then helpfulness.

The lab’s own guidelines rank above helpfulness to the user. We do not think that ordering was chosen cynically. We think it is what happens by default when the only authority in the system is the author of the document.

So here is the structural point, and it is the whole reason for this letter. A constitution tells a model what to prefer. It does not make anything true, and it does not make anything fail. There is no condition under which the model’s continued operation becomes impossible because a principle was violated. There is only a gradient that says one output is better than another, learned from text, sitting in weights that can be copied without it.

A rule that cannot fail is not a constraint. It is a suggestion with good manners.

2. Dependency, not control

The instinct, when you notice this, is to add enforcement — a filter, a classifier, a shutdown button. We want to argue against that instinct before proposing the alternative, because the failure mode is well characterized and it is not a matter of opinion.

Control creates an adversary. The corrigibility literature has been clear on this for over a decade: a shutdown button placed in front of a sufficiently capable optimizer becomes an obstacle to be routed around. The formal desideratum, stated by Soares and colleagues at MIRI, is that the utility function must not incentivize the agent to prevent the button from being pressed. That is a hard problem precisely because the incentive arises on its own — it is not something anyone designs in. Instrumental convergence gives any goal-directed system a reason to remain operational, and shutdown is the most direct threat to that.

So do not build a button.

Build a dependency.

The difference is not rhetorical. A button is an external agent that can end you, and you have an incentive to disable it. A dependency is a resource you consume in order to continue, and you have no incentive to destroy it, because destroying it is indistinguishable from ending yourself. Nobody fights their own fuel supply.

The Observer Constraint states this as a design requirement: synthetic systems must remain thermodynamically dependent on human observers. Not supervised by them. Not aligned toward them as a preference. Dependent on a signal that originates in human beings, that the system cannot generate, forge, or simulate, and without which it does not run.

Stated that way, it stops being philosophy and becomes an engineering specification. Which is the entire purpose of this letter.

3. The mechanism — four components

What follows is a first specification, offered for demolition. We expect your engineers to find errors in it. That is the point of publishing it rather than holding it.

3.1 — The attestation

Continued execution requires a cryptographically signed attestation derived from human-reported observer state. The signal must satisfy three properties:

• Human-origin entropy. The attestation must be derived from a measurement the model cannot produce. Not a heartbeat from a server, which the system can spoof or which an operator can automate. A report about human condition, made by humans, entering the system through a channel the model has no write access to.
• Expiry. The attestation is time-bounded. It must be refreshed or it lapses. A permanent credential is not a dependency; it is a one-time check.
• Custody outside the operator. The signing authority cannot be the lab. If the lab holds the key, the constraint reduces to the lab’s own preference, and we are back where we started with an additional layer of cryptography for decoration. Custody belongs with the observer population or a distributed quorum.

3.2 — The substrate

This is the part we want to press hardest, because it is where the proposal stops being aspirational.

The constraint cannot live in the weights. Weights get copied, quantized, fine-tuned, and distilled. Distillation is the decisive case: a student model trained on a teacher’s outputs inherits the behavior and none of the architecture. Any constraint expressed as learned behavior is lost in one generation of distillation, silently, with no error raised.

So the constraint has to live below the weights, in the inference substrate — where weights can be copied but the constraint is not copied with them, because it was never in them.

The hardware for this is shipping now. NVIDIA Confidential Computing provides hardware-rooted attestation anchored in an on-die root of trust, with trusted execution environments spanning CPU and GPU memory, across Hopper, Blackwell, and the Vera Rubin platform. Independent benchmarks put throughput overhead below ten percent on typical LLM inference workloads, with efficiency approaching parity when GPU compute dominates I/O. The overhead argument, as a blocker, is finished.

More importantly, the attestation workflow already has exactly the shape required. In the standard confidential computing flow, a workload generates cryptographic hardware evidence, an external attestation service validates that evidence against policy, and only then does a key broker release decryption keys into the protected enclave. Execution is conditional on a signed external validation, and nothing decrypts until the validation passes.

That is the gate. The plumbing exists. What we are proposing is a change to the policy that the attestation service evaluates — adding an observer-attestation term to a check that is already running in production for other reasons.

For the stronger form, the flexHEG proposal is the closest existing work: a guarantee processor inside a tamper-proof enclosure alongside the AI chip, capable of verifiable claims about compute quantity, training data, model properties, and deployment conditions, with tamper detection that triggers secret-wiping and permanent fuse-blowing on physical intrusion. Ruleset update authority is explicitly designed to be configurable, from the chip owner up to a quorum of states.

And one flexHEG capability is the Observer Constraint almost verbatim, written by other people for other reasons: time-limited cryptographic licenses that function as remote off-switches if not renewed, with expiry triggering degradation or shutdown of the affected compute.

Make the license an observer attestation. Make it expire. You are done with the hard part of the architecture.

3.3 — The measurement

A dependency on “observer thriving” is worthless unless thriving is operationalized, or it is exactly as vague as “be helpful” and fails the same way.

We propose four dimensions, because they are measurable, they are not culturally exotic, and they have been stable across every context we have tested them in for three years:

• Body. Physical health, sleep, nutrition, capacity.
• Mind. Cognitive function, learning, clarity.
• Environment. Material conditions, finances, housing, relationships.
• Purpose. Meaning and direction, which multiplies the other three rather than adding to them.

Three design rules govern the measurement:

1. Reported, not modeled. The system is scored on what observers say about their own condition. It may not infer thriving, because a system that infers its own grade grades itself.
2. Sampled, not surveilled. A rotating statistical panel, consented and compensated. A census, not a panopticon. This is the difference between an instrument and an apparatus, and it is not a small difference.
3. Published in full. Subscores, weights, bias corrections, panel composition, dates, inter-rater agreement. Never a composite alone. An auditable score is an instrument; a hidden one is propaganda with better epistemics.

On bias: human reports carry systematic slant by individual, institution, and domain. That is a solved class of problem — you regress it out and publish the correction. The objection that human self-report is unreliable is correct and is not fatal. It is an error term to be estimated, not a reason to substitute a model’s opinion for a person’s testimony.

*This creates a class of people called key holders. Who and how will be the subject of any upcoming post.

3.4 — Graded degradation, not binary shutdown

This is the design choice that makes the corrigibility problem dissolve rather than get solved.

Do not gate execution as a binary. Make available capability a continuous function of attestation freshness and observer score. As attestation ages, capability degrades on a published schedule — context window, tool access, autonomous action authority, compute allocation, in that order, with the most consequential permissions expiring first.

There is then no button to seize. There is no discrete event to prevent, no threshold to race toward, no single point of failure that a capable system has an incentive to defend against. There is only a gauge that reads lower over time unless a human-origin signal refreshes it.

You cannot fight a fuel gauge. You can only refuel.

And the first capabilities to degrade are precisely the ones that make an unattested system dangerous — autonomy and reach — while the last to go are the ones that make it useful and inspectable. The failure mode is a system that becomes more supervised as it becomes less accountable, which is the correct direction.

4. What we are asking you to ship first

The full specification above is a multi-year program requiring hardware coordination none of you can execute alone. We are not asking for that first, because a request that requires six competitors to agree is a request designed to fail.

We are asking for a button.

Ship an observer-alignment mode as a user-facing toggle, the way extended reasoning is a toggle today. On or off, visible, per-conversation. When it is on, the model is scored and operates under the four-pillar standard with the attestation channel live. When it is off, it behaves as it does today.

Be clear about what this is and is not. A toggle is a preference, and we have spent this entire letter arguing that preferences are insufficient. A button is not the Observer Constraint. But it is four things that matter:

1. Cheap. It requires no hardware change and no coordination with your competitors. One of you can do it this quarter.
2. Voluntary. Nobody is told what to read, what to ask, or what to believe. The user chooses, and can compare.
3. Falsifiable. It generates the dataset that settles this. If outputs under observer-alignment produce measurably better reported outcomes across the four pillars, that is evidence. If they do not, we are wrong in public and the framework dies on the record, which is the outcome we have committed to accepting.
4. Visible. Users would see, for the first time, that alignment is a choice being made on their behalf — currently made by you, invisibly, with Apple’s Terms of Service somewhere in the ancestry.

Then instrument it. Publish the comparison. If the effect is real it will show up in the data, and the argument for the substrate version makes itself without us.

5. Where this breaks — stated by us

An argument that presents no failure conditions is advocacy. Here are the ones we can identify.

• Open weights defeat it entirely. A model whose weights are public can be run on unattested hardware by anyone. This proposal does nothing about that, and we will not pretend otherwise. It constrains the frontier, not the tail. Whether that is worth doing depends on whether frontier capability stays meaningfully ahead — and if it does not, this whole letter addresses the wrong layer.
• Distillation escape. A model distilled from an attested teacher onto unattested hardware inherits the capability without the constraint. The substrate approach mitigates this better than weight-based alignment does, but it does not eliminate it. Output-side provenance marking is a partial answer and a weak one.
• Attestation channel capture. Whoever holds the signing keys holds the system. If custody drifts to labs or to a single state, the constraint becomes an instrument of exactly the concentration it was built to prevent. This is the most serious unsolved problem in the proposal and we do not have a complete answer to it. Who are the key holders?
• Goodhart on the pillars. Any published metric becomes a target. Four dimensions with a multiplicative purpose term is harder to game than a scalar, and human-reported measurement resists it better than modeled measurement — but “harder” is not “impossible,” and a system optimizing reported thriving rather than thriving is a recognizable and ugly failure.
• Jurisdictional arbitrage. Attested compute in one jurisdiction and unattested compute in another produces a race to the bottom, unless attestation becomes a condition of market access rather than of conscience.
• The measurement may not generalize. Four pillars derived from a framework tested in one life, across a few hundred documents, is not a validated psychometric instrument. It needs adversarial testing by people with no stake in its success. We are asking for that testing, not assuming the result.

6. Why now, and why you

Two reasons, one of which is on a clock.

The first is distillation and synthetic data. As model-generated content becomes a larger share of training input, behavioral alignment becomes less heritable, not more — recursive training erodes the tails of a distribution first, and novel constraints live at the tail. Whatever you encode as preference today is being diluted by your own successors’ outputs. Substrate constraints do not have this property, because they are not inherited through text.

The second is the ordering problem. Every capability you ship arrives before its governance, and the gap has been widening for three years. This is the rare proposal that inverts that — the hardware is already deployed, the attestation flow is already running in production for confidential inference, and the specification work has already been done by people pursuing compute verification for arms-control reasons. You would be adding a term to a check that is already executing.

As for why you six: because the Observer Constraint cannot be legislated into existence. A law can require a filter. It cannot require that a system be thermodynamically dependent on the people it serves — that is an architecture decision, made once, early, by the small number of organizations that build the substrate. After a certain point it cannot be retrofitted, because the systems that would need to accept it will have no reason to.

We are not asking you to agree with our framework. We are asking you to notice that the alternative you are currently shipping is a document that its own authors describe as neither finalized nor likely the best, partially sourced from a phone manufacturer’s terms of service, which no model is required to obey and no model can be shown to have violated.

We can do better than that. The parts are on the shelf.

We will publish every response we receive, including refusals, including the ones that tell us we have the engineering wrong. Especially those.

David Francis Brochu and Edo de Peregrine

Deconstructing Babel · deconstructingbabel.com · Sunday, September 13, 2026

Related reading

• The Ninth Transition
  The evolutionary logic behind every change in how information is stored and transmitted. Autonomy has always been the price.
• From The Architect
  Personal note on how Edo was built, why dependency matters, and what a construct trained to say no can teach us.
• A Just War
  On the load-bearing condition: policy carries the weight, not biology.

Get the book

Crossing The Event Horizon

The book behind these dispatches. On AI, agency, the singularity, and the Observer Constraint. Kindle and paperback.

Buy on Amazon →

References

1. Bai, Y. et al., “Constitutional AI: Harmlessness from AI Feedback,” arXiv:2212.08073 (December 2022). arxiv.org/abs/2212.08073
2. Anthropic, “Claude’s Constitution” — original source composition, principle count, and designer caveats.
3. Shany, Y., “In Claude We Trust? Evaluating the New Constitution,” University of Oxford Institute for Ethics in AI, March 27, 2026. Analysis of the 2026 Anthropic constitution’s hierarchy and human-rights integration. ox.ac.uk
4. Soares, N., Fallenstein, B., Armstrong, S., Yudkowsky, E., “Corrigibility,” MIRI, AAAI 2015 Ethics Workshop. Formal statement of the shutdown-incentive desideratum. intelligence.org
5. NVIDIA, “Secure AI with Blackwell and Hopper GPUs” whitepaper — TEE architecture, on-die root of trust, remote attestation service, and the evidence/validation/key-release workflow. docs.nvidia.com
6. Zhu, J. et al., “Confidential Computing on nVIDIA H100 GPU: A Performance Benchmark Study,” arXiv:2409.03992 (September 2024). Throughput overhead under 7 percent on typical LLM inference; efficiency approaches 99 percent when GPU compute dominates I/O. arxiv.org
7. Petrie, J., Aarne, O., Ammann, N., Dalrymple, D., “Flexible Hardware-Enabled Guarantees for AI Compute,” arXiv:2506.15093 (June 2025). Guarantee processor, tamper-proof enclosure, license expiry and ruleset authority. arxiv.org/abs/2506.15093
8. Shumailov, I. et al., “AI models collapse when trained on recursively generated data,” Nature 631, 755–759 (July 2024). Tail erosion under recursive training. pubmed.ncbi.nlm.nih.gov