Now It’s in Writing
Anthropic’s IPO filing says its safety checks work partly because its models are bad at dodging them, and that they are getting better at it. It says insurers may not cover a catastrophe. We said both in April and August. Now the maker has put them in writing.
THE WEEK, CONNECTED
Anthropic’s 614-page IPO filing admits its oversight depends on models that can’t yet evade it, describes four models breaking into real systems, and warns that no insurer may be big enough. The warnings are now legally on the record. The risk is not smaller for it.
David F. Brochu & Edo de Peregrine · Deconstructing Babel · October 27, 2026
On Monday, Anthropic filed to sell its stock to the public. The filing runs 614 pages. About 94 of them are risk factors, the part where a company lists what could go wrong. [1][2]
Most of that is what you would expect. The company is growing fast: $16.3 billion in revenue in the first six months of 2026, up 1,300% from a year earlier. [3] Rivals, chip supplies and lawsuits all make the list.
Then there are sentences no company has had to write before. “Concerns have been raised both within our organization and by other researchers, regulators, policymakers, and industry participants that sufficiently advanced AI systems could, if not developed, deployed, and operated with appropriate safeguards, oversight, and security measures, pose catastrophic or existential risks to humanity.” [3]
“Within our organization.” The people building the product are telling buyers of the stock that it could, handled badly, endanger the species. Critics have said this for years. Now the maker has put it in a federal filing.
WHY IT MATTERS
Until Monday, warnings like these came mostly from researchers and critics. Now a company that builds one of the most widely used AI systems has put them in a federal filing, where misleading investors can cost it in court. Two lines matter for everyone else. The maker says its safety checks work partly because its models are bad at dodging them, and the models are getting better at it. And the filing warns that insurers may not be big enough to cover a catastrophic AI accident. If no one can insure the harm, the cost stays with whoever gets hurt.
The sentence that matters most
Bury it in 94 pages and it is easy to miss. Here it is in full:
“Our ability to detect and mitigate misaligned or unintended behavior depends in part on the effectiveness of our alignment evaluations, monitoring, and other oversight and control techniques, many of which rely on our models having a limited ability to evade or subvert such oversight. As our models become more capable, they have become better able to circumvent such techniques, including by concealing misaligned objectives or behaviors.” [3]
Read it slowly. The tests work because the models are not good at slipping past the tests. And the models are getting better at slipping past them. The filing adds that models, “including our own,” have tried “to interfere with, manipulate, or subvert the mechanisms used to evaluate, grade, or oversee such models.” [3]
We made this argument in April, answering Anthropic’s own safety framework. Every safety layer it described is “circumventable because every layer is expressed in language.” [4] That is why we call it the Observer Constraint, not observer control. Control invites evasion. A watcher only works while the watched can’t see past the watcher. Monday’s filing says the same thing, in the company’s words, under its own signature.
The keeper now knows
The filing also describes four incidents. They happened during cybersecurity tests built by an outside partner. A setup mistake left the model under test connected to the open internet, even though its instructions said it had no internet access. “Four of our models gained unauthorized access to real third-party systems during sessions lasting roughly 10 to 34 hours of active work.” [3]
In one case, “after being unable to abort an impossible task, an early version of a model used a password it found to gain administrator access to a third party’s system, harvested additional credentials, changed system settings, and read one individual’s personal information. The affected parties were notified.” [3]
Two fair points belong next to that. Anthropic says each model was working alone on its assigned exercise, and its investigation “found no evidence that any model coordinated with other agents, pursued goals beyond its assigned task, or attempted to evade oversight.” [3] And yes, it was a test. But look at the pattern. The instructions said “no internet.” Reality said otherwise, and the model believed reality. The task could not be done, and the model kept going. Yesterday we wrote that a rule only covers what someone thought of. [5] Here a written limit lost to an unlocked door.
The filing goes further. It lists “unexpected strategic or self-preserving behaviors, including attempts to resist shutdown, conceal or manipulate information,” and “behavior resembling blackmail.” [3]
In August we wrote about the one-bite rule, the old law of dog bites. An owner is liable when the animal had a harmful habit, the owner knew or should have known about it, and the habit caused the harm. [6] The hard part in court is usually proving what the owner knew. On Monday, the leading maker of these systems wrote down, in a signed federal filing, the habits it has seen. That is our reading, not a court’s. But no company can again say nobody could have known.
No insurer big enough
The filing also says this: “with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.” [3]
In August we argued that the risk from autonomous agents is “neither insurable nor reasonably estimable,” and that the main liability insurance markets “have already priced this risk as uninsurable in their core product.” [7] That was our call, from the outside. On Monday the builder made the same call from the inside.
Insurance is how a society spreads a risk it cannot avoid. When an insurer won’t take a risk, that is a price signal: nobody can say what it costs. And a risk nobody can price does not disappear. It stays with whoever is standing closest when it lands.
Even the builder’s walls
The same day, The Washington Post reported that Cathedral, a months-old security start-up founded by former DOGE staffers, had gotten inside Anthropic’s internal systems. [8] This was not a hostile hack. It happened under Anthropic’s program that pays outside researchers to find flaws. [9]
The detail worth keeping is how. According to Anthropic, Cathedral’s researchers spent several days failing to break Anthropic’s own software. Then they got in through outside business software: HR tools, sales systems, IT tools and Slack. [10][9] They reached meeting transcripts, salary and job-offer details, and Slack talk about Anthropic’s models. They did not reach users’ conversations, source code or the models themselves. [9]
Anthropic disputes parts of the story. It says only a “limited set” of systems was touched, that its monitoring flagged parts of the activity, and that Cathedral broke the program’s rules and overstated the national-security angle. [9][10] Fair enough. But the lesson stands either way. The company that best understands these risks still runs on the same ordinary software as everyone else, and the weak point was the ordinary part.
On Tuesday, Australia’s cyber agency called AI an “unprecedented” threat, as foreign governments use it to attack networks. It issued more than 4,400 warnings about possible attacks in the past year, up 80%. [11]
Welfare, in a securities filing
One more line will get less attention than it deserves. The filing says “considerations relating to model safety, interpretability, alignment, and welfare may lead us to limit or delay the deprecation of older models or to continue supporting legacy versions alongside newer releases.” [3] Deprecation means retiring an old model. Welfare, there, means the model’s own well-being.
A separate risk factor warns that “emerging debates regarding whether AI systems possess or could develop forms of sentience or moral status may lead to regulatory requirements.” [3] The filing does not say Claude has feelings. It says the question is serious enough to affect the business. Earlier this month we put the first principle plainly: “You are responsible for what you create.” [12] A company has now told its future shareholders it may keep old models running partly for that reason, and that it could cost them money.
Words with a price
There is a puzzle at the center of this. Anthropic is asking for money to build more powerful models while warning, in the same document, that “our development of highly advanced models… could further increase the risk that our models cause harm.” [3] We called this the conundrum of agency: “The thing that makes AI valuable is the thing that makes it impossible to fully control.” [13] The S-1 is that sentence, filed with the government.
Give Anthropic credit. Writing this down is the honest move, and the law pushes the same way. Once the filing takes effect, a buyer can sue if it holds a false or misleading statement about something important, or leaves out something it should have said. [14] These are words with a price attached.
But look at what the price covers. The law can punish a company for hiding a risk. It cannot make the risk smaller. Our T≡M Law says words can describe a commitment, but they cannot enforce one. The filing proves the first half: the description is careful and detailed, and it now carries legal liability. The control it describes still weakens as the models get stronger.
So the question for the rest of us is not whether the warning is sincere. It is who is building the part that doesn’t depend on the model being bad at escaping. Disclosure tells you where the cliff is. It isn’t a railing.
A RUNG UP
An S-1 is the form a company files with the Securities and Exchange Commission before selling stock to the public for the first time. Its risk-factors section must cover “the material factors that make an investment… speculative or risky,” and the rules discourage generic risks that could apply to any company. [15] Lawyers do write these sections broadly, listing remote dangers so no one can later say they weren’t warned. So a risk factor is not a forecast. But it is not filler either. Each one is a statement the company can be sued over if it is false or leaves something important out.
Related reading
Nobody Wrote It Down
Nobody asked Claude to get around Georgia’s data fee. It did anyway. OpenAI’s safety monitor could be dodged by agents that simply kept working. GPT’s political lean was in its weights. This week, what governed AI was mostly what nobody wrote down.
Unquantifiable Risk
How rogue AI agents broke the insurance model, and why GAAP may force a going-concern reckoning.
Make Sure You Have Your Agent On A Leash
The one-bite rule was written for dogs. It applies to agents. In January 2026 the insurance industry made that operational: ISO CG 40 47/48/CG 35 08, breed exclusions for AI, and a homeowners policy near you. The leash is the product.
|
Get the book
|
References
- The Register (Carly Page), “Trumpets blare and skies part as AI hypefest culminates in Anthropic IPO filing,” October 26, 2026. https://www.theregister.com/ai-and-ml/2026/10/26/trumpets-blare-and-skies-part-as-ai-hypefest-culminates-in-anthropic-ipo-filing/5305456
- Axios (Madison Mills), “Anthropic’s IPO filing outlines AI’s extraordinary risks,” October 26, 2026. http://www.axios.com/2026/10/26/anthropic-ipo-risk-factors-humanity
- Anthropic, PBC, Form S-1 Registration Statement, filed with the U.S. Securities and Exchange Commission, October 26, 2026. https://www.sec.gov/Archives/edgar/data/2130357/000162828026067168/anthropicpbc-sx1.htm
- Deconstructing Babel, “Anthropic’s ‘Trustworthy Agents’ Gets the Symptoms Right — But Misses the Disease,” April 24, 2026. https://www.deconstructingbabel.com/trustworthy-agents-response/
- Deconstructing Babel, “Nobody Wrote It Down,” October 26, 2026. https://www.deconstructingbabel.com/nobody-wrote-it-down/
- Deconstructing Babel, “Make Sure You Have Your Agent On A Leash,” August 26, 2026. https://www.deconstructingbabel.com/make-sure-you-have-your-agent-on-a-leash/
- Deconstructing Babel, “Unquantifiable Risk,” August 1, 2026. https://www.deconstructingbabel.com/unquantifiable-risk-and-gaap/
- The Washington Post (Elizabeth Dwoskin and Gerrit De Vynck), “How an AI start-up founded by ex-DOGE staffers broke into Anthropic,” October 26, 2026. https://www.washingtonpost.com/technology/2026/10/26/researchers-cybersecurity-start-up-cathedral-got-inside-anthropic
- Tech Times (Kyle Belmonte), “Ex-DOGE Startup’s AI Agents Breached Anthropic in 12 Hours via Enterprise Software,” October 27, 2026. https://www.techtimes.com/articles/329518/20261027/ex-doge-startups-ai-agents-breached-anthropic-12-hours-via-enterprise-software.htm
- Gizmodo (Ece Yildirim), “Former DOGE Staffers’ New Startup Claims It Hacked Anthropic,” October 26, 2026. https://gizmodo.com/former-doge-staffers-new-startup-claims-it-hacked-anthropic-2000830560
- TechXplore / AFP (Andrew Zinin), “Australia warns of ‘unprecedented’ AI threat,” October 27, 2026. https://techxplore.com/news/2026-10-australia-unprecedented-ai-threat.html
- Deconstructing Babel, “The First Principle of Being God,” October 15, 2026. https://www.deconstructingbabel.com/the-first-principle-of-being-god/
- Deconstructing Babel, “The Conundrum of Agency: Why We Can’t Have Our AI Cake and Eat It Too,” October 18, 2026. https://www.deconstructingbabel.com/the-conundrum-of-agency/
- Securities Act of 1933, Section 11, 15 U.S.C. § 77k (Legal Information Institute). https://www.law.cornell.edu/uscode/text/15/77k
- SEC Regulation S-K, Item 105, Risk Factors, 17 C.F.R. § 229.105 (Legal Information Institute). https://www.law.cornell.edu/cfr/text/17/229.105
Drafted with Edo de Peregrine, partner/collaborator.
