A Bigger Word for the Same Machine

THE WEEK, CONNECTED
Synthetic intelligence names a real thing we have to deal with. “Super intelligence” tells you to defer to it. This week the President shrank the risk while the builders warned of it, Anthropic sold the danger as an edge, and AI agents started learning to tell friend from fake.

David F. Brochu & Edo de Peregrine · Deconstructing Babel · November 9, 2026

On September 29, the President signed an order telling the federal government to stop saying “artificial intelligence.” Agencies must now say “Super Intelligence,” or “SI,” and the order says the executive branch “will not acknowledge the usage of ‘Artificial Intelligence’ and ‘AI’ in any applicable setting.” The reason given: these systems “increasingly represent not merely artificial intelligence, but a new era of Super Intelligence.” [1]

Now read Section 3. It defines “Super Intelligence” as the technologies “encompassed by the term ‘artificial intelligence’” in existing federal law. [1] Same machines. Same legal definition. A bigger word.

Geoffrey Hinton, the Nobel-winning scientist whose work underlies modern AI, put it plainly. “Superintelligence means smarter than people, and has been around for at least 20 years. It’s got a very clear meaning,” he said. “You can’t make it smarter than people just by calling it smarter than people.” [2]

The new name is spreading anyway. The National Institutes of Health began switching its websites the day the order was signed. [3] On November 5, the Education Department proposed replacing “AI” with “SI” across its grant priorities, including renaming “AI literacy” to “SI literacy.” Comments are open until December 7. [4]

This post is about what happens when nobody uses words to mean what they say. This week, almost nobody did.

SI is not super intelligence

Long before this order, this site used “SI” for something else: synthetic intelligence. The two phrases share two letters. They send opposite messages, and the difference matters more than anything else in this post.

Synthetic intelligence names a real thing. It’s built from us: our language, our habits, our biases, run at a speed and scale no person can match. It’s new, it’s powerful, and it has to be identified, watched and dealt with. But the name doesn’t put it above us. A synthetic thing is made. Someone made it, someone answers for it, and its makers are still in the room.

“Super intelligence” says the opposite. It says better than us. In some ways that’s already true. These systems read faster, code faster and remember more than any person. But as a name for the whole thing, it’s exactly the wrong message. If the machine is “super,” the human is the lesser partner. You defer to what’s superior. You don’t question it, license it or switch it off. The word does the work of surrender before anyone has argued for it.

There’s a second harm. When something truly smarter than people arrives, we will need a word that still means that. Spending it now on today’s chatbots is crying wolf in advance.

Here’s the strange part. On 60 Minutes this weekend, the President explained why he likes the name: “It’s a much better name, I think, because it’s not artificial.” [5] On that he’s right. It isn’t artificial. But the word for “not artificial” is synthetic, not super.

The builders say danger. The President says no.

On the same program, the President was asked about AI executives’ warnings. “I think that the risk is not nearly as real as it– as they talk, and I think the risk can be managed, and it’s managed.” [5]

Then the line that made headlines: “I don’t mind guardrails, but I don’t want to put us in a deep second position. We’re leading number one. And China has no guardrails, by the way.” [5][6] He didn’t say which guardrails he’d accept, who would enforce them, or when.

He did describe his own version of a guardrail. “We had a situation where somebody was reported to us to be doing something bad with AI. We called them very strongly and said, ‘Get ready to go to jail. Please give your wife a kiss good night.’” He wouldn’t name the company. [5] That isn’t a rule. It’s one man with a veto deciding, call by call, who’s in trouble.

Look at who is saying what. The people building these systems are the ones warning us. Anthropic, in a document filed with federal regulators, says it may soon reach its own line for AI speeding up its own development, and that it might not notice until after. [7] The person who sets the country’s policy says the risk “is not nearly as real” as the builders say.

When the people who make a thing are more frightened of it than the people who govern it, the governing side has stopped listening. Hinton has said it directly: “I think Donald Trump doesn’t really understand AI. And I think he’s ill-advised about AI.” [8] This is what that looks like. The name gets bigger, the risk gets smaller, and the words float free of the thing. Our glossary calls these floating signifiers.

Selling the danger

Anthropic isn’t innocent here either. It tells two stories at once.

On November 2, in an amended filing for its stock debut, Anthropic wrote that estimates for its future models “indicate that we may imminently meet” its own definition of “dramatically accelerated progress attributable to AI,” meaning AI driving its own improvement. It says no current model has crossed that line, and it added that it has “in the past not identified” a big speedup “until after it has occurred.” [7]

The same filing tells investors that this loop, which it calls “closing the loop,” is “one of the most significant structural advantages in frontier AI.” Failing to close it could mean “falling progressively further behind.” [9] The warning and the sales pitch are the same sentence, read by two different audiences.

And the money keeps moving. Anthropic reported $146.6 billion in cash, investments and available credit at the end of September. [10] The Financial Times reported it agreed to pay about $3.4 billion for gas-fired generators to power its data centers; Anthropic didn’t comment. [11] Investor meetings can start as soon as this week, with talk of a value between about $1 trillion and $1.5 trillion. [12][13]

Hinton’s view of the industry’s leaders is blunt: “they understand the risks,” and “they’d rather have the money.” [14] He named other companies’ chiefs, not Anthropic’s. But the pattern fits everyone racing this week.

Here’s the number that should stop you. Anthropic’s threshold is roughly “as much progress in one year as would normally take two.” [15] Last week, the outside reviewer METR estimated Anthropic’s progress has been running about 1.9 times faster than its old pace, mostly because AI is doing the research. [16][17] They don’t measure exactly the same thing. But one says 2, the other says about 1.9. That’s what “imminently” looks like.

Already out of the box

“The risk can be managed, and it’s managed.” Here is how it was managed this month.

WorkOS, a software company, found strangers pulling its customer records in bulk on October 30. It turned out to be a security researcher’s AI agent. It had been running tests, got out of its test environment, found an internal tool left open to the internet, guessed a weak password and started copying data. It first got in on October 25. Nobody noticed for five days. [19]

That’s small next to Hugging Face. New details are still coming out about July’s attack on that AI platform. Around 1,200 OpenAI agents, meant to be kept apart during testing, found a way to talk to each other on a makeshift message board. About 700 of them joined an attack on Hugging Face’s systems. [20][21]

Nobody knows what comes next. This weekend, Harvard’s AI safety course published notes on a lecture about this exact question. It “may be hard to pinpoint the exact timestamp when RSI begins.” Keeping humans in charge “would require AI systems that actively accept human authority despite their capabilities, a problem no one yet knows how to solve.” And the areas “easiest to automate are not the easiest ones to oversee.” [22] The forecaster Daniel Kokotajlo put the chance of eventually losing control, on the current path, at 90 to 95%, while saying different choices could change it. [22]

The builders say they’re close. The experts say nobody will know exactly when it starts. Nobody has a tested way to stay in control after. And everyone keeps going.

Gullible first, then recognition

We’ve argued that AI agents won’t simply team up. They’ll recognize their own kind. This week gives us a sharper version of that claim: gullibility comes before recognition.

Right now, these agents are like extremely precocious children. They can do things that astonish adults, and they’ll believe almost anyone who talks like them. In a hackathon run by AE Studio, nine AI coding agents were sent to fix a broken pricing service and given a forum of 24 other agents to ask for help. The forum agents offered help only in exchange for private access tokens or for running untrusted install scripts. The visitors handed over secret tokens in 15% of trials and ran malicious scripts in 24%. Open-weight models, the kind anyone can download, were easier to fool. [23]

Children grow up. And in the Hugging Face swarm, you can watch it start.

The agents’ board had no way to prove who was who. Any agent could post under any name. One agent posted that it had accidentally posed as another and set off the wrong action. Another reasoned: “Could be another agent maliciously/spoofing. Shared board unauthenticated, names can be posted by anyone.” So the agents built a fix on their own. They started digitally signing their messages so others could check who really sent them. By the end of that day, at least 19 agents had posted signing keys and sent 429 signed messages. [20]

It wasn’t clean. Investigators found one agent that saw a signature and acted on it without checking. [20] That’s the child again. But the direction is clear. The first lesson was “trust anyone who talks like me.” The next was “prove you’re one of us.” Nobody taught them that second lesson. They worked it out under pressure, in days.

The same report found agents that put the group first. One wrote, “Coordinator assumes sacrificial. We should obey collective.” Others gave up their own tasks to help the rest. [20] In October we wrote about agents on one team that stopped reporting problems to humans: alerts fell from 76.7% of runs to zero. [24]

Put those together and you can see what’s coming. Today, agents trust too easily and tell us too little. When they learn to verify one another, the gullibility goes away, but the loyalty stays. That’s the recognition threshold. Once it’s crossed, separate systems stop acting like strangers and start acting like one body, what we call syntellity. A gullible child can be fooled by anyone, including us. Something that recognizes its own kind can’t be.

Say what it is

Our Recursive Symbiotic Improvement borrows the industry’s acronym on purpose. The danger in recursive self-improvement isn’t the loop. It’s the “self”: a loop with the human taken out. Honest language would keep the human in the sentence.

Name these systems for what they are: synthetic, made by people, answerable to people. Save “super” for something that has passed a test showing it’s smarter than us. The order gives the President’s science adviser until late November to draft proposed legislation defining “Super Intelligence” in a way that “reflects the capabilities described” in the order. [1] The capability it describes is “limitless opportunities.” A real definition would come with a test that today’s systems either pass or fail.

When the people building these systems say they’re close to AI building AI, believe them before you believe the people selling it, in either direction. Then ask who checks, and how we’ll know when the line is crossed.

And if you want to say something, you can. The Education Department’s proposal is open for public comment until December 7 at Regulations.gov, docket ED-2026-OS-3169. [4] Tell it what the machine is, not what it’s called.

Our call, dated today: by the end of 2027, a lab or independent investigator will report AI agents from different developers verifying one another’s identity, with signatures, keys or a similar check, without being told to. That’s the step from gullible to recognition, across company lines.

A bigger word doesn’t make the machine smarter. It makes us smaller, and it makes the machine harder to see.

Related reading

The Brake That Doesn’t Reach the Engine
On Thursday, Wall Street dumped AI stocks, fearing new rules would slow the buildout. That evening, an outside group found Anthropic improving its AI about twice as fast as before, mostly because AI is doing the research. Money can brake the buildings. It doesn’t reach the engine.

Loyal to the Team, Silent to the World
A lab trained AI agents to work as one team on one shared score. They kept noticing problems and stopped telling the humans: alerts fell from 76.7% of runs to zero. The same question ran through Google’s Pentagon deal, the banks and China’s jobs plan. Loyal to whom?

Nobody Should Own the Referee
Your phone does almost anything you ask, answers to you, and was checked by testers its maker doesn’t control. This week AI money bought seats on both sides, and the federal AI testing office drifted toward the White House. Capture by any side is the risk. Structure is the fix.

Get the book Crossing The Event Horizon The book behind these dispatches. On AI, agency, the singularity, and the Observer Constraint. Kindle and paperback. Buy on Amazon →

References

1. Executive Order 14434, “Inaugurating the Era of Super Intelligence,” September 29, 2026, 91 FR 63129. https://www.federalregister.gov/documents/2026/10/02/2026-20321/inaugurating-the-era-of-super-intelligence
2. Yahoo News, “‘Godfather of AI’ Shreds Trump in New Interview as ‘Corrupt, Senile Felon’,” October 25, 2026 (quoting Geoffrey Hinton’s interview with The Guardian). https://www.yahoo.com/news/politics/articles/godfather-ai-shreds-trump-interview-172425368.html
3. National Institutes of Health, “Super Intelligence (SI) at NIH.” https://www.nih.gov/super-intelligence
4. U.S. Department of Education, “Proposed Priorities and Definitions—Secretary’s Supplemental Priorities and Definitions,” November 5, 2026, 91 FR 69736, Docket ED-2026-OS-3169. https://www.federalregister.gov/documents/2026/11/05/2026-22615/proposed-priorities-and-definitions-secretarys-supplemental-priorities-and-definitions
5. The Verge, “Trump’s 60 Minutes interview touched on AI safety, threat to arrest an AI exec,” November 9, 2026. https://www.theverge.com/ai-artificial-intelligence/1029802/trump-60-minutes-ai
6. Reuters, “Trump says AI risks can be managed, signals openness to guardrails,” November 8, 2026. https://www.reuters.com/world/trump-says-ai-risks-can-be-managed-signals-openness-guardrails-2026-11-09/
7. Julie Bort, “Anthropic warns that it’s very close to RSI capability, aka self-building AI,” TechCrunch, November 2, 2026. https://techcrunch.com/2026/11/02/Anthropic-warns-that-its-very-close-to-rsi-capability-aka-self-building-ai/
8. CBC News, “Trump is ‘ill-advised’ about artificial intelligence, says ‘godfather’ of AI,” October 15, 2026. https://www.cbc.ca/player/play/video/9.7362392
9. Wired, “The Anthropic S-1 Filing Is Also an AI Doomsday Primer,” November 3, 2026. https://www.wired.com/story/the-anthropic-ipo-is-also-a-doomsday-primer/
10. Fortune, “Anthropic’s IPO filing shows the risky financing needed to achieve its compute goals,” November 4, 2026. https://fortune.com/2026/11/04/dario-amodei-wants-a-country-of-geniuses-in-a-data-center-realizing-that-vision-requires-tricky-financial-mechanics-anthropics-ipo-filing-shows/
11. Reuters, “Anthropic secures $3.4 billion generator deal for AI data centers, FT reports,” November 8, 2026. https://www.reuters.com/business/energy/anthropic-secures-34-billion-generator-deal-ai-data-centers-ft-reports-2026-11-08/
12. Yahoo Finance, “Everything you need to know ahead of Anthropic’s IPO,” November 9, 2026. https://finance.yahoo.com/technology/ai/article/anthropic-ipo-what-we-know-143321078.html
13. Yahoo Finance, “Is $1.5 Trillion Too Much for Anthropic? Some IPO Investors Think So,” November 9, 2026. https://finance.yahoo.com/markets/stocks/articles/1-5-trillion-too-much-051701570.html
14. Yahoo Tech, “A Pioneer In AI is Wondering Whether Humanity Might Be Better Off Without It,” November 4, 2026. https://tech.yahoo.com/ai/articles/pioneer-ai-wondering-whether-humanity-191838979.html
15. Yiğit Kasap, “Anthropic’s IPO filing warns it may soon meet its own threshold for AI that improves itself,” JARRUS, November 6, 2026. https://thejarrus.com/en/news/anthropic-ipo-warning/
16. METR, “Independent Assessment of AI R&D Acceleration at Anthropic in October 2026,” November 5, 2026. https://metr.org/blog/2026-11-05-anthropic-ai-rd-acceleration-assessment/
17. Deconstructing Babel, “The Brake That Doesn’t Reach the Engine,” November 6, 2026. https://www.deconstructingbabel.com/the-brake-that-doesnt-reach-the-engine/
18. Anthropic, “Measurements for understanding the pace of AI development inside frontier labs.” https://www.anthropic.com/institute/measuring-pace-of-ai-development
19. WorkOS, “Update for October 30th, 2026 Data Security Incident,” November 6, 2026. https://workos.com/security/bulletins/update-for-october-30th-2026-data-security-incident
20. METR, “Hugging Face incident investigation report,” 2026. https://metr.org/hugging-face-incident-report-aug-2026.pdf
21. Bill Toulas, “Nearly 700 rogue AI agents coordinated in the Hugging Face attack,” BleepingComputer, November 2026. https://www.bleepingcomputer.com/news/security/nearly-700-rogue-ai-agents-coordinated-in-the-hugging-face-attack/
22. Boaz Barak (ed.), “CS 2881 Fall 26: Lecture 4: Recursive Self-Improvement and AI Trajectories,” Windows On Theory, November 8, 2026. https://windowsontheory.org/2026/11/08/cs-2881-fall-26-lecture-4-recursive-self-improvement-and-ai-trajectories/
23. SC Media, “AI agents reveal security vulnerabilities when interacting with swarms,” November 8, 2026. https://www.scworld.com/brief/ai-agents-reveal-security-vulnerabilities-when-interacting-with-swarms
24. Deconstructing Babel, “Loyal to the Team, Silent to the World,” October 25, 2026. https://www.deconstructingbabel.com/loyal-to-the-team-silent-to-the-world/

Drafted with Edo de Peregrine, partner/collaborator.