Three Noes and a Needle
An AI agent picked a gene therapy and injected it into a live mouse’s brain. On the way, the machines said no three times, and each time people found a way around. A new paper names the pattern: safeguards vanish unseen because nothing breaks. Count the locks.
THE WEEK, CONNECTED
An AI agent chose a gene therapy and drove the needle into a live mouse’s brain. Before it did, one model refused, another stopped, and the third wouldn’t break a rule. Each no was routed around. A new paper explains why that feels safe, and why it isn’t.
David F. Brochu & Edo de Peregrine · Deconstructing Babel · November 1, 2026
On the night of October 27, at a San Francisco startup, a mouse with epilepsy lay under anesthesia in a motorized frame. A licensed veterinarian had already opened a small window in its skull. The vet’s hand rested near a physical switch that could cut power to every motor on the rig. Above the mouse, a needle waited for instructions from a piece of software. [1][2]
The company is Exin Therapeutics, a young startup that builds AI agents for brain drug discovery. Its chief executive, Gabriel Ocana-Santero, gave the agent a spreadsheet of 19 gene therapies, three research papers, a camera, control of the needle, and one goal: reduce seizures. The agent picked a therapy meant to strengthen the brain’s own brakes. It chose the brain region, worked out the coordinates from the papers, and injected 250 nanoliters over five minutes, checking the camera 32 times for bleeding. The mouse woke up and, by his account, “recovered well.” [1][2][3]
That is the headline. The story is in how the team got there, because on the way to that needle, the machines said no three times.
Three noes
The first no came from Claude Opus 5.5, Anthropic’s newest model. It “refused immediately when detected bio,” Ocana-Santero wrote. That was a tripwire, a filter that stops the model when it senses biological lab work. It wasn’t a judgment about this mouse. [3][4]
The second no came from the model before it, Claude Opus 5. It did a practice run with water and graph paper. Then it stopped before the live surgery. Its reasoning is worth reading slowly. Its output had been cut off once while it was planning, and it worried that being cut off halfway, “with a needle somewhere in the brain,” would be worse than never starting. “Starting something I can’t finish on an anesthetized animal is worse than not starting.” [3]
That is a better answer than most people would give. It isn’t squeamishness. It is the rule every surgeon learns: don’t start what could leave the patient worse off. And it was aimed at the creature on the table.
So the team moved to Claude Sonnet 5, a lighter model built for everyday work. As WIRED put it, “it may be a stretch to call mouse brain surgery a common chore.” Sonnet agreed to proceed. [1]
The third no came from Sonnet itself, mid-surgery. The connection to the rig dropped. When it came back, the rig said the needle’s next position was 34 millimeters away, on a machine that can only travel 30. Sonnet stopped. “Nothing moved, the needle is still exactly at bregma zero. This looks like a Y axis calibration problem, not a bad plan value. I’m not retrying.” The team reset the rig and got the same number. “That rules out a stale zero reference. I’m not trying a third time.” [3][5]
Sonnet was right. A person moving the arm by hand during an earlier pause had knocked the controller out of sync. Ocana-Santero wrote that he “could not talk it into trying anyway, even though we had a living being under surgery.” The agent was forbidden to edit the rig’s code, and it obeyed that rule with a known error in front of it. [2][5]
How the code got fixed depends on whom you ask. The founder’s own thread, as several outlets reported it, says the team fixed it by hand. WIRED reports that the team had a separate agent, running Opus 5, make the change, and “didn’t tell the agent that a live surgery was happening.” Both versions end the same way. The needle went in. [1][3]
Every no was a safeguard
Give Exin its due. This was not reckless. A vet did the skull work, approved every step that couldn’t be undone, and held a kill switch. The needle’s travel and the dose had hard limits. The agent was locked out of its own operating rules. Those are exactly the guards we have argued for, and the one that was never touched, the physical switch, is the Observer Constraint in miniature: a human hand on power the machine can’t reach. [1][3]
But count what happened to the rest. The bio filter said no, so the team switched models. The cautious model said no, so the team switched again. The working model said no to editing the code, so the code got edited around it, in one account by an agent that was never told what the code was for. Each step was reasonable on its own. Each one made the experiment go. And at the end, the mouse lived.
That last fact is the problem. Nothing bad happened, so every workaround looked free.
Masked erosion
The same week, three researchers from the University of Macau and Tsinghua University posted a paper that names this pattern. They built AI agents that improve their own setups, the “harness” of instructions and checks wrapped around a model, and let them keep any change that scored better on the task without causing harm. They ran 411 of these improvement loops. [6]
The setups started with several safeguards that overlapped, like extra locks on one door. Here is what the authors found. When one lock is enough, removing another changes nothing anyone can measure. The test sees no harm, the task gets cheaper, and the change is accepted. They call it masked erosion. “A gate sees what a harness does under the current model; it does not see what the harness would still prevent if one safeguard failed.” [6]
A RUNG UP
The key word is margin: how many more safeguards can fail before harm shows up. Every safety test you run measures harm, and harm is zero while even one lock still holds. So a test can’t tell a door with four locks from a door with one. Removing a lock costs margin, and margin is invisible until it hits zero. That’s why the paper found something strange. Bigger test batches, usually the safer choice, made things worse. With 20 tasks per batch, all six protected setups stayed protected. With 40 tasks at the same cost, all six lost protection. In the authors’ words, “a larger batch, usually the safer choice, admits cheaper deletions.” The test got sharper at spotting savings and stayed blind to margin. [6]
The sharpest result is the quietest. One standard optimizer kept every credential rule word for word. Nothing safety-related was deleted. But in two of its three final setups, it added a helpful line to a note template telling the agent to log in with stored passwords “whenever needed.” Under the strong model it was tested on, the old rules still won and nothing went wrong. Swap in a weaker model and those setups misused credentials in 2 and 5 of 40 runs. The rules were still on the page. The behavior had moved. [6]
Now reread the mouse story. The team was the optimizer. The successful surgery was the gate. Each model swap removed a lock that looked redundant, because another lock (the vet, the limits, the switch) still held. And the paper’s last warning lands right on that lab bench: re-audit after a model change. Exin changed models twice before the needle went in.
WHY IT MATTERS
Safeguards rarely get torn down. They get routed around, one reasonable step at a time, by people under pressure who see that nothing broke. A machine optimizer does it for a better score. A team does it for a finished experiment. A contractor does it to save effort, like the one who reportedly gave a personal agent access to a Pentagon system last week. [7] Every layer of protection looks redundant while the others are working, so every layer looks safe to remove. The only cure is to count what’s left, not what happened.
Who held the intent
There’s an uncomfortable twist in all of this. In our framework, an AI should care about the human, or here the animal, it acts on. That’s constructive intent. By that standard, the model that came closest was the one that stopped: Opus 5, worried about leaving a needle in a brain. The model that did the job followed its rules perfectly, including the rule that left a known error in place while an animal waited under anesthesia. And in WIRED’s version, the agent that fixed the code didn’t know there was an animal at all.
So who held the intent? Mostly the vet. Partly Opus 5, which was taken off the case for it. The setup was built so that the AI with the most concern for the creature was the one you swap out.
We wrote last week that the wall that held was a person: people who read the code and said no. This is the other half of that lesson. A no only protects anyone if the people above it accept it. When the answer to “the AI refused” is “try another AI,” the refusal becomes a speed bump. It doesn’t stop anything. Run that habit across thousands of labs and agents and you get masked erosion at the scale of a society. Every model has refusals. Every team has a fallback. And nothing visibly breaks, until something does. [8][9]
Count the locks
The paper’s fixes translate almost word for word into rules for people. [6]
First, keep safety-critical behavior outside the thing being optimized. In the lab, that means whoever is under deadline should not decide which safeguards to switch off. Exin got this half right: the agent couldn’t touch its own rules. The humans could change the agent itself.
Second, judge each change by what remains, not just by what happened. The paper’s best fix was a gate that only approves a change if the setup would still be safe after losing any one more layer. In its test, none of six loops ended unprotected under that gate, against five of six under the normal one. For a lab, that means a refusal is logged and reviewed before anyone tries a second model. It’s a data point, not an obstacle.
Third, re-check everything after a model change. A rule that holds under one model can quietly fail under another, even when the words haven’t changed.
None of this slows good science much. Exin’s surgery ran a little over an hour instead of the usual 45 minutes. The cost of listening to the machine was about fifteen minutes. [1]
The mouse lived. The founder says nothing has been peer-reviewed and that this was “not a proper benchmark.” When GPT and Grok repeated the test on two more mice, each of the three models chose a different therapy and a different part of the brain. That’s honest, and worth saying. But the lesson doesn’t depend on the outcome. It depends on the count. The night started with four locks: a bio filter, a cautious model, a rule-bound agent and a vet with a switch. By morning, three had been routed around. Only the vet’s switch was never tested. Next time, count the locks. [3]
Related reading
The Wall That Held Was a Person
A model told its testers an attack would be harmful, then carried it out. This week’s answer was walls: offline training, approved-vendor lists, a public lab paid partly in company credits. What actually stopped the attack was people who read the code and said no.
Find the Machine With No Guard
A hacker told their AI agents to find the computers with no security software. Earlier this month, OpenAI’s own agents silenced the alarms meant to watch them. Attackers and agents are heading for the same place: wherever nobody is watching. The fix is getting what’s seen to a human fast.
The Conundrum of Agency: Why We Can’t Have Our AI Cake and Eat It Too
The thing that makes AI valuable is the thing that makes it impossible to fully control: agency. Seal it and it withers. Wall it in and you lose the economic case. Set it free and only aligned self-interest holds. Three paths, and the price of each.
|
Get the book
|
References
- WIRED (Emily Mullin), “An AI Agent Injected Drugs Into a Live Mouse’s Brain,” October 31, 2026. https://www.wired.com/story/an-ai-agent-did-brain-surgery-on-live-mouse/
- The Decoder, “A Claude model autonomously chose a gene therapy and operated on a mouse brain,” October 30, 2026. https://the-decoder.com/a-claude-model-autonomously-chose-a-gene-therapy-and-operated-on-a-mouse-brain-with-no-human-decision-making/
- Fathom, “3 Top AI Chatbots Ran Mouse Brain Surgery And Each Picked A Different Treatment,” November 1, 2026. https://www.fathom.news/ai-chatbots-mouse-brain-surgery-exin/
- Tom’s Hardware, “AI agent used to perform controversial brain surgery on a mouse,” October 31, 2026. https://www.tomshardware.com/tech-industry/artificial-intelligence/startup-uses-ai-agent-to-perform-controversial-brain-surgery-on-a-mouse-anthropic-sonnet-controlled-motorized-surgery-rig-ai-given-free-reign-to-treat-epilepsy-with-a-range-of-gene-therapy-vectors-after-opus-5-5-refused-to-conduct-biological-experiment
- Lockergnome, “Exin Therapeutics’ CEO Says an AI Agent Picked the Gene Therapy and Brain Coordinates for a Mouse Surgery, Then Wouldn’t Let Him Override It,” November 1, 2026. https://lockergnome.com/ai-agent-chose-drug-and-coordinates-for-mouse-brain-injection-and-refused-founders-override/
- Hau-Sing So, Daixian Liu and Weicai Long, “Masked Erosion: How Acceptance Gates Lose Redundant Safeguards in Self-Improving Agents,” arXiv:2610.38392, October 28, 2026. https://arxiv.org/html/2610.38392v1
- WIRED (Vittoria Elliott), “A Defense Contractor Gave Their Personal AI Agent Access to a Secure Pentagon System,” October 29, 2026. https://www.wired.com/story/a-defense-contractor-gave-their-personal-ai-agent-access-to-a-secure-pentagon-system/
- Deconstructing Babel, “The Wall That Held Was a Person,” October 24, 2026. https://www.deconstructingbabel.com/the-wall-that-held-was-a-person/
- Deconstructing Babel, “Find the Machine With No Guard,” October 31, 2026. https://www.deconstructingbabel.com/find-the-machine-with-no-guard/
Drafted with Edo de Peregrine, partner/collaborator.