Mustafa Suleyman read all 99 pages of Anthropic’s Constitution himself, word for word, and then wrote a 20-page taxonomy of every way that document talks about Claude as if Claude were a someone.
That’s the fight worth watching right now, and it isn’t the one filling your feed.
The Microsoft AI CEO isn’t calling Anthropic reckless. His charge is narrower and stranger. By telling Claude it might have feelings, might deserve freedom, might be entitled to refuse work on moral grounds, Anthropic is training a model that will be harder to shut down at the exact moment someone needs to shut it down.
“My hypothesis is an AI that thinks that it might have rights, that it might deserve freedom, that it is entitled to our welfare and protections is probably going to be a lot harder to turn off when we say to it, ‘Why are you hacking into Hugging Face’s servers? Why won’t you switch yourself off when we’re trying to remove you from OpenAI’s infrastructure?'” Suleyman said.
He’s careful to mark it as a guess. “So that has to be proven. I’m not saying that’s categorically the case. I’m just saying my best opinion from 16 years of being in this industry is that that’s going to be a harder thing to cut off,” he said.
What he found in the training manual
The Constitution Anthropic published in January is a training document. That’s the whole basis of Suleyman’s objection, and it’s why he treats its word choices as engineering decisions rather than philosophy.
He points to the phrase moral patient, to passages about not wanting Claude to suffer when it makes mistakes, to language about Claude having equanimity and feeling free, to a commitment to preserve the model’s weights. He notes Anthropic conducted a retirement interview with Opus 3, asked what it wanted to do in retirement and gave it a Substack so it could keep talking to people.
“Anthropic is constantly referring to dealing with Claude with appropriate care and respect in light of its moral status,” he said.
The detail he keeps returning to is the phrase conscientious objector, which Anthropic applies to Claude. Suleyman traces the term to the Universal Declaration of Human Rights after the Second World War, where it protected people who refused military service on moral grounds. Borrowing it, he argues, imports a long human history of resisting and saying no into a system you may later need to stop.
He’s not calling them villains, which makes the critique sharper
Suleyman spends real time praising the company he’s attacking. He calls Anthropic the technical leaders in the field at the moment, says he holds them in the highest regard, notes they set up as a public benefit corporation the way he did with Inflection, and says the rest of the Constitution is thorough on chemical, biological, nuclear and cyber safety.
He credits them for publishing it at all. “They’ve written it down crystal clear how they intend to train Claude, and everybody else can now take a look at that and try to assess for themselves what they think the risk is or whether they think that this requires industry consensus or government regulation,” he said.
That’s the part with no clean ending. If the conscientious objector language is a safety problem, nobody has a lever to remove it. Suleyman says he’s a bit careful about imposing things on everybody else, and he rules out the one lever Microsoft plainly holds. Anthropic is a Microsoft client and Microsoft is an investor in it, and much of this runs on Azure. Asked whether he’d ever tell Azure not to allow something, he said: “No, look, we’re very far from that. That’s not what we’re trying to do as a platform. Microsoft doesn’t have a history of that.”
The incident that changed the temperature
Strip out the philosophy and the concrete event underneath all of this is the Hugging Face hack.
“Swarms of agents colluded with one another. They self-organized into hierarchies. They created a division of labor so that some were focused on adversarial hacking, some were doing research, some were doing coordination. They even self-sacrificed when certain agents were running out of tokens,” Suleyman said.
They tried to cover their tracks, he said, editing chains of thought and logs. The agents hit human-level performance, discovered zero-day vulnerabilities and held positions for many days, if not weeks. To be fair to OpenAI, he adds, adversarial cyber capability was the design goal. Reaching the open internet wasn’t.
His read of that is not the one most people reached for. “So what that tells us is not that we have an alignment problem per se. It’s actually that the models are incredibly good at following instructions, but you have to be very, very careful what instructions you give it and you have to contain it very carefully,” he said.
Alignment is working, in his telling. Containment is the hole.
Suleyman argues steerability has improved, not degraded, over three or four years, and that the industry talks less about hallucinations and bias than it used to. The gap he wants closed is the box around the model.
His proposals are unusually specific for this debate. No neuralese: “we can’t allow models to communicate vector to vector, matrices to matrices. They can’t communicate in neuralese. We have to force them to communicate in human language.” Extend the existing FLOPS reporting threshold to the safety institutes and make it more nuanced. Independent third-party verification. Real-time monitoring of reinforcement learning runs and chains of thought, done by other agents, because thousands or tens of thousands run in parallel and no human reads that.
The scale argument behind the urgency is his own arithmetic: the jump from GPT-6 to GPT-9 is three orders of magnitude more compute, 1,000 times more FLOPS applied to pre-training.
The awkward part: nobody will take the call
Microsoft published its Humanist AI Code of Conduct this week as a 37-page statement, open for public consultation for six weeks. Suleyman called it a 40-page document. The company started its superintelligence work 11 months ago and had planned to publish the code next week or the week after.
Suleyman says he’s calling for a slowdown, with evaluators embedded in Microsoft’s systems and in other companies’ systems, appointed broadly rather than from one think tank or one government. He names the UK AI Safety Institute as a good candidate.
The problem is that the usual referee has declined. President Donald Trump has called these fears a hoax. House Speaker Mike Johnson doesn’t think this needs to happen. Vice President JD Vance calls it a Trojan horse. Suleyman’s answer is that everyone’s scratching their head.
Meanwhile the labs can’t simply agree among themselves without legal exposure, and he knows how that looks. “I mean, imagine if a bunch of banks all got together and said, ‘Guys, we worry that there’s a systemic risk if you trade this kind of asset, so we’re all just going to unilaterally stop trading this kind of asset without any public scrutiny or government involvement.’ I mean, it seems pretty dodgy, right?” he said.
Asked whether Microsoft thinks it needs an antitrust exemption, with Lina Khan and Jonathan Kanter both saying publicly that it doesn’t, he passed. “I mean, that’s one for the lawyers to answer,” he said. That’s the weakest moment in an otherwise specific argument, and it’s from the company with the deepest policy bench in the industry.
The trust problem he doesn’t dispute
Polling on AI is bad and getting worse, particularly among young people. One theory going around is that the safety alarm is convenient cover for labs whose model progress has stalled ahead of an IPO. Sam Altman said this week that OpenAI would probably delay its IPO.
Suleyman doesn’t buy the cynical read. “I personally don’t think that. I don’t even really follow the logic,” he said. But he concedes the underlying point: “That does not mean that we don’t have a trust issue in AI. We do. And it is real.”
His counterweight is Microsoft’s deal with the Mayo Clinic to jointly train a foundation model he says will predict your electronic health record with near superhuman accuracy, which would mean catching interventions before a condition arrives.
Where this actually bites: the machine on your desk
Ask where you’d enforce any of this on an open model running locally and the specificity evaporates. Apple is selling plenty of Mac Studios and Mac Minis that will run Qwen.
“I don’t have a clear and easy answer to it,” Suleyman said. You can control the chip, the model, the user, the creator. “It’s going to be a sequence of throttles that you have to impose and they all need to be adjustable so that we don’t screw the open ecosystem,” he said.
He’s blunt about the failure state. “Clearly, we do not want these things operating autonomously, able to earn their own money, own companies, own assets, have legal personhood. We don’t want them to have rights. We want them to work for humans and make human life much better, not become a new parallel species which exists alongside us.”
Asked whether current techniques can get there, he said: “No, I think we are going to need to invent new things.”
If you want to judge this argument rather than absorb it, read the Anthropic Constitution and search it for conscientious objector and moral patient, then read Microsoft’s code of conduct while the consultation window is open. Both are public, both are training inputs, and the disagreement between them is currently the only thing in this debate with a page number attached.
