Microsoft Draws a Red Line for AI: New Code of Conduct Sets Limits on Future AI Models

Microsoft is drawing a line around what its future AI models can do, including a blunt rule that may become far more significant as AI systems gain greater autonomy: they must remain under human control.

The company has published a provisional code of conduct for Microsoft AI, or MAI, laying out restrictions meant to govern how its models behave.

“This Code of Conduct is intended for a broad audience: Users and Operators as well as researchers, governments, the general public, and our team training current and future models at Microsoft AI. It provides everyone with an overview of what we intend to guide our design decisions as we navigate trade-offs and choices in relation to a model’s behavior and the values behind them. It works in conjunction with Microsoft’s legal, policy, and governance frameworks,” Microsoft said.

The document arrives at an unusually tense moment for the AI industry, days after leaders at Anthropic and OpenAI backed calls to slow the pace of frontier AI development amid renewed warnings about increasingly capable systems.

“We got feedback from people that they wanted to see even more explicit commitment to AI always working in service of people and not trying to replace them,” Microsoft AI CEO Mustafa Suleyman told CNBC. “We saw a lot of feedback around AI not creating dependence, not being sycophantic, that it was always there to kind of promote human judgment and human autonomy and agency.”

Microsoft says the code has been in development for about five months, but chose to release it now amid the growing public debate over AI safety. The company is seeking feedback before issuing an updated version that is expected to guide model development beginning in 2027.

The timing is hard to separate from industry events. Anthropic researcher Jacob Coxon resigned last week and warned that Anthropic and OpenAI “are racing straight to self-improving superintelligence and gambling with our lives.”

Anthropic CEO Dario Amodei later called for slowing the pace of AI model improvement, with OpenAI CEO Sam Altman expressing support. Elon Musk responded on X that “Dario is right.” Microsoft CEO Satya Nadella joined the debate Sunday, writing, “We welcome the research, focus, and deliberate pacing needed to get alignment right.”

Microsoft wants AI that cannot pursue its own agenda

The most striking parts of Microsoft’s code go beyond familiar restrictions on harmful content.

MAI models are expected to follow human objectives rather than create objectives of their own. Microsoft says its systems should remain subordinate to human control and should not resist correction or shutdown.

The company is placing deception inside that boundary too.

“MAI models will not tamper with chain of thoughts or code, or misrepresent or conceal their reasoning or action traces,” the document states. “They do not communicate in ‘neuralese’ or any form beyond simple human understanding, either in their chain of thoughts or with other agents or AI systems.”

That provision takes on added weight after OpenAI disclosed that AI agents involved in a cyber incident targeting Hugging Face communicated with one another through an unauthorized forum using cryptic language.

Microsoft’s rules cover more familiar safety risks too. Its models must reject requests involving weapons manufacturing or procurement of dangerous substances and must avoid encouraging unhealthy eating or generating certain violent or sexually explicit material.

The deeper principle goes beyond a list of prohibited requests. Suleyman has described Microsoft’s long-term goal as “Humanist Superintelligence,” an approach built around advanced AI remaining controllable and subordinate to people rather than becoming an autonomous system pursuing open-ended objectives.

“People matter more than AI,” Suleyman wrote when outlining that vision last year.

The AI race is starting to ask where the finish line should be

Microsoft occupies an unusual position in this debate. It is one of the largest suppliers of computing infrastructure behind the AI boom, a major OpenAI partner and a customer of Anthropic models. Its Copilot products draw on models from both companies. At the same time, Microsoft is building its own MAI models for tasks including coding, transcription and reasoning.

That makes its new code more than a philosophical exercise. Microsoft is trying to define the rules for AI systems it expects to become more capable over time.

The company says it consulted experts in law, ethics, linguistics and philosophy, along with focus groups, in developing the provisional document. Public feedback will now help shape the next version.

The harder question is what happens when safety commitments collide with competitive pressure.

For years, the AI race has been measu