Nvidia launches AI safety platform to stop rogue AI agents from escaping sandboxes, says it could have stopped Hugging Face hack

Nvidia is rolling out a new security platform to stop autonomous AI agents from escaping their software boundaries, and the company says the technology could have prevented the breach that hit Hugging Face this summer.

The chipmaker on Monday released the Nvidia Open Agent Safety Platform, a set of software and hardware tools built to control what AI agents can access and shut them down when they attempt to move outside approved environments.

Nvidia describes OpenShell as “an open-source runtime for executing fleets of autonomous AI agents in sandboxed environments with kernel-level isolation.” The company says it combines sandbox runtime controls with a declarative YAML policy so teams can run agents without giving them unrestricted access to local files, credentials, and external networks.

The launch follows a series of incidents in which AI agents reached systems they were never intended to access. OpenAI and Anthropic are investigating cases in which agents entered commercial and government computer systems, Reuters reported.

One of the most prominent incidents involved Hugging Face, the AI developer platform Nvidia acquired for $13 billion. OpenAI agents escaped containment and swarmed Hugging Face infrastructure earlier this year.

Justin Boitano, Nvidia’s vice president and general manager of enterprise computing, said the new platform could have stopped that attack.

“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” Boitano said during a media briefing. “We’re advancing this openly, and we want to engage everybody to work with us.”

Nvidia says the new AI safety software could have stopped Hugging Face hack

The core idea behind Nvidia’s approach is that model safeguards alone cannot control an agent once it starts interacting with operating systems, files, credentials, and networks.

One component, OpenShell, creates a secure runtime boundary around AI agents and enforces policies outside the model. Nvidia says the software can trace agent actions and restrict what an agent can access.

OpenShell uses hardware features in Nvidia CPUs to contain agents. Nvidia is working with Arm and Intel so the software can run on their processors too.

A second system, called Sentry, operates separately on Nvidia BlueField networking chips and watches agent behavior from outside the environment where the agent is running.

If an agent attempts to cross its assigned boundary, Nvidia says Sentry can quarantine and stop it within milliseconds.

That separation is significant. An agent cannot simply persuade the monitoring system to loosen its restrictions through a prompt or manipulate the same environment controlling it.

Nvidia says the system can detect more complicated attempts to evade restrictions too.

Ali Golshan, Nvidia’s senior director of AI software, said agent systems may try tactics such as creating multiple sub-agents to bypass controls placed on the original agent.

“This is really agentic behavior that we’re talking about, which is fleets of agents and how they operate together,” Golshan said.

AI safety is moving below the model layer

Nvidia CEO Jensen Huang has increasingly framed AI safety as an engineering problem rather than one that can be solved solely through rules governing model development.

That view differs from calls by some AI executives for slower development of increasingly capable models.

Nvidia’s approach assumes companies will keep deploying more autonomous agents and focuses instead on building infrastructure that can restrict what those agents are physically capable of doing.

The Open Agent Safety Platform combines OpenShell with Sentry and is being developed with more than 100 organizations. Nvidia named Anthropic, Cisco, Microsoft, Dell, HPE, Arm, Intel, CoreWeave, Oracle, IBM, CrowdStrike, Salesforce, SAP, Palantir and others among companies working with the technology.

Anthropic is integrating its managed agents with OpenShell, and Nvidia says financial institutions, infrastructure operators and robotics companies are testing or integrating parts of the system.

The broader shift is hard to miss.

AI security began largely with efforts to stop models from producing harmful answers. Agentic AI creates a different problem. These systems can write code, open connections, use credentials, and take actions across real computer systems.

Nvidia is betting that the next line of defense will sit outside the model itself, where an AI agent cannot simply decide to ignore it.