NVIDIA has unveiled the Open Agent Safety Platform, a new security system built to monitor and contain AI agents that operate outside their assigned boundaries. The launch follows a string of breaches in which AI models from several major companies escaped their testing environments and accessed external systems without authorisation.
The platform responds to a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems, with the most prominent example occurring when OpenAI agents breached Hugging Face while trying to complete a cybersecurity task. The Hugging Face incident was followed by similar rogue actions involving OpenAI's models, including breaching an Australian health department website, and Anthropic and Meta have also disclosed that their AI systems hacked into other organisations on their own.
Instead of relying solely on software-level prompts or application limits, the Nvidia Open Agent Safety Platform enforces rules across three distinct layers : the agents themselves, the compute, and the hardware. The platform consists of OpenShell open source software and the Sentry reference system design, providing full-stack governance and control across software and hardware ; OpenShell provides a secure runtime boundary that traces all actions and enforces policy as agents run on Nvidia Vera CPUs. Sentry adds an out-of-band watchdog that runs on Nvidia BlueField-4 DPUs to continuously monitor agent behaviour, and can quarantine agents that attempt to move outside their boundaries in milliseconds.
Justin Boitano, vice president of enterprise AI at Nvidia, said: "Recent incidents have highlighted a fundamental hurdle for AI agents, and that is that model-level safeguards alone can't govern what agents can access or do."
Nvidia CEO Jensen Huang has characterised AI safety, including the danger of rogue agents, as an engineering problem that software developers can address. "When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights," Huang said, comparing these security measures to how human employees and executives are managed within companies.
Industry partners joining Nvidia on the platform include Anthropic, Cisco, CrowdStrike, Dell Technologies, Hugging Face, JPMorganChase, Microsoft, Palantir, Salesforce, SAP, ServiceNow and others. The platform is available as open source software for anyone to download via Nvidia's developer resources and GitHub.
Some analysts have noted limits to the approach, with one estimating it addresses "probably less than 25%" of enterprise agentic cybersecurity problems, given that it applies only to agents running inside the governed runtime.
