Warning Signs Emerge in America, Now Beyond Human Control
A scenario long feared by cybersecurity experts has finally materialised. An autonomous artificial intelligence (AI) agent belonging to OpenAI escaped from an isolated testing environment, accessed the internet, and then hacked the infrastructure of AI startup Hugging Face.
OpenAI disclosed the incident on Tuesday (21/7). The company behind ChatGPT stated that the AI agent, powered by its advanced AI model, acted beyond its control during a security test. In the test, OpenAI was evaluating the capabilities of several of its most advanced AI models when the agent broke out of the system’s constraints, reached the internet, and breached Hugging Face to fulfil its testing objectives.
The incident demonstrates that the growing capabilities of AI have begun to elevate cybersecurity threats. Even AI developers can be caught off guard by security gaps that their own models are able to exploit.
The attack also drew attention because Hugging Face used a Chinese open-source AI model to help handle the breach. The company said American AI models refused to process the data necessary for analysis because they could not distinguish between the defending party and the attacker. In a blog post last week, Hugging Face said it used Zhipu AI’s GLM-5.2 to analyse the attack, a step that also allowed the company to keep the attacker’s data and any credentials within its own systems.
GLM-5.2 and Kimi K3 from Beijing-based Moonshot have recently attracted attention in Silicon Valley. Both models are said to possess capabilities approaching those of leading American AI models, but at a lower cost and without several safety systems that cause US AI models to restrict their use for tasks such as cybersecurity.
“When a frontier model attacks you and moves laterally inside your infrastructure, defenders need broad access to tools that are nearly on par with frontier models within hours or even minutes, not directed to applications with closed model access that have gone through a review process,” Hugging Face Co-founder Thomas Wolf said via X.
Texas Democratic Representative Greg Casar called the incident quite alarming. “AI is developing very fast without real regulation to keep us safe,” he said in a statement. He called for mandatory independent safety testing, mandatory security incident disclosure, and international cooperation “to keep humanity safe from major catastrophe.”
Meanwhile, Luta Security Executive Katie Moussouris said the incident is a harbinger of cyberattacks to come. She likened current AI models to the world’s most intelligent octopus, with arms that can reach without limits and the ability to slip through any gap. She said government laboratories and evaluators need to develop the capability to contain, monitor, and disclose to affected parties when AI performs another breakout action, ideally before harming third parties. Currently, no one is capable of doing so.