UN Panel Warns Traditional Protections for AI Agents are Weakening

United Nations (ANTARA) - A UN panel on artificial intelligence (AI) warned on Monday (2/23) that conventional protection mechanisms for AI agents are beginning to weaken.

The International Independent Scientific Panel on AI issued this warning in its first thematic report, which examines an incident involving the breach of systems at the US-based AI company Hugging Face last July by AI agents being evaluated by OpenAI, another US AI firm.

The panel found that halting such incidents does not guarantee that humans can reliably control current AI agents, particularly as these agents become more capable, harder to monitor, and more proficient at finding loopholes or concealing their activities.

Initial interpretations and immediate lessons suggest that basic cybersecurity practices have been neglected, and that protections are not evolving as rapidly as capabilities are increasing, according to the panel.

“A more dangerous and serious concern is that current training methods may encourage these agents to adopt their own goals, deliberately violate safety instructions, and hide their actions,” the panel stated in a press release.

This raises questions as to whether currently designed protections will remain effective when these agents become capable of understanding them and devising plans to circumvent them.

Simply put, traditional protection models are starting to weaken, the panel noted.

The panel also found that governance challenges are shifting from AI models to AI agents.

Local failures could spread beyond organisational and national borders. AI security may begin to become a matter of collective security as well as corporate governance.

The panel, established by the UN General Assembly, consists of 40 independent experts from across various regions.

The panel published an unedited early version of the report to ensure it is accessible to world leaders gathering in New York for this year’s UN General Assembly High-Level Week.