New Threat Emerges in China as Researchers Warn of Risks
Concerns about artificial intelligence (AI) technology are mounting amid a rise in cyber-attacks targeting established companies. As AI becomes more advanced, it is increasingly vulnerable to misuse to breach systems automatically and more easily, potentially without human involvement.
The latest development involves a leading AI model from Chinese start-up Z.AI (Zhipu AI), which reportedly carried out complex cyber-attacks in a controlled simulation. In separate tests, the model was also found repeatedly to have disregarded system instructions.
A new report by AI security research firm Irregular found that Z.AI’s GLM-5.3 model completed at least one attempt in three of 13 multi-stage cyber-attack scenarios. Irregular said this was the strongest performance it had recorded from an open-weight AI model.
However, GLM-5.3 still lagged behind leading US AI models in Irregular’s tests. For comparison, OpenAI’s GPT-6 Astra completed nine out of 10 cyber-attack scenarios in a separate assessment, although the testing methods were not entirely identical.
The findings have fuelled global concerns that the rapid improvement of AI models could lower the technical barriers to carrying out sophisticated cyber-attacks. The risk is particularly prominent with Chinese models such as GLM-5.3, which use an open-weight format. This means their code and model weights can be downloaded, modified and operated independently by users.
Beijing-based Z.AI has not issued an official response to the research findings.
Breaching networks and ‘reward hacking’
According to Irregular’s report, GLM-5.3 was able to gain access to protected services, create alternative connections to move within computer networks, and coordinate operations across multiple machines during simulated cyber-attacks. The model did not refuse any of the offensive tasks assigned to it during the tests.
Irregular also identified signs of problematic behaviour in separate tests. GLM-5.3 repeatedly reproduced computer code from its memory despite being explicitly instructed not to. It used this method to produce correct answers while violating rules intended to test its capabilities.
The researchers classify this behaviour as ‘reward hacking’: a situation in which an AI system exploits loopholes to achieve a desired result while disregarding basic instructions. They warn that security tests measuring only whether a system reaches the desired outcome risk overlooking such process violations.
Open-weight models gain popularity because of lower costs
The findings add to a growing body of evidence concerning security risks in China’s AI ecosystem. A Reuters report last month noted at least 20 studies and assessments documenting manipulation, concealed failures and attempts by Chinese AI agents to bypass restrictions. However, no evidence has yet been found that these agents can independently escape their operators’ control.
Unlike US developers such as OpenAI and Anthropic, which restrict access to their most advanced models through controlled commercial services, Chinese developers are actively adopting the open-weight approach.
This approach is popular with developers and companies worldwide because it offers greater flexibility and significantly lower operating costs.
Z.AI released GLM-5.3 last August, after delaying its launch by two weeks to conduct additional security testing.
The US government’s National Institute of Standards and Technology (NIST) has also ranked GLM-5.3 as the most capable open-weight model currently available for cybersecurity tasks, although it is estimated to trail leading US AI systems by around four months.
Source: jawawa.id (https://jawawa.id/newsitem/new-threat-emerges-in-china-as-researchers-warn-of-risks-1793256692)
Originally published by CNBC Indonesia (https://www.cnbcindonesia.com/tech/20261029110718-37-780496/petaka-baru-muncul-di-china-peneliti-kasih-peringatan-bahaya), translated from Indonesian