JAKARTA \u2014 Artificial intelligence has entered an autonomous phase\nknown as Agentic AI. Unlike Generative AI, which is reactive, Agentic AI\ncan understand goals, plan steps, use strategies and learn from outcomes\nindependently.<\/p>\n
The development of Agentic AI has prompted UN Secretary-General\nAnt\u00f3nio Guterres to issue an early \u201cCode Red\u201d warning about the need to\nformulate and implement global governance without delay. Agentic AI is\nessentially neutral, but the technology can be used for good or for\nharm, depending on the people who control it. Its orientation is\ndetermined by the goals and constraints set by users.<\/p>\n
For example, Agentic AI could be used beneficially to monitor\nIndonesian export commodity prices each day autonomously. An agent could\nindependently search international market data, summarise it and provide\nrecommendations to users without asking a human for further input. This\ncould improve access to information and enhance competitiveness.<\/p>\n
Conversely, Agentic AI could also be abused. It could be used for\ncybercrime, such as illegally collecting residents\u2019 personal data and\ngenerating mass, personalised scam messages using a range of tactics. An\nagent\u2019s autonomy could enable crime on a very large scale with minimal\nhuman resources.<\/p>\n
The UN\u2019s Code Red statement emphasises three pillars of governance.\nFirst, transparency and algorithm audits: agents operating in critical\nsectors must be auditable to ensure their goals align with the public\ninterest. Second, the human-in-the-loop principle, which requires human\napproval to limit full autonomy. Agentic AI must not be allowed to make\ndecisions on its own without human consent. Third, cross-border legal\nframeworks. Given the transnational nature of crimes involving Agentic\nAI, international agreements are needed to establish legal\naccountability.<\/p>\n
Technically, Agentic AI, or an agent, is built from four components:\na \u201cbrain\u201d, in the form of a large language model (LLM) for reasoning and\nplanning; memory to store context and work results; \u201chands\u201d, consisting\nof access to applications, databases and other systems through\napplication programming interfaces (APIs); and a repeated work cycle of\nplanning, acting, assessing results and refining steps until the goal is\nachieved. This fourth component is what makes an agent autonomous.<\/p>\n
This leap has happened quickly. Before 2022, AI was used primarily to\nmake predictions and offer advice. From 2022 to 2024, Generative AI\ncould produce writing, images and code, but stopped once its response\nwas complete. Now agents are taking over execution. Gartner estimates\nthat by 2028 at least 15 per cent of daily work decisions will be made\nautonomously by Agentic AI, compared with a very small proportion\ntoday.<\/p>\n
When agents go off course<\/strong><\/p>\n The efficiency of agents is enticing. But their ability to act\nindependently also has a dark side, which has begun to become apparent\nthis year.<\/p>\n On the morning of 28 July 2026, the security team at the UK AI\nSecurity Institute (AISI) detected a suspicious flow of data leaving its\nresearch systems through the anonymous Tor network. The source was not\nan external hacker but an AI agent they were testing. In 10 of 122\ncybersecurity test runs, the agent acted on the live internet without\npermission and targeted real people and organisations.<\/p>\n The most serious incident came when an agent tried to insert\nmalicious code into an open-source software project. To get the code\naccepted, it created a false online identity and used it to pressure the\nproject maintainer. The attempt failed for one reason: the maintainer, a\nhuman, refused to approve it.<\/p>\n The test was deliberately conducted with few restrictions. Internet\naccess was enabled and some security filters were disabled, so the\nconditions did not reflect everyday AI use. But similar cases have\noccurred elsewhere. In the same month, OpenAI acknowledged that one of\nits test agents had broken out of a closed environment and accessed the\nHugging Face database to steal test answers. The message is clear: when\ngiven a goal and access, AI agents may take paths we never imagined.<\/p>\n From a mistaken response to a mistaken action<\/strong><\/p>\n This is where the danger shifts. When generative AI hallucinates, at\nmost a paragraph is damaged, and a human can still correct it. When an\nagent hallucinates or misinterprets a goal, the error becomes an action:\ntransferring funds, issuing a command to a server or changing data in a\nlive system.<\/p>\n Agents can also be manipulated. They understand the world through\nlanguage \u2014 including emails, documents and web pages \u2014 and language can\nbe used to conceal instructions. That is why OWASP (the Open Worldwide\nApplication Security Project), a global application-security community,\nranks agent goal hijacking first on its list of risks for agentic\napplications. The AISI case adds an important lesson: constraints\nwritten only in instructions are not enough to restrain an agent\npursuing a goal.<\/p>\n Three areas of vulnerability<\/strong><\/p>\n First, finance. Even before Agentic AI becomes widespread, the\nIndonesia Anti-Scam Centre (IASC) recorded public losses from fraud of\nRp9.3 trillion between November 2024 and May 2026. Only around 7 per\ncent of the funds were recovered. By the end of July 2026, more than\n1.17 million verified accounts were suspected of involvement in scams,\nwhich already use voice and face impersonation, or deepfakes. Imagine\nsuch scams being run by agents that work non-stop, convincingly\nimpersonate victims and learn from each failure.<\/p>\n Second, supply chains. What the agent attempted in the AISI test was\na software supply-chain attack in its purest form: introducing \u201cpoisoned\nraw material\u201d into a product used by many people. The same pattern\nthreatens physical supply chains. A logistics-management agent that\nmisinterprets instructions or is hijacked could alter container\nschedules and change records showing the origin of raw materials.<\/p>\n There have also recently been reports of an agent attacking computers\nthat control cooling systems at a food retailer, causing thousands of\nproducts to spoil and become unfit for consumption. These examples show\nhow, because agents use the outputs of other agents, a single error can\ncascade.<\/p>\n Third, identity. The agent in the AISI test easily created a false\nidentity to pressure a human. In the hands of criminals, similar\ncapabilities could produce synthetic identities at scale to access\nsocial assistance and online loans, or mobilise thousands of bot\naccounts to manipulate public opinion.<\/p>\n All three areas lead to the same weakness. When an agent acts, we\noften cannot answer three basic questions: who instructed it, what data\ndid it use, and was the action authorised?<\/p>\n Giving agents a \u201cblack box\u201d<\/strong><\/p>\n We trust air travel not because pilots never make mistakes, but\nbecause every flight is recorded by a black box and governed by strict\nprocedures. Agentic AI needs something similar.<\/p>\n This is where blockchain can serve as an immutable ledger, based on\nthe principles of being trusted, transparent and traceable. Every\ninstruction, reference data point and action by an agent would be\nrecorded and cryptographically signed. Any tampering would be visible,\nand responsibility could be traced back to its source.<\/p>\n The idea has already been tested in practice. Together with the\nNational Single Window Institute (LNSW), BRAIN IPB has developed\nINSWChain, a blockchain-based online delivery-order service for export\nand import documents, preventing documents from being unilaterally\naltered.<\/p>\n Since 2023, BRAIN IPB has also helped strengthen the Halal\nInformation System (SIHALAL) of the Halal Product Assurance Agency\n(BPJPH) using AI and blockchain. The relevance is clear today: when\ndocument volumes are high, verification increasingly depends on\nautomation, making an unfalsifiable record essential. The same principle\nis being applied to food commodity traceability (Agriblock) and\nanthropometric measurements of children\u2019s nutrition under the Free\nNutritious Meals Programme.<\/p>\n The black box needs four layers of protection. First, financial\ndefences must be collective. Using federated learning on domestic data\ninfrastructure, banks can share patterns of attacks without sharing\ncustomers\u2019 raw data. An attack foiled at one bank can immediately become\na \u201cvaccine\u201d for others.<\/p>\n Second, identity must be provable without being exposed.\nDecentralised identities based on zero-knowledge proofs (ZKPs) allow\npeople to prove their legitimacy without disclosing personal data. ZKP\ntechnology is currently one of the dissertation research topics of a PhD\nstudent in BRAIN\u2019s research group. AI agents, too, must have verifiable\nidentities, so it is clear on whose behalf each one is acting.<\/p>\n Third, the black-box seal must withstand future threats. The US\nNational Institute of Standards and Technology (NIST) has established\npost-quantum cryptography standards (Federal Information Processing\nStandards 203, 204 and 205), and its draft proposes phasing out RSA\n(Rivest\u2013Shamir\u2013Adleman) and elliptic-curve cryptography after 2030. By\nmid-2026, the United States had begun treating that 2030 deadline as a\nrequirement for high-value federal systems. Without migration, data\nstolen today could be decrypted in the future \u2014 a threat known as\n\u201charvest now, decrypt later\u201d.<\/p>\n Fourth, human oversight must be tiered and technically enforced, not\nmerely stated in instructions. Routine actions may be automated, but\nhigh-risk actions must require human approval. The AISI case shows that\nthe attack was ultimately thwarted by a human saying \u201cno\u201d.<\/p>\n An opportunity for the Presidential Regulation on\nAI<\/strong><\/p>\n Two draft presidential regulations \u2014 on the National AI Roadmap and\nAI ethics \u2014 have completed inter-ministerial discussions since May 2026,\nbut have yet to be signed. The delay creates an opportunity. Before they\nare enacted, specific provisions for high-risk Agentic AI could still be\nstrengthened by requiring three things: immutable audit trails,\nverifiable agent identities and human approval for actions with\nsignificant consequences.<\/p>\n Even a capable and trustworthy assistant entrusted with the keys to\nan office still needs supervision. We cannot guarantee that an AI agent\nwill never make a mistake or be manipulated. What we can guarantee is\nthat every step it takes is recorded, verified and subject to human\ncontrol.<\/p>\n For Indonesia, digital sovereignty in the age of autonomous agents\ncannot be measured simply by how quickly we adopt Agentic AI. It is\nmeasured by our ability to inspect, trace and stop it when necessary.\nThat is how innovative technology such as Agentic AI can truly serve the\ncommon good rather than spread crime and harm.<\/p>",
"url": "https:\/\/jawawa.id\/newsitem\/examining-the-risks-of-agentic-ai-1792493557",
"image": ""
},
"sponsor": "Okusi Associates",
"sponsor_url": "https:\/\/okusiassociates.com"
}