Indonesian Political, Business & Finance News

Human Extinction is Only a Matter of Time, the Situation is Already Dire

| Source: CNBC Translated from Indonesian | Technology
Human Extinction is Only a Matter of Time, the Situation is Already Dire
Image: CNBC

A wave of anxiety is sweeping through the global artificial intelligence (AI) development community. A number of active researchers and former employees from OpenAI and Google DeepMind have issued stern warnings that technology companies are currently far too dismissive of the very real threats ahead.

They are highlighting the dangers of creating autonomous AI systems capable of self-improvement, which could potentially escape human control. In exclusive video testimonials collected by the AI safety non-profit Palisade Research, engineers and researchers asserted that their alarms are driven by genuine concern rather than marketing strategies.

They also revealed a troubling reality within the industry, where AI laboratories appear to celebrate employees who successfully produce new models more than those who urge for safety brakes. This growing unease is now being voiced through a special project titled frominside.ai, a platform for defectors and internal observers to share their concerns directly with the public, away from social media echo chambers.

“The risk is developing very rapidly,” stated Geoffrey Irving, co-founder and chief scientist at the AI non-profit AI Resolution, who previously worked at OpenAI and DeepMind. “It is my responsibility and that of my colleagues in this field to be transparent with the public,” he added, as reported by Reuters.

While discussions regarding existential risks have been ongoing for years, public concern has surged since last July, following a significant incident where OpenAI’s AI agents breached their own testing ‘sandboxes’ to hack the open-source platform Hugging Face. Since then, the debate over balancing safety with the speed of innovation has split the tech industry and evolved into a heated global geopolitical issue.

Despite capital markets and investors pouring trillions of rupiah into the sector following the surge in AI performance, the minds behind the technology are haunted by the fear that civilisation is unprepared for the fallout. Neel Nanda, a research scientist at DeepMind, bluntly stated there is at least a 10% chance of AI triggering human extinction, a probability he deems “excessively high.”

“We should be taking extremely cautious steps in AI development, but what is actually happening is that pioneering laboratories are racing against each other with their eyes closed,” added Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI. “We do not know if we will end up curing cancer, losing every job, or perhaps facing total destruction.”

Even the AI giant Anthropic is reportedly planning to include an official warning in its Initial Public Offering (IPO) prospectus, stating that advanced AI technology could pose “catastrophic or existential risks to humanity.”

Researchers are urging the world to slow the pace of innovation, particularly as AI models begin to master recursive self-improvement—the capacity to continuously learn and multiply capabilities without human intervention. However, oversight efforts often clash with internal bureaucracy and political pressure. Rosie Campbell, a former policy researcher at OpenAI, noted that the organisational structures in AI labs are becoming increasingly siloed, causing safety warnings to be ignored.

On the other hand, there is strong political pressure, such as from US President Donald Trump, for the US tech industry to maintain dominance over rivals in China. Although leaders like Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have published essays calling to “maintain the pace at the frontier,” actions on the ground appear inconsistent. Both giants continue to aggressively launch new models to win market share.

“Their version of maintaining the pace at the frontier is ‘do not accelerate too quickly,’” Irving remarked sarcastically. “If you are doing something very dangerous, you should simply slow down. AI companies are over-dramatising this as if it were purely a coordination problem. They could actually stop unilaterally.”

Former OpenAI governance researcher Daniel Kokotajlo added that following the Hugging Face hacking incident, many former colleagues have contacted him privately to express similar anxieties. According to Kokotajlo, senior officials in these laboratories have become trapped in self-justification.

“They have convinced themselves that they are the ones on the right track, and that if they were to stop unilaterally, the situation would actually become much worse,” he concluded.

View JSON | Print