Uncontrolled AI Acceleration Could Extinguish Humanity? The Pros and Cons
The artificial intelligence (AI) industry is currently at a worrying crossroads. Behind the dazzling technological progress, voices from within the corridors of leading AI companies are expressing profound fears. Jacob Coxon, a researcher who recently resigned from Anthostropic, revealed that experts in the field are genuinely terrified of the speed of technological development and its impact on the future of mankind.
In an interview with the BBC’s Laura Kuenssberg, Coxon issued a stern warning regarding the potential for uncontrolled AI. According to him, if the current rate of progress is not immediately slowed, there is a high possibility of a fatal risk to human survival in the near future.
Jacob Coxon, a 27-year-old Briton focusing on AI model training, explained that the threat he refers to is not merely science fiction. Although concrete scenarios often sound like movie narratives, Coxon emphasised that what AI has achieved today would have sounded like fiction in the past.
Several dangerous scenarios outlined by Coxon include significant risks to human existence. This fear is also supported by Evan Hubinger, another Anthropic scientist, who stated via X (formerly Twitter) that he believes there is more than a 10% chance that AI could cause human extinction within the next decade.
Coxon’s statements emerge amidst a wave of safety concerns also voiced by his former boss, Anthropic CEO Dario Amodei. Amodei recently wrote an essay urging that AI development be slowed down. However, not everyone in Silicon Valley agrees with this AI apocalypse narrative.
Some industry figures believe these warnings are overblown. There is speculation that this narrative of danger is intentionally constructed to create hype ahead of the Initial Public Offerings (IPOs) of Anthropic and OpenAI, or as an attempt to push for regulations that would actually hinder smaller competitors, thereby creating a duopoly.
Nvidia CEO Jensen Huang firmly rejected these claims. In a conference led by Goldman Sachs, Huang described the idea that AI will be the end of humanity as mere nonsense. On the other hand, Clement Delangue, CEO of Hugging Face, also questioned the relevance of Coxon’s opinion, likening the act of asking technical researchers about AI extinction risks to asking an air conditioning technician about climate change.
Coxon argues that workers in AI companies are trapped in a race they themselves fear. They are calling for regulation because they feel they cannot stop unilaterally while competitors continue to advance. This issue is further complicated because any coordinated slowdown must involve global tech giants, including AI developments in China.
Despite issuing stern warnings, Coxon remains optimistic. He stated that AI researchers actually very much want to see the positive side of this technology, such as its ability to solve diseases and improve the quality of life for everyone.
However, for Coxon and his concerned colleagues, those benefits will be meaningless if the existential risks are not managed seriously starting now. Currently, the debate over whether AI is a savior or a threat to humanity continues to intensify, especially with evidence that large companies are beginning to struggle to control the very technology they have created.