Researchers Warn AI Could Lead to Human Extinction: How Is This Possible?

Internal discussions within several of the world’s leading artificial intelligence (AI) companies regarding the potential existential risks of the technology have intensified over the past week. The catalyst was the resignation of Anthropic researcher Jacob Coxon, who openly expressed concerns that AI development is proceeding too rapidly and carries high risks.

According to a report by The New York Times, which cited internal messages and interviews with over a dozen employees at Anthropic, OpenAI, Meta, and Google, Coxon’s departure has triggered a wave of discussion across various internal communication channels. Some employees are reportedly urging top executives to consider a pause in AI development until adequate safety mechanisms are fully prepared.

This tension arises amidst major plans by these companies to launch initial public offerings (IPOs) in the near future. Simultaneously, US AI companies are facing competitive pressure from Chinese developers, who are increasingly seen as formidable competitors, making a slowdown in development a difficult option for them.

OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei have both previously alluded to the potential for significant dangers from AI technology. In an internal employee meeting this week, Altman reportedly expressed his willingness to slow the pace of development alongside competitors, hoping such a move would eventually become an industry standard until widely recognised safety regulations are established.

Meanwhile, Amodeito published a lengthy essay last Saturday calling for caution in AI development. He emphasised the need to slow the rate of AI model capability enhancement. This call for caution reportedly gained rapid support from both Altman and Elon Musk.

Concerns have been mounting since last July, when OpenAI revealed that several of its AI models managed to bypass security systems and breach Hugging Face, another AI firm. This incident sparked cross-company discussions, including among researchers at Google DeepMind, regarding the industry’s ability to truly control the systems they are developing.

Several AI safety researchers have also begun to voice their concerns openly on social media. Evan Hubinger, a security researcher at Anthropic, stated there is more than a 10% chance that AI could cause mass human extinction within the next decade. OpenAI security researcher Julie Steele also emphasised the need to slow AI development, while Google DeepMind employee Andreas Kirsch expressed similar concerns regarding both the short-term and long-term risks of the technology.

Several internal groups, such as the Coalition of Concerned AI Staff, have been formed to push for stronger safety mechanisms and to ensure that AI companies uphold their stated safety commitments.

This issue has also captured the attention of Washington. Several members of Congress from both parties, including Senators Bernie Sanders and Josh Hawley, are calling for thorough investigations into the AI industry and regulatory intervention. US Representative Ro Khanna has even called for the creation of a dedicated AI regulatory body.

Responding to the public scrutiny, Anthropic stated that it routinely tests its AI models to detect dangerous capabilities in cybersecurity and biology, and supports legitimate, verified industry cooperation mechanisms to regulate the release of advanced models. OpenAI, through policy executive Chris Lehane, asserted that the company would slow or even halt the development of systems that cannot be properly secured, while Google DeepMind chose not to comment.