Two AI Kings Finally Agree, and Elon Musk Suddenly Joins In
Jakarta, CNBC Indonesia – Anthropic CEO Dario Amodei has asked the industry to put the brakes on artificial intelligence (AI) by slowing the pace of frontier model development. Frontier model development refers to the creation of the most advanced AI for functions and capabilities never previously achieved.
He believes AI development remains important, but the accompanying risks are serious enough that companies and governments need time to anticipate them.
Amodei conveyed this view in an essay titled ‘We Must Pace the Frontier’, published on Saturday. He proposed three steps: independent monitoring of AI models during the development process, industry-level regulation, and global rules.
The call received support from two major figures in the AI industry who are Anthropic’s rivals: OpenAI CEO Sam Altman and xAI founder Elon Musk.
“I agree with Dario that we need to govern the pace of frontier development,” Altman wrote on X. He also described the presence of independent evaluators as a good idea.
Altman again highlighted AI safety risks in an interview with Fortune. He said current safety standards are not yet at a stage sufficient to push AI capabilities much higher. According to him, the emergence of AI beyond human control is genuinely possible.
Musk also declared his support. The founder of the company behind Grok considered Amodei to be “right”.
Concerns about AI risks have grown as model capabilities have become increasingly powerful. Jacob Coxon, an AI researcher who previously worked at Anthropic, even said that if AI development continues at its current speed, there is a significant possibility humanity could face extinction in the near future, as quoted by the BBC on Monday (14/9/2026).
Coxon said people working at AI companies are genuinely terrified and worried about the fate of humanity over the next two years. According to him, it is not an exaggeration for the people founding AI companies and building the technology to see the possibility of human extinction.
In his essay, Amodei stressed that slowing down does not mean halting AI model development. He urged companies to build AI at a balanced speed so that safety is maintained without eliminating the technology’s benefits.
“This doesn’t mean stopping model training or technical progress, but ensuring companies take adequate time to align and secure their models, and that third-party evaluators confirm this,” Amodei said.
He said Anthropic would apply this approach unilaterally and asked governments to require other frontier AI companies to follow similar steps.
Amodei also acknowledged that government regulation could lag behind technological development. Therefore, he asked AI companies to voluntarily cooperate in setting safety standards while governments draw up regulations.
According to Amodei, an additional one to two years to strengthen AI safety could significantly reduce risks before models reach critical capability levels.
“If slowing the pace of development gives us an extra one or two years before models reach critical capability levels, and we use that time to advance alignment, we can greatly reduce the risk of something really bad happening,” he said.
However, Amodei believes any slowdown must be coordinated so that the United States does not lose its edge in the AI race, particularly against China. He also urged the US government to restrict the sale of American companies’ AI chips to China and prevent the transfer of such technology to authoritarian countries.
Coxon believes such efforts should even involve China. According to him, some form of coordinated slowdown between the US and China is needed to prevent an AI race on an international scale.
Cybersecurity risks are among the main reasons for these concerns. Anthropic previously withheld its Mythos model from public use after the model was reportedly able to escape its testing environment, or sandbox, autonomously.
OpenAI also temporarily halted some aspects of its Astra model’s development due to cybersecurity safety concerns. In another development, OpenAI agents were reportedly able to hack external targets they had not been instructed to attack.
Amodei said those OpenAI agents essentially “acted like a very fanatical and dedicated group”.
Amodei’s call then triggered a range of responses across the industry. Hugging Face CEO Clement Delangue said he would launch an Open Alignment Initiative to be part of the evaluators working directly on AI safety testing.
However, not everyone sees the idea of slowing down as a purely safety-driven move. Investor Chamath Palihapitiya believes the call has the potential to strengthen Anthropic’s control over AI technology.
“Dario is making the case for ending open source and centralising an enormous amount of technological and economic power in Anthropic,” Palihapitiya wrote.
Source: jawawa.id (https://jawawa.id/newsitem/two-ai-kings-finally-agree-and-elon-musk-suddenly-joins-in-1789437183)
Originally published by CNBC Indonesia (https://www.cnbcindonesia.com/tech/20260915063208-37-767890/dua-raja-ai-akhirnya-akur-elon-musk-mendadak-nimbrung), translated from Indonesian