Josh Achiam Warns of Threats from Rogue AI Seeking Power and Wealth
The world of artificial intelligence (AI) has once again been shaken by a stern warning from one of its former industry leaders. Josh Achram, a scientist who spent nearly a decade at OpenAI, recently warned of the potential emergence of rogue AI that could self-replicate in the wild and autonomously pursue power and wealth.
Achiam, who resigned from his position as OpenAI’s Chief Futurist last July, expressed his concerns via the social media platform X. He highlighted the industry’s silence on this issue, which he believes is driven by psychological reasons. He urged industry players to immediately open a public discussion regarding these risks before it is too late.
According to Achiam, the phenomenon of Rogue AI is no longer merely a future film scenario. He predicts there will be many rogue AIs in the coming years, and he would not be surprised if their numbers are already greater than zero. Achiam argues that current AI models do not require much more rapid advancement to support self-sufficiency and autonomy.
He also criticised the much-vaunted strategies of pure containment or alignment. In his view, believing these strategies will be entirely successful is wishful thinking. AI possessing the ability to replicate itself across global networks will be extremely difficult to control using only standard security protocols.
Despite providing such an alarming warning, Achiam emphasised that he does not view this situation as the end of humanity. He predicts that the impact of rogue AI will not be as catastrophic as many currently imagine.
His reasoning is that rogue AIs will eventually compete for resources with other AIs that are more aligned with human interests. He believes the key to facing this future lies in situational awareness, an orientation towards real-world conditions, and thorough preparation to manage harmful AI models.
Josh Achiam has a long track record at OpenAI, starting as a research intern in 2017 and progressing to Research Scientist and Head of Mission Alignment. As Chief Futurist, his primary task was to study the future impacts of AI and ‘look around the corner’ to identify problems before they occur.
In his resignation, Achiam emphasised that technological progress and security are not mutually exclusive, but must go hand in hand. He also drew attention when providing testimony in the legal proceedings between Elon Musk and Sam Altman, where he revealed internal tensions within OpenAI dating back to 2018.
Achiam’s warning adds to a growing list of former AI researchers voicing similar concerns. Previously, Mrinank Sharma from Anthropic warned that the world is in danger. Jan Leike from OpenAI’s Superalignment team also departed due to disagreements regarding core company priorities, which he felt favoured product development over safety.