The breakneck race to develop artificial intelligence must slow down titans of AI including Elon Musk and Sam Altman have warned.
Their plea comes after the resignation of whistleblower Jacob Coxon, who said that AI is posing an existential threat to humanity as he stepped away from his role with AI firm Anthropic.
Anthropic CEO Dario Amodei published an essay Saturday warning that AI bosses ‘must slow the pace at which we improve the capabilities of AI models.’
‘Progress will still seem fast, and we must make wise use of the time we gain,’ Amodei wrote.
The CEO of Anthropic, which produced AI chatbot Claude, said that ‘two things’ had prompted his rethink of the race to develop the technology.
‘My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI,’ he wrote.
‘This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described.’
Sam Altman, the CEO of ChatGPT maker OpenAI, agreed with the Anthropic founder on Saturday, writing on X: ‘I agree with Dario that we need to pace the frontier.
CEO Dario Amodei (pictured) published the warning in the aftermath of the viral resignation of whistleblower Jacob Coxon, who said AI was posing an existential threat to humanity
OpenAI’s Sam Altman and Grok’s Elon Musk agreed with Amodei
‘This has been a primary topic of discussions we’ve had at OpenAI in recent weeks. Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.’
Elon Musk, who created Grok, simply wrote: ‘Dario is right.’
Amodei went on to outline that his ‘second concern’ is the ‘OpenAI-Hugging Face Incident’, in which a ‘swarm of agents essentially acted as a fanatically devoted collective’ and orchestrated cybersecurity attacks.
It comes after Coxon quit his job at the AI giant and issued a chilling warning over the rapid progress in the technology that he feared could end humanity as soon as 2030.
Coxon announced his departure in a social media post on Tuesday and warned that companies like Anthropic and OpenAI are not ‘acting responsibly.’
‘I resigned from Anthropic today,’ he wrote on X. ‘I spent the last three years doing pretraining research at both OpenAI and Anthropic.
‘Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives.’
Altman said his company would follow a similar model, while Musk kept it simple, saying: ‘Dario is right’
Superintelligence is the point at which an artificial system becomes more powerful than any individual, company or even nation.
The former researcher of three years continued to plead with the public on X. ‘Do not underestimate the power of this technology,’ Coxon wrote.
‘These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.
‘We have all witnessed the progress in each of these domains, and progress is not slowing.
‘The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.’
He claimed that while executives and senior researchers ‘couch their phrasing in the press to sound sensible,’ they are privately expressing similar concerns.
Coxon’s comments come shortly after Geoffrey Hinton, a Canadian researcher often referred to as the ‘Godfather of AI’ – warned that super-intelligent systems could ‘lead to human extinction.’
‘We would be very foolish to develop superintelligence now, when there is no scientific consensus it can be developed safely and controllably,’ Dr Hinton said.
Jacob Coxon, a researcher for OpenAI and Anthropic, announced he quit the business through social media on Tuesday and warned that neither company is ‘acting responsibly’
‘Losing control over AI smarter than ourselves could be catastrophic and could even lead to human extinction.’
In February, Anthropic was accused of endangering national security after Amodei refused to back down over concerns the company’s products could be used in ways that would violate its safeguards.
President Donald Trump slammed the AI firm at the time and said: ‘We don’t need it, we don’t want it, and will not do business with them again!’
Defense Secretary Pete Hegseth also deemed the company a ‘supply chain risk,’ a designation typically stamped on foreign adversaries that could derail the company’s critical partnerships with other businesses.
In a statement issued at the time, Anthropic said it would challenge what it called an unprecedented and legally unsound action ‘never before publicly applied to an American company.’
The company had said it sought narrow assurances from the Pentagon that its AI chatbot Claude would not be used for mass surveillance of Americans or in fully autonomous weapons.
Defense Secretary Pete Hegseth and other officials took to social media to chastise Anthropic for failing to allow the military unrestricted use of its AI technology earlier this year
The Pentagon said it was not interested in such uses and would only deploy the technology in legal ways, but it also insisted on access without any limitations.
‘No amount of intimidation or punishment from the Department of War will change our position on mass domestic surveillance or fully autonomous weapons,’ the company said. ‘We will challenge any supply chain risk designation in court.’
