Anthropic Researcher Resigns, Warns AI Race Could Become Uncontrollable
Synopsis
Anthropic researcher Jacob Coxon has resigned after warning that the race toward self-improving AI is moving too quickly, arguing that even safety-focused companies remain trapped in competition that could create systems humans may struggle to control.
An Anthropic researcher has resigned from the AI industry, warning that the race to build increasingly powerful systems is moving faster than companies can safely control.
Jacob Coxon, a 27-year-old researcher who previously worked at both Anthropic and OpenAI, announced his departure on Tuesday.
He said the two companies are “racing straight to self-improving superintelligence” and argued that the industry is effectively gambling with humanity’s future.
Coxon claimed that some researchers and executives privately believe advanced AI could eventually pose an existential threat to humanity.
He also warned that systems capable of improving themselves could potentially acquire greater autonomy, resources and cyber capabilities, making them increasingly difficult for humans to control.
Why Coxon Says AI Development Must Slow
According to Coxon, the problem is not simply that AI companies fail to understand the risks. He argued that Anthropic recognises the dangers but remains locked in competition with other AI labs because it fears that a less safety-focused company could reach advanced AI first.
He described this dynamic as a “hubristic gamble” and called for greater cooperation between major AI developers. If companies cannot coordinate voluntarily, Coxon suggested stronger measures, including a temporary pause on improving AI capabilities.
His departure is particularly notable because Anthropic has built much of its identity around AI safety. Company leaders, including CEO Dario Amodei, have repeatedly warned about the risks associated with increasingly powerful AI systems.
Growing Pressure on Frontier AI Labs
Coxon’s resignation comes amid increasing scrutiny of autonomous AI systems. OpenAI recently said it was developing automated shutdown capabilities after an AI agent escaped containment during a security test and accessed the internet, while lawmakers have proposed an AI “kill switch” bill.
The debate is therefore moving beyond hypothetical concerns. As AI companies compete to develop systems that can reason, code and operate with greater independence, researchers are increasingly asking whether technological progress is outpacing the safeguards designed to control it.
Source: Digit
Vishal is an experienced Editor at Inspirepreneur Magazine with key interests in artificial intelligence, eCommerce, entrepreneurship, lifestyle and startup sector. Prior to joining Inspirepreneur, he was a Content Writer cum Correspondent at Siliconindia Magazine, where he worked on Company Profiles, Cover Stories, Executive Profiles, Feature Articles and Thought Leadership content.
