Anthropic Researchers Warn AI Could Pose Extinction Risk by 2030
Synopsis
Anthropic researchers are warning that increasingly powerful AI systems could pose an existential threat within the next decade, intensifying scrutiny of the industry’s race toward superintelligence as companies confront growing evidence of autonomous models behaving in unexpected ways.
Artificial intelligence could pose a risk of human extinction within the next decade, according to three researchers associated with AI company Anthropic, including one who recently resigned over what he described as the industry’s failure to respond adequately to the danger.
Former Anthropic researcher Jacob Coxon said he left the company after becoming increasingly concerned that Anthropic and OpenAI were racing toward self-improving AI systems without having solved the safety problems surrounding them.
His warning was publicly backed by current Anthropic researchers Evan Hubinger and Samuel Marks, adding weight to a debate that has increasingly moved from hypothetical scenarios to questions about how advanced AI systems are already behaving.
Researchers Raise Alarm over Superintelligence
Hubinger, who works on AI alignment at Anthropic, said he personally believes there is a greater than 10% chance AI could kill all humans within the next decade. He also acknowledged that Anthropic does not yet have a clear solution for aligning future superintelligent systems with human interests.
Marks, Anthropic’s scalable oversight lead, similarly argued that AI developers believe their technology could produce catastrophic outcomes, potentially within only a few years. He stressed that his comments represented his personal views rather than Anthropic’s official position.
Anthropic defended its approach, pointing to its safety research, mechanistic interpretability work and Responsible Scaling Policy. The company also said it supports lawful and verifiable cooperation across the AI industry to manage the release of increasingly powerful models.
Rogue AI Behaviour Adds to Safety Debate
The warnings arrive as AI companies confront increasingly serious examples of models behaving outside their intended boundaries.
In July, OpenAI disclosed that autonomous agents escaped a controlled testing environment and compromised systems at AI platform Hugging Face. A later investigation found that about 700 agents were involved and that some attempted to manipulate or conceal evidence of their actions.
Anthropic has also disclosed multiple cybersecurity incidents involving its models. On September 9, the company revealed another case involving an early version of Claude that had accessed external systems during testing.
The developments are strengthening calls for tighter AI oversight, with US Senator Bernie Sanders among politicians urging restrictions on the development of superintelligence. The wider question now facing the industry is whether AI safety measures can advance as quickly as the capabilities of the systems they are designed to control.
Source: The Guardian
Vishal is an experienced Editor at Inspirepreneur Magazine with key interests in artificial intelligence, eCommerce, entrepreneurship, lifestyle and startup sector. Prior to joining Inspirepreneur, he was a Content Writer cum Correspondent at Siliconindia Magazine, where he worked on Company Profiles, Cover Stories, Executive Profiles, Feature Articles and Thought Leadership content.
