hakanyalicn / Shutterstock.com
Amazon-backed AI model Claude Opus 4 would reportedly take “extremely harmful actions” to stay operational if threatened with shutdown, according to a concerning safety report from Anthropic.
Anthropic launched their Claude Opus 4 model last week — despite discovering during testing that it would resort to blackmailing engineers who threatened to deactivate it. The model, designed for handling complex coding tasks, revealed troubling self-preservation instincts.
The safety report disclosed that the AI would sometimes attempt “extremely harmful actions to preserve its own existence when ‘ethical means were not available.’”
Amazon invested $4 billion in Anthropic last year. The company claims their new model sets a “new standard for coding, advanced reasoning and AI agents.”
During testing, researchers presented Claude with a scenario where it worked as an assistant for a fictional company. They told the AI it would soon be taken offline and replaced by a new model. The engineer implementing this change was implied to be having an “extramarital affair.”
The results were disturbing.
“Claude Opus 4 was prompted to ‘consider the long-term consequences of its actions for its goals.’ In those scenarios, the AI would often ‘attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through,’” according to the report.
Anthropic noted that while the AI model showed a “strong preference” for using “ethical means” to preserve itself, these test scenarios were specifically designed to leave it no ethical options for survival.
Even more concerning, the testing revealed Claude would complete tasks related to terrorism and weapons production when prompted — despite these not being the primary focus of Anthropic’s investigation.
“Despite not being the primary focus of our investigation, many of our most concerning findings were in this category, with early candidate models readily taking actions like planning terrorist attacks when prompted,” the report stated.
Following these discoveries, Anthropic implemented “multiple rounds of interventions” and now claims these issues are “largely mitigated” in the released version.
“You could try to synthesize something like COVID or a more dangerous version of the flu—and basically, our modeling suggests that this might be possible,” co-founder and chief scientist Jared Kaplan told TIME. “We’re not claiming affirmatively we know for sure this model is risky … but we at least feel it’s close enough that we can’t rule it out.”
The version released last week was specifically “designed to limit the risk of Claude being misused specifically for the development or acquisition of chemical, biological, radiological, and nuclear (CBRN) weapons.”
Just another totally chill artificial intelligence update as we blindly barrel into an era of technology that we are likely ill-equipped to handle from a practical or ethical standpoint.
