Anthropic Researcher Quits AI Industry Over Superintelligence Risks

Jacob Coxon said he resigned from Anthropic after three years of pre-training research at OpenAI and Anthropic, and was leaving the AI industry. He argued that the labs were racing toward self-improving superintelligence and “gambling with our lives.”

Anthropic alignment researcher Evan Hubinger said he personally puts the probability of human extinction from AI within the next decade above 10%. He also said Anthropic is trying to act responsibly but does not yet have a plan to solve alignment for superintelligence and is not clearly on track. That estimate drew criticism over its methodology and what evidence could change it.

The debate also raised questions about whether employees at large AI companies have enough authority to change corporate policy from inside. The effectiveness of Coxon’s departure as a way to reduce risk or influence Anthropic remains uncertain.