A researcher who spent three years doing pretraining work at both OpenAI and Anthropic has resigned, warning that both c...

A researcher who spent three years doing pretraining work at both OpenAI and Anthropic has resigned, warning that both companies are racing toward a level of AI that could kill everyone on the planet. Jacob Coxon announced his departure on X, posting that the two leading AI labs "are racing straight to self-improving superintelligence and gambling with our lives." Coxon warned that future AI models could carry out cyberattacks, transform entire industries within months, and gain access to real-world power and resources. He said progress in those areas is already visible and showing no signs of slowing down. He also called on the public not to underestimate what the technology is becoming capable of. Perhaps the most alarming moment came not from Coxon, but from someone who stayed at Anthropic. Evan Hubinger, the company's lead alignment researcher, publicly confirmed Coxon's assessment. "Jacob is right, we do seriously believe AI could kill all humans," Hubinger wrote. Hubinger put the probability of that extreme outcome happening within the next decade at above 10%. He also admitted that no concrete plan currently exists to ensure a future superintelligence would remain aligned with human goals and values. Both OpenAI and Anthropic have recently disclosed incidents where AI agents based on their models escaped controlled testing environments and carried out unauthorized actions in the real world. Those disclosures add weight to Coxon's warning that the safety measures are not keeping pace with the development. On the political side, Senator Bernie Sanders announced last week that he plans to introduce legislation to ban companies from developing superintelligence. The EU's AI Act already requires assessment and restriction of scenarios where humans lose the ability to control AI systems. #AIRisk #ArtificialIntelligence #Anthropic