Anthropic warns: AIs could soon create their own successors

Independent programming through artificial intelligence offers enormous opportunities for science. But researchers at Anthropic strongly warn of an impending loss of control. They are therefore calling for an international development pause.

AI programs itself

In software development, artificial intelligence is increasingly taking on tasks that were previously reserved for humans. Language models write, test and optimize code more and more autonomously. This leads to what is known as recursive self-improvement, where systems develop their own successors. What is increasingly worrying experts is that the speed of progress is significantly exceeding current forecasts. Internal evaluations show rapid acceleration. In May 2026, over 80 percent of the integrated code when developing new models came from the AI ​​itself. Developers often only check the suggestions. This increased the amount of code processed daily per employee eightfold compared to 2024.

From helper to developer

Like anthropic in one Blog post writes, the duration of the tasks that models solve without errors increases. While systems in 2024 only managed four minutes of programming work, current versions processed complex problems independently for twelve hours. People increasingly become bottlenecks in the process. The systems now also achieve almost perfect results in standardized tests such as SWE-bench, which measures the resolution of real software errors. The same applies to CORE-Bench, a test for reproducing scientific studies. The models evaluate data and confirm results without human intervention. This offers great opportunities for areas such as medical research and science. Complex problems can be solved much more quickly. At the same time, the risks are growing. When systems train their successors without human intervention, monitoring becomes more difficult. Loss of control over the orientation of the models is a possible scenario.

Demand for an AI break

In order to adapt social structures and security concepts to the pace, the researchers propose a temporary slowdown in global AI development. However, such a stop requires international agreements and strict control mechanisms. This is the only way to prevent individual actors from building up a secret advantage. Implementing such an agreement is technically and politically complex. Model training runs are harder to monitor than physical facilities. Nevertheless, the company is pushing for a dialogue with governments in order to create a timely, verifiable framework for future research.