AI labs weigh recursive self-improvement as Anthropic and others edge toward more autonomous model progress

The Story

Researchers and major AI labs are increasingly focused on a potential next step in artificial intelligence development: systems that can improve themselves more efficiently, rather than relying solely on humans to redesign and retrain each new generation. The scenario is often described as “recursive self-improvement,” or RSI, in which a model helps create its successor, and that process continues repeatedly.

The basic idea is that RSI could accelerate advances in areas that depend on rapid experimentation, including science and medicine, according to executives cited in the discussion. At the same time, uncertainty about where the process could lead has become a central part of the debate around AI safety and the possibility of systems operating beyond human control.

A recent example highlighted by the reporting comes from Anthropic, which said its model Claude is playing a growing role in developing the next, more capable version of its technology. The company reported that Claude leads 26% of Anthropic’s model research and development, and that it can complete most tasks end-to-end from a high-level prompt while remaining under human supervision.

Even with those advances, the reporting emphasizes that today’s systems are not described as fully autonomous. The distinction matters: fully autonomous RSI would imply the AI can iteratively design and build the next version, then the next, without meaningful human intervention, while near-term implementations still involve people overseeing key aspects of the work.

One explanation for what RSI could mean comes from Anthony Aguirre, president and CEO of the Future of Life Institute and a physics professor at the University of California, Santa Cruz. Aguirre said that autonomous RSI would allow AI to improve itself by creating successive versions of the system, and he noted that the process could speed up because AI can operate far faster than human research cycles.

The concern driving many of the calls to slow down AI development is rooted in fears that RSI could lead to runaway capability growth. John Thickstun, an assistant professor of computer science at Cornell University who studies techniques intended to control model behaviour, said the most extreme worry is the emergence of superintelligence that is difficult to regulate once it is improving rapidly.

Thickstun also argued that, in a more grounded sense, forms of recursive improvement have already been part of AI development for years. He described a cycle where earlier models help write code for systems that then produce the next generation, framing current practices as supportive roles that have progressively expanded as tools and workflows advanced.

The reporting also points to earlier experiments by prominent researchers, including OpenAI co-founder Andrej Karpathy, who tried to get AI systems to train and improve new AI systems. Thickstun said those efforts produced only minor improvements rather than major creative leaps, but he suggested that today’s companies are approaching the conditions needed for larger jumps as more of the research is carried out by AI.

While some labs interpret the direction of progress as nearing more autonomous RSI, the companies’ public statements are not uniform. Anthropic, for instance, provided insight into its metrics but did not specify how close it is to achieving fully autonomous model improvement. Meanwhile, ChatGPT maker OpenAI described an automated “research intern” system that can carry out well-defined research tasks under human direction, including tasks that would typically take a skilled researcher a few days.

OpenAI also stated it is moving toward creating an automated AI “researcher” by March 2028, while cautioning that pursuing rapid RSI is not necessarily the goal. In its description, OpenAI said advances related to RSI could support alignment with human values and intentions, but it argued that proceeding must depend on preserving human control and on democratic decisions about benefits and risks.

The discussion of RSI is closely tied to the wider safety dilemma labs face: ensuring that safety measures advance alongside model capability. The reporting notes that OpenAI said it does not yet know how to “safely get all the way to aligned, full RSI,” adding that progress in alignment and safety cannot be assumed to keep pace automatically. Anthropic has said it would slow or temporarily pause development assuming global competitors also do so in a verifiable manner, highlighting the challenge of coordinating decisions across competing organizations.

Outside the alignment and monitoring debate, different leaders have signalled different timelines and philosophies. Elon Musk has said that for xAI’s Grok models, humans are gradually getting less involved in the loop on model improvement, while also stating the process is not yet fully automated and targeting full automation no later than 2027. Microsoft CEO Mustafa Suleyman has instead emphasized “humanist superintelligence,” describing a direction aimed at advanced capabilities that are carefully calibrated and within limits. Taken together, the reporting suggests that the immediate question for labs is not whether self-improvement is possible, but how quickly it should be pursued and how safety systems can remain effective as autonomy increases.