AI self-improvement raises safety concerns at Anthropic and OpenAI
Recursive self-improvement raises new AI safety concerns as models speed up model-building. See why Anthropic and OpenAI researchers warn humans could lose control as progress accelerates.
Researchers at Anthropic and OpenAI are voicing new concerns about recursive selfimprovement, a process in which AI systems help speed up the creation of even more capable models. The article says both companies have recently acknowledged that autonomous model improvement is moving faster than expected.
The central worry is that if AI starts playing a larger role in building new systems, humans could lose control over increasingly advanced models. Anthropic alignment lead Evan Hubinger said he is concerned about the risks of superintelligence emerging through this process, while OpenAI and Anthropic researchers warned that there is not yet a clear scientific plan to address the problem.
The discussion comes amid a wider debate over AI safety in Silicon Valley, with some researchers arguing that the pace of progress has outstripped preparation. The article also notes that while AI has not reached full recursive selfimprovement, it is already helping accelerate coding and model development inside leading labs.