Anthropic Warns About AI Self-Improvement Risks - ForkLog: Cryptocurrencies, AI, Singularity, Future

# Anthropic Warns of AI Self-Improvement Risks

Anthropic team members increasingly delegate most of the development of new models to AI systems. The company sees this as a sign of approaching recursive self-improvement.

According to internal data, over 80% of the code for the company's current products was written by Claude. At the same time, in the second quarter, the amount of code per engineer increased eightfold compared to 2024.

Source: Anthropic Institute. Head of Anthropic Institute Marina Favaro and co-founder Jack Clark wrote that with sufficient computational resources, the trend could lead to a system capable of "completely autonomously designing and developing its successor."

"We have not yet reached the point of no return, and recursive self-improvement is not inevitable. But it could happen sooner than most institutions are prepared for," — experts emphasized.

Benchmarks and Metrics

In April, Claude completed over 800 fixes — according to the supervising engineer, it would have taken a human four years.

On open tasks, Claude’s success rate increased to 76% in May 2026 — a 50 percentage point increase over six months.

Source: Anthropic Institute. Anthropic stated that the duration of tasks that AI can reliably perform independently doubles approximately every four months (compared to the previous seven).

In the task of accelerating training for a small AI model, Claude Opus 4 in May 2025 provided an average speed increase of about three times, and Mythos Preview in April 2026 — about 52 times.

Source: Anthropic Institute. During internal tests, the Mythos Preview model demonstrated the ability to solve research problems in AI safety. Over 800 hours of work, a group of agents closed 97% of the problematic gap in the experiment, while two human researchers managed only 23% of the volume in a week.

New Bottlenecks

Despite successes in coding, humans still hold an advantage in "research judgment" and setting strategic goals.

Anthropic believes that in the near future, the role of developers will shift from writing lines of code to in-depth review of neural network results. Human verification may become the main bottleneck in the speed of developing new models.

The company also acknowledged that it might be beneficial for the world to have the ability to slow down or temporarily halt the development of advanced AI systems so that societal institutions and alignment research can keep pace with progress.

Meanwhile, startup representatives warned that unilateral slowing down could backfire on those who delay — less cautious players might reduce the gap. Without a global coordination mechanism, safety decisions will have to be made under competitive and geopolitical pressure.

Recall that in May, Anthropic published its first report on Project Glasswing — a vulnerability detection program using the Claude Mythos model.

In the same month, the company released Claude Opus 4.8 and separately introduced a dynamic workflows feature for Claude Code.