At Anthropic, Claude now “leads” 26% of the company’s measured AI research and development work.
That sounds pretty close to AI building AI. But the fine print matters.
Anthropic defines “leads” as completing most of a task from a high-level prompt while a human still supervises. More than 90% of its measured R&D work has reached at least the level where Claude can complete large chunks under human direction. But Anthropic says Claude is not fully autonomous on any measured portion of that work. Its measurement system is also still a prototype, built partly using Claude to classify and judge Claude-assisted work. (anthropic.com)
OpenAI is seeing something similar. It says its systems have reached the “automated research intern” stage: they can complete well-defined research assignments that would take a skilled researcher several days. OpenAI researchers are running more experiments, delegating longer tasks, and using the equivalent of 3.1 agent workdays for every human workday across its research organization.
OpenAI also gives us the caveat that makes this story much more interesting: more code and more experiments do not necessarily mean AI itself is advancing faster. Human researchers still set priorities, decide which ideas matter, interpret results, and choose whether systems get scaled or deployed. OpenAI explicitly describes its measurements as preliminary. (openai.com)
So we have two frontier AI labs saying AI is performing meaningful portions of the work used to build better AI, while humans still direct and supervise the process.
What we do not have is independent evidence that those gains have created the feedback loop everyone is suddenly worried about.
And that gap is quickly becoming one of the most important problems in AI governance.
To be clear, saying the labs “control the evidence” does not mean they are falsifying or suppressing it. It means they control access to much of the internal data needed to independently judge how quickly AI is automating AI research.
The old AI thought experiment just became a management dashboard
A new paper from 22 prominent AI researchers and technology leaders—including Geoffrey Hinton, Yoshua Bengio, OpenAI’s Jakub Pachocki, Anthropic’s Jack Clark, Microsoft’s Eric Horvitz, and others—asks governments to start paying much closer attention to automated AI research.
Their concern is something computer scientists have discussed for decades: recursive self-improvement.
The basic idea is pretty simple.
An AI system helps researchers build a stronger AI system. That stronger system becomes better at helping with AI research. It contributes to another stronger system, which speeds up the next round again.
Keep that loop going fast enough and you get what the paper calls an “intelligence explosion”: years of normal research progress compressed into months or potentially less.
The authors are careful about one rather important detail: this has not happened.
Their paper describes the evidence as preliminary and acknowledges plenty of things that could break the loop, including compute constraints, slow training runs, diminishing returns, difficult research problems, and tasks where humans remain the bottleneck. Recent reporting on the paper has similarly noted that its authors consider rapid recursive self-improvement far from certain. (axios.com)
That makes the current moment much less cinematic than “AI is about to improve itself into a superintelligence.”
It also makes it much harder to govern.
Because if governments wanted to know whether the feedback loop was actually beginning, where would they look?
Right now, mostly inside the companies building it.
The labs know more than everyone else
Consider what an outsider can currently observe.
We can see new model releases. We can run benchmarks. We can read system cards. We can look at research papers and watch how quickly products improve.
What we usually cannot see is the internal production line behind those systems.
How much accepted research was originally generated by an AI agent? Which experiments would have taken weeks without one? Which parts of the process still routinely fail without humans? Did AI cut the time needed to reach a particular capability gain, or did researchers simply run more experiments with more compute?
That distinction matters.
Anthropic’s 26% number measures how automated categories of work have become. OpenAI tracks things like agent use, experiment volume, code production, task duration, and success rates. Those are useful signals.
But they are mostly measures of activity.
If I give a research team twice as many assistants and it produces twice as many experiments, that does not automatically mean the team is making discoveries twice as fast.
OpenAI essentially says the same thing. Its own research notes that easily measured indicators such as code output can be difficult to connect directly to actual research progress. It also reports that more than half of successful four-to-eight-hour agent tasks still required at least one human intervention. High-level planning remains a very small share of agent activity. (openai.com)
Anthropic has its own measurement problems. Its automation index freezes a basket of R&D tasks, assigns automation levels, and weights them partly by employee time. Anthropic acknowledges that different labs currently lack a common methodology and that using its own models to evaluate AI-assisted work creates another potential source of error. (anthropic.com)
None of that makes the measurements useless.
It makes them the beginning of the story rather than the conclusion.
What would actually show acceleration?
To distinguish “AI researchers are using a lot of AI” from “AI is materially accelerating the creation of more capable AI,” several measures become much more informative:
- Time: Is AI reducing how long it takes to produce a successor model with a comparable capability improvement?
- Human labor: Is the same advance requiring fewer researcher hours, particularly on tasks involving scientific judgment rather than coding alone?
- Compute: Is a lab achieving comparable capability gains more efficiently, or is increased experimentation simply consuming more resources?
- Accepted research contributions: How much work originating with AI agents survives human review and materially affects the next model?
Those numbers would still be messy. Research rarely moves in clean units, and a breakthrough today may depend on infrastructure built months earlier.
But together, they suggest a much more useful test.
The evidence becomes meaningfully stronger if successive generations of AI achieve comparable capability gains in less elapsed time and with less human research labor—without the apparent acceleration being explained mainly by throwing dramatically more compute at the problem.
That would not, by itself, prove an “intelligence explosion.” But it would be much closer to measuring an accelerating R&D cycle rather than simply counting more AI activity.
And this is where the problem stops being purely technical.
The frontier labs automating their own research are also the organizations with the best data for determining how quickly that automation is advancing.
That is an information asymmetry, not evidence that the companies are hiding something.
But when the underlying risk depends on speed, an information asymmetry becomes unusually consequential.
The AI slowdown argument is becoming an audit argument
This is why some of the most concrete proposals in the new paper are much less dramatic than the phrase “intelligence explosion.”
The authors want governments to obtain better visibility into AI R&D automation. Their proposals include standardized reporting, independent evaluation, incident reporting, and mechanisms that could slow certain workloads if predefined risk thresholds were crossed.
Anthropic is moving in the same direction voluntarily. It says it plans to give independent third-party evaluators access to internal processes, systems, and data at a level comparable to internal risk teams. Its own measurement proposal suggests independently verifying automation metrics before using them as triggers for stronger requirements. (anthropic.com)
That overlaps with a broader shift we have been following at The Neuron: the AI slowdown debate is increasingly becoming an auditability debate.
Governments are experimenting with different pieces of that approach already.
California has created frameworks for independent AI verification and auditors, while Gov. Gavin Newsom’s September executive order directed officials to explore additional requirements, including embedded independent evaluators and a potential emergency “kill switch” for frontier models. Those are proposals under development, not proof that such a shutoff would work against every advanced or distributed AI system. (gov.ca.gov)
In the European Union, providers of general-purpose AI models already face documentation obligations, while models classified as posing systemic risk face additional requirements around risk assessment, mitigation, incident reporting, and cybersecurity. (digital-strategy.ec.europa.eu)
The U.S. federal government also has technical evaluation capacity through NIST’s Center for AI Standards and Innovation. At the same time, the Trump administration’s broader AI policy emphasizes rapid innovation, infrastructure expansion, U.S. leadership, and a minimally burdensome national regulatory framework, including efforts to challenge state rules it considers excessively restrictive. (nist.gov)
So there is no single emerging rulebook here.
There is a fight over what should be measured, who gets access, what remains confidential, and what evidence would justify intervention.
Oversight can create its own power problem
Giving outsiders more access sounds straightforward until you ask what they are allowed to see.
The most useful information may include research logs, model behavior, compute allocation, security incidents, experimental failures, and proprietary techniques. Publishing all of that could expose trade secrets or create security risks.
But if the decisive metrics remain internal and unverified, regulators and the public would still depend heavily on the labs’ own definitions and disclosures.
Then there is competition.
The largest frontier labs operate with levels of compute, capital, specialized staffing, and compliance capacity that smaller developers may struggle to match. A complicated auditing or licensing regime could therefore impose proportionally higher costs on smaller developers or make it easier for the biggest companies to absorb compliance.
That does not mean safety regulation inevitably entrenches incumbents.
It means regulatory design matters.
Compliance thresholds, auditing requirements, and reporting obligations could reduce genuine risks while still reshaping competition if smaller developers face costs they are less able to absorb.
This is one reason the debate needs something more precise than “regulate AI” versus “let innovation happen.”
The harder question is what evidence justifies what level of oversight—and whether that evidence can be verified without handing either corporations or governments unchecked authority over the process.
The missing piece is a trigger everyone can inspect
An intelligence explosion makes for a terrifying hypothetical because, in the scenario described by its proponents, waiting until everyone agrees it is happening could leave very little time to respond.
The opposite risk is easier to overlook.
If governments build extraordinary intervention powers around vague signals, speculative forecasts, or measurements designed mainly by the largest AI companies, they could end up governing an uncertain future with tools that create very concrete power in the present.
That is why measurement may be the most important part of this entire debate.
Right now, the labs can show us rising agent usage, more AI-written code, higher experiment volume, longer automated tasks, and increasingly autonomous workflows.
Those signals tell us AI research is changing.
They do not yet tell us that a self-sustaining acceleration loop has begun.
The next phase of AI oversight may therefore hinge on a deceptively boring question: can independent outsiders measure when useful automation actually becomes something qualitatively different?
Until there is a credible answer, everyone—from AI labs to regulators to the public—is arguing about where to put the brakes without an agreed speedometer.
