A preprint proposes measuring AI agents by how efficiently they improve

Researchers are proposing a measure of AI agents that separates a strong final score from the ability to learn efficiently. Called “agent plasticity,” it measures how effectively an agent turns experience into better performance on held-out interactions. Harman Singh and ten co-authors introduce the idea in a preprint listed on arXiv as submitted on 6 October 2026.

The distinction matters in the authors’ reported experiments: across multiple environments, some frontier models made substantial, persistent gains, while others stayed near or below their starting performance despite comparable opportunities to learn. Improvements also transferred only partly from the training regime to out-of-distribution conditions. The authors say the agent that eventually performs best is not necessarily the one that improves most efficiently.

Their evaluation follows improvement over time rather than recording only an endpoint. Agents turn past experience into reusable artifacts, which later instances inherit. At successive checkpoints, the researchers measure performance on both training and held-out interactions while accounting for the cost of learning. The proposed measure therefore asks not simply whether an agent improves, but how efficiently it acquires gains that extend beyond the interactions it learned from.

The authors also trace where that process can break. Agents with low plasticity often fail to reuse relevant artifacts, they report. More plastic agents can reuse the relevant material and still fail—a pattern the authors describe as pointing to possible problems with artifact quality, generalization or application, rather than a single established cause.

The available source is the abstract, not the full paper. It does not name the tested models or environments, quantify the gains, or explain the calculation of learning cost. That supports an account of the proposed framework and the authors’ reported findings, but not a ranking of particular systems or a judgment about the strength of the experimental results.