New preprint proposes "agent plasticity" to measure how well AI agents learn from experience

A preprint posted to arXiv on 6 October 2026 argues that tests of AI agents miss something important: whether an agent gets better with experience. The authors, led by Harman Singh with ten co-authors, say existing evaluations mostly measure what an agent can do at a single moment, not how effectively it learns.

The paper's central proposal is a metric it calls agent plasticity, defined as the efficiency with which an agent converts experience into gains in future performance on held-out tasks. The authors frame evaluation around three questions: whether later performance improves and generalizes beyond the interactions that enabled learning, how efficiently new capabilities are acquired, and where the improvement process breaks down.

To test this, the researchers use a controlled setup in which agents distill past experience into reusable artifacts that later instances of the agent inherit. At each checkpoint they measure performance on both training and held-out interactions, while accounting for the cost of learning.

The authors report that, across multiple environments, frontier models followed sharply different improvement paths even with comparable chances to learn. Some made substantial, lasting gains; others stayed near or below where they started. Gains inside the training regime often carried over only partly to out-of-distribution conditions, and the model that ends up performing best is not necessarily the one that improves most efficiently.

The paper also traces failures through the improvement loop. According to the authors, low-plasticity agents often fail to reuse relevant artifacts, while more plastic agents can still fail even when they do reuse them, which the authors say points to limits in artifact quality, generalization or application.

This account rests on a single source: the paper's abstract page on arXiv. We did not read the full text, so the abstract does not tell us which models were tested, which environments were used, or how large the differences were. The paper is a preprint and has not been peer reviewed, and we found no independent reporting or replication of its results. The findings and the bottleneck explanations are the authors' own.

Many teams are building agents that record lessons and reuse them. If the paper's argument holds, a single benchmark score would hide differences in how quickly and reliably such agents improve, and a strong final score could come from an inefficient learner. The proposal gives evaluators a way to ask that question directly, though how well the metric works in practice remains to be shown by the full results and by other groups.

Revised 16 October 2026: Guth re-read the original sources and rewrote this article. The earlier version remains available as revision 1.