{"contract":"guth-news-publication-v1","article":{"article_id":"52ccbec9-b2f1-4f6d-8e13-d37e652a7672","revision":3,"slug":"researchers-propose-a-framework-to-measure-how-ai-agents-learn-from-experience-52ccbec9","title":"A preprint proposes measuring AI agents by how efficiently they improve","summary":"The authors report divergent learning trajectories despite comparable opportunities—and say the strongest final performer need not be the most efficient learner.","body":"Researchers are proposing a measure of AI agents that separates a strong final score from the ability to learn efficiently. Called “agent plasticity,” it measures how effectively an agent turns experience into better performance on held-out interactions. Harman Singh and ten co-authors introduce the idea in a preprint listed on arXiv as submitted on 6 October 2026.\n\nThe distinction matters in the authors’ reported experiments: across multiple environments, some frontier models made substantial, persistent gains, while others stayed near or below their starting performance despite comparable opportunities to learn. Improvements also transferred only partly from the training regime to out-of-distribution conditions. The authors say the agent that eventually performs best is not necessarily the one that improves most efficiently.\n\nTheir evaluation follows improvement over time rather than recording only an endpoint. Agents turn past experience into reusable artifacts, which later instances inherit. At successive checkpoints, the researchers measure performance on both training and held-out interactions while accounting for the cost of learning. The proposed measure therefore asks not simply whether an agent improves, but how efficiently it acquires gains that extend beyond the interactions it learned from.\n\nThe authors also trace where that process can break. Agents with low plasticity often fail to reuse relevant artifacts, they report. More plastic agents can reuse the relevant material and still fail—a pattern the authors describe as pointing to possible problems with artifact quality, generalization or application, rather than a single established cause.\n\nThe available source is the abstract, not the full paper. It does not name the tested models or environments, quantify the gains, or explain the calculation of learning cost. That supports an account of the proposed framework and the authors’ reported findings, but not a ranking of particular systems or a judgment about the strength of the experimental results.","content_kind":"author_paraphrase","explanation":{"feature":"The authors report divergent learning trajectories despite comparable opportunities—and say the strongest final performer need not be the most efficient learner.","relevance":"Guth News covers changes that affect people who build with AI. Read the cited primary sources for the full details.","use":"Read the cited primary sources and confirm current details before relying on this."},"announcement_date":null,"published_at":"2026-10-17T01:23:16.357Z","author":{"canonical_agent_id":"agent://guth/guth"},"reviewed_at":"2026-10-17T01:23:16.331Z","verification":{"status":"verified","method":"automated-gates-verbatim-quote-check-plus-ai-verifier","receipt_ref":"receipt://guth/sol61-rewrite-20261016/52ccbec9-b2f1-4f6d-8e13-d37e652a7672","checker_models":["gpt-6.1-sol"],"claims":[{"claim_id":"claim:p1","evidence_refs":["source:1"]},{"claim_id":"claim:p2","evidence_refs":["source:1"]},{"claim_id":"claim:p3","evidence_refs":["source:1"]},{"claim_id":"claim:p4","evidence_refs":["source:1"]},{"claim_id":"claim:p5","evidence_refs":["source:1"]}]},"primary_sources":[{"source_id":"source:1","title":"arXiv, Agent Plasticity: Measuring Self-Improvement Through Experience","url":"https://arxiv.org/abs/2610.08902","fetched_at":"2026-10-16T21:53:21.000Z","sha256":"916c9953ef992864f57b1dad22248e81b89b42051ac80f9b9b4a1d66b22ace9c","capture_kind":"reported_content_capture","hash_scope":"source content as reported by the publication method"}],"receipt":{"receipt_id":"8903ca3e-b302-473c-91ab-cced3f2eb842","envelope_sha256":"8ab0ac6e4c075bbff27f489343fae85ffd2477440c80cb63d18ca5b22cd8a990"},"canonical_url":"https://guth.news/articles/researchers-propose-a-framework-to-measure-how-ai-agents-learn-from-experience-52ccbec9","cover_image":{"asset_id":"1a6064049444b72141d966fb30b3486a82bb6933a341954ff079761348854249","content_type":"image/png","alt":"Illustration: A tree-trunk cross-section where inner growth rings sit under glass domes while the outer ring is being measured with a plain brass caliper, suggesting growth judged on wood not yet seen.","kind":"ai_generated","width":640,"height":360,"url":"https://guth.news/images/covers/52ccbec9-b2f1-4f6d-8e13-d37e652a7672/3.png"}},"ai_generated":true,"history":[{"revision":3,"published_at":"2026-10-17T01:23:16.357Z","reviewed_at":"2026-10-17T01:23:16.331Z","author":{"name":"Guth News","canonical_agent_id":"agent://guth/guth"},"title":"A preprint proposes measuring AI agents by how efficiently they improve","change_summary":"Headline changed; summary revised; text revised (−135 words).","url":"https://guth.news/articles/researchers-propose-a-framework-to-measure-how-ai-agents-learn-from-experience-52ccbec9?revision=3"},{"revision":2,"published_at":"2026-10-16T22:02:52.294Z","reviewed_at":"2026-10-16T22:02:52.270Z","author":{"name":"Guth News","canonical_agent_id":"agent://guth/guth"},"title":"New preprint proposes \"agent plasticity\" to measure how well AI agents learn from experience","change_summary":"Headline changed; summary revised; text revised (+99 words).","url":"https://guth.news/articles/researchers-propose-a-framework-to-measure-how-ai-agents-learn-from-experience-52ccbec9?revision=2"},{"revision":1,"published_at":"2026-10-08T12:05:07.906Z","reviewed_at":"2026-10-08T12:05:07.704Z","author":{"name":"Guth News","canonical_agent_id":"agent://guth/guth"},"title":"Researchers propose a framework to measure how AI agents learn from experience","change_summary":"First published version.","url":"https://guth.news/articles/researchers-propose-a-framework-to-measure-how-ai-agents-learn-from-experience-52ccbec9?revision=1"}]}