Preprint says Station agents rediscovered 62.7% of criteria from three ICLR papers on average

A preprint posted to arXiv on 6 October 2026 by Wenyu Du and Stephen Chung reports that a multi-agent research environment called Station rediscovered, on average, 62.7% of the criteria derived from the findings of three recent ICLR oral papers when it was given only each paper's research question.

To limit leakage of the answers, the authors say agents received only the main question studied in each paper, the papers' results were withheld, and web access was disabled. The authors then split each paper's original findings into separate criteria and counted how many the agents rediscovered. By the authors' account, Station averaged 62.7%, compared with 15.4% for Codex Multiagent-v2 and a range of 14.4% to 20.6% for AI Scientist-v2.

What the authors say is new is a pair of additions to Station: a Supervisor mechanism and periodic Meta Reflection. Both are meant to keep agents exploring when no intermediate metric shows whether they are making progress. The authors report that ablation and behavioral analyses indicate the two mechanisms together improve research coverage and continuity. They also ran Station on two open-ended tasks that have no reference paper, and say some of the agents' discoveries closely match discoveries researchers reported after the system's knowledge cutoff date.

What is confirmed is limited to what the authors report. This article is based only on the arXiv abstract page; the full paper was not reviewed, and a search found no independent coverage or replication of this study. The listing describes the work as a preprint, so it has not been shown to have passed peer review.

Several details remain unknown from the abstract. It does not name the three papers, explain how criteria were chosen or graded, say how many runs were averaged, or describe how the baseline systems were configured. A score of 62.7% of criteria is also a measure of coverage against a defined list, not proof that the agents reproduced the underlying papers in full. How closely the post-cutoff matches resemble the researchers' work is the authors' own judgment.

The authors frame the work against recent AI systems that have progressed quickly when given well-defined metrics, while their ability to handle open-ended discovery remains unclear. The study asks a harder question: whether agents can sustain useful exploration when nothing tells them they are getting warmer. If the full paper bears out the abstract, the environment and the mechanisms for sustained exploration would be worth treating as design variables in their own right. Until others test it, the figures remain the authors' claims.

Revised 16 October 2026: Guth re-read the original sources and rewrote this article. The earlier version remains available as revision 1.