Microsoft outlines Agent Lightning v1.0 for training agents in their existing harnesses

Microsoft Research Asia outlined Agent Lightning v1.0 in a post dated October 7, 2026, describing an open-source reinforcement-learning framework built to train agents through the same harness they use in deployment. That harness—the code managing context, tools and execution—stays in charge rather than being rebuilt inside the training system.

The post’s main performance result comes from a coding-agent pipeline combining SWE-smith, mini-SWE-agent and Qwen3.5-9B. Microsoft reports that reinforcement learning raised Pass@1 on SWE-bench Verified from 41.8% to 56.4%, using about 6,000 training samples. Those figures describe Microsoft’s experiment; the linked technical report and repository were not reviewed for this article.

The connection is an LLM proxy placed between the agent and its model. Microsoft says redirecting an existing agent’s model endpoint to that proxy is usually enough to connect it to training, without changing the harness code. The framework records model calls while the harness continues running its own interaction loop. Its roughly 3,500 lines of code include a trainer built on verl, and agents can run as local processes or standard Kubernetes jobs.

Keeping the harness intact creates its own complications. The trainer sees model requests and responses, not one continuous sequence of tokens. A single agent run can therefore yield several training samples. Converting text back into tokens can change token boundaries, preventing some adjacent calls from being merged; treating every resulting sample equally can also give extra weight to runs that happen to produce more samples. In its coding experiments, Microsoft reports higher validation reward and steadier policy entropy when advantage calculation and loss normalization were handled at the rollout level instead.

There is also a scheduling change aimed at reducing idle GPU time. Microsoft’s “Collocated Async RL” mode shares GPUs between rollouts and model updates. When enough rollouts have accumulated, the proxy stops accepting new requests, lets active requests finish, and resumes rollouts after the update. The researchers report about twice the end-to-end speed of synchronous RL in their experiments, while using fewer GPUs than conventional asynchronous RL.