Ai2 reports shorter debugging queues with GPU-time budgets

Ai2 says a new GPU scheduler brought the 90th-percentile queue wait for short debugging jobs down to 30 seconds, from two hours. In an October 9, 2026 engineering post, the institute described results from a cluster-by-cluster rollout that began at the end of July.

That striking improvement comes with a caveat: Ai2 had fewer debugging jobs in its baseline, making those measurements more variable. These are small jobs requesting a few GPUs and no more than 15 minutes of protected runtime—enough to check whether training code starts successfully or crashes early.

The new system gives managers budgets of GPU time to divide among projects and researchers. A hierarchical fair-share scheduler looks back over seven days by default, favoring allocations that have used less than their share. Jobs declare a minimum runtime, capped at eight hours, during which funded work is protected from interruption. Afterward, the scheduler can reclaim the GPUs and put resumable work back in the queue. Eight hours is a limit on protection, not necessarily on how long a job can run.

That arrangement replaces a system in which declaring high priority carried no cost. Eventually, Ai2 says, every scheduled workload used HIGH priority. Researchers also parked idle jobs on GPUs so they would have somewhere to debug later. Under the budget system, that tactic spends the team's allocation without doing useful work. The pressure was substantial: Ai2 reports outstanding GPU requests at two to three times its available capacity.

Fair-share scheduling itself is established practice. Hadoop's YARN documentation describes hierarchical queues, weighted resource shares and redistribution of unused guaranteed capacity. Ai2 says its change is in the inputs: a hierarchy matching its research organization, with weights set through managers' budgets rather than static quotas.

The broader queue results were less dramatic than the debugging figure. On Ai2's largest H100 cluster, the reported median wait dropped from five minutes to 24 seconds, while the 90th-percentile wait fell from 2.8 hours to 1.8 hours. Substantial waits remained for some work.

Across a 30-day test, Ai2 says teams received 98% of the GPU hours they were owed, with that entitlement capped by actual demand. Cluster occupancy stayed at 98%. Occupancy measures time assigned to workloads; it does not establish how efficiently applications used the GPUs while running.

The interruptions also created a cost for interactive work. Sessions that researchers could previously hold for a week became preemptible after their protected runtime if they exceeded their allocation, sometimes forcing users to rebuild working state by hand. Ai2 was also investigating whether the new protections fragmented available capacity and lengthened waits for its largest jobs—an unresolved trade-off in the reported results.