LLMs miss key decision factors when asked in advance, preprint reports

Asking a language model which questions could change its recommendation may not reveal the questions that matter across its generated answers. An arXiv preprint submitted on Oct. 14, 2026, reports little relationship between models’ advance predictions and the factors researchers identified after the models produced responses. Examining one completed answer helped, but still missed part of the picture.

The paper, Can LLMs Anticipate Their Cruxes?, is by David Reber, James Reber, Ari Holtzman and Victor Veitch. They define a “crux” as a sub-question on which generated responses—called rollouts—disagree, and whose answer could have changed the conclusion. Under that definition, a crux is a feature of the distribution of answers, not something that can necessarily be found in one answer’s explanation.

That makes comparison central to their approach. Rather than relying only on a model’s forecast of what will matter, or extracting factors from a single response, the researchers generate many responses and look for disagreements across them. They report that the cheaper alternatives fall short across the models and domains they evaluated.

To reduce the cost of that comparison, the researchers use a reranker. They estimate that its calls require 86 times less compute than a judge model—a comparison of those components, not a stated 86-fold saving for the entire process.

The authors conclude that many-response analysis is necessary for high-stakes decisions. The abstract, however, supplies no model names, response counts or detailed performance figures. It supports a warning about the limits of the cheaper approaches, but leaves the practical computing budget for any particular application unresolved.