Exa introduces ATLAS to measure what search agents miss

Exa announced ATLAS in a post dated Oct. 8, 2026, targeting a weakness that a tidy list of search results can conceal: missing answers. In the company’s tests, more than 90% of the entities returned by the best search agents were correct, but those agents still missed a third or more of the possible matches.

The benchmark sets 547 “wide search” tasks: find every entity meeting specified conditions, then fill in attributes for each. Exa frames completeness as especially important for work such as assembling sales prospects or researching adverse media about an investment target. A correct result is useful; it does not tell you what was left out.

ATLAS relies on an answer key built by agents rather than human researchers. Two builders—one running on ChatGPT, the other on Claude—search independently, and a separate verifier checks each row against its evidence. Exa then runs research systems against the tasks and investigates disagreements with the key. Disputes that cannot be settled are left ungraded rather than assigned a guessed answer.

That shifts much of the work into constructing the key. Subsequent grading is largely a table comparison, though an LLM still handles fuzzy matching. Exa says grading one system across the entire benchmark costs about 24 cents; that figure covers scoring, not the agent’s cost of performing the searches.

On Exa’s reported results, its Agent Ultra got all attributes for an entity right about two-thirds of the time. Opus 5.5, which the company identified as the best-performing frontier model in its tests, managed a little under 50%. Those figures measure complete entity records, a different question from whether an entity belongs on the list at all.

Exa is both the benchmark’s designer and a seller of search tools and an agent evaluated on it. Its announcement also left an important check unfinished: the company said a blind human audit of sampled tasks was under way. It promised to open-source ATLAS “in a few weeks”—a plan stated in the announcement, not a completed release documented by it.