AI models break spatial rules in GeoSpatialBench tests, authors report

Answering a spatial question and making a valid spatial edit proved different challenges in GeoSpatialBench, according to a preprint submitted to arXiv on 9 October 2026. Its authors report that models performed worse on editing tasks involving ordering relations than on question answering. Most unsuccessful edits violated at least one specified spatial constraint.

The input format also mattered. In the authors’ evaluation of multimodal large language models, structured GeoJSON consistently produced better results than map images. Combining the two offered no reliable improvement over GeoJSON alone. The team also reports a split between types of reasoning: models showed reasonable competence on qualitative topological and ordering relations, but weaker performance on quantitative metric relations.

Developed by Zheng Ren and six co-authors, GeoSpatialBench moves from understanding spatial relationships through geometric reasoning to editing configurations under explicit constraints. The team says it draws on urban data from six cities and covers 28 relations across four categories: topology, direction, distance and ordering. Reference answers and constraints are checked using geographic information system operations, according to the abstract.

The distinction the benchmark probes is useful: identifying a relationship does not necessarily demonstrate an ability to change it while preserving other rules. But the captured evidence is limited to the abstract page, which names no tested models and supplies no numerical scores or detailed evaluation procedures. It supports reporting the authors’ findings—not judging how large the performance gaps are or which systems handle these tasks best.