The tool behind the findings is PlaceBot, an open-source Python workflow developed with the Natural History Museum in London. It sends each locality description individually to a large language model through a standardised prompt, asking the model to extract a geographic hierarchy, estimate coordinates where none are given, expand historical place names, interpret UK Vice County codes, and return everything as structured JSON together with a confidence level and an uncertainty radius in metres. The system is deliberately vendor-agnostic: models are swapped through configuration files rather than code changes, so different artificial intelligences can be compared under identical prompting conditions.

To test the approach, the researchers evaluated twelve models from four providers on two very different datasets drawn from the museum’s collections: 700 locality descriptions from UK bumblebee specimens and 900 verbatim locality strings from globally distributed dragonfly type specimens, all collected between 1839 and 2016. Records that already contained coordinates or grid references were excluded beforehand. Each record was processed once through every model, and the resulting coordinates were compared with manually curated reference coordinates using Haversine distances. The evaluation sets were fixed convenience samples taken in existing file order rather than random samples, a limitation the authors state plainly.

Gemini 2.5 Pro produced the lowest median error in both datasets: 1.61 kilometres for the UK bumblebee records and 5.83 kilometres for the global dragonfly records, each with bootstrap confidence intervals. GPT-4.1 followed at 1.78 kilometres in the UK dataset, while Claude Opus 4 and Gemini 2.5 Flash also performed strongly. But the rankings were not stable across datasets. Gemini 2.5 Flash moved from fourth place in the UK sample to second globally, and GPT-4.1 slipped from second to fourth, showing that model performance does not follow a simple vendor hierarchy. Statistical tests on single runs did favour Gemini 2.5 Pro over the other leading cloud models in both datasets after multiplicity correction, yet when the comparison was repeated across ten calls per record on a deliberately challenging subsample, Gemini 2.5 Pro could not be distinguished from GPT-4.1, Claude Opus 4 or Gemini 2.5 Flash in the UK data after correction.

Return reliability mattered as much as accuracy. Most model-dataset combinations returned usable coordinates for more than 95 percent of attempts, but o4-mini managed usable UK coordinates for only 383 of 700 calls, a failure rate that would cripple any production workflow. The three locally hosted Qwen models, run on a single workstation with a consumer GPU, produced far larger median errors, ranging from about 28 kilometres to nearly 124 kilometres on UK records and up to 610 kilometres on global ones. The authors caution that this comparison does not isolate deployment mode or licence, because the local models were small and the cloud models differed simultaneously in scale, training and serving infrastructure.

Every model showed run-to-run variation, even under controlled prompting with temperature set to 0.1. GPT-4.1 was the most stable, with a median per-record pairwise distance of just 0.30 kilometres in the UK sample and 0.011 kilometres globally, but low medians sometimes coexisted with wide spreads, and no model was perfectly deterministic. The authors argue this non-determinism reinforces the need to treat model-derived coordinates as probabilistic estimates rather than fixed answers, since even a small deviation in latitude or longitude can shift a predicted locality by kilometres.

Perhaps the most striking result came from the ensemble analysis. When the outputs of Gemini 2.5 Pro and GPT-4.1 fell within 2 kilometres of each other, both models were within 5 kilometres of the reference coordinate for 95.3 percent of such UK records and 75.0 percent of such global records. Close agreement between two independent models therefore identifies a higher-reliability subset of records. The authors are careful to stress that this is a post hoc, internally evaluated signal: the pair was selected and evaluated on the same data, two models can converge on the same plausible but wrong interpretation, and agreement does not establish correctness. They propose it as a triage tool for prioritising curator review, not as automatic acceptance.

The study also turned its lens on the reference data itself. For records where both top models placed their estimates more than 50 kilometres from the manual coordinate and outside the manual error radius, an author re-examined the locality descriptions. In the UK dataset, 27 such cases were found, and on reassessment the AI estimates were judged correct in 11, the original manual estimates correct in 9, and 7 remained ambiguous because the locality data were simply insufficient. Among 90 comparable global cases, the AI was right 14 times, the manual georeference 21 times, and 55 cases were ambiguous or unresolvable. Because only conspicuous disagreements were re-examined, the overall prevalence of reference-coordinate error remains unknown, but the finding demonstrates that manual georeferencing is itself error-prone and that some ambiguity is irreducible in historical labels.

Confidence reporting proved only partly useful. Several models did assign higher confidence to more accurate coordinates, but category use varied sharply, some models never used the high-confidence label, and some apparently extreme percentages rested on one or two records. Model-reported uncertainty radii were not uniformly calibrated either: empirical coverage, the proportion of cases where the observed error fell within the model’s own radius, ranged from roughly half for the leading cloud models to below 5 percent for Qwen3 14B, and the radii carried no stated nominal probability. The authors conclude these values should travel with outputs as model-reported metadata, not as verified bounds.

The full retained study cost 264.03 US dollars in commercial API-token charges, with per-thousand-call rates ranging from 17 cents for Gemini 2.5 Flash-Lite to 35.76 dollars for Claude Opus 4. The authors emphasise that the model-specific rankings are a dated snapshot, since model aliases, capabilities and prices change rapidly, and that PlaceBot’s durable contribution is a reusable harness for repeating the evaluation on current models and local records. Their recommendation for deployment is clear: machine-generated coordinates should enter collections databases explicitly marked as unreviewed, carrying machine-readable provenance including the model identifier, prompt version, confidence and uncertainty radius, with any human decision recorded separately. Concordance, confidence and return status should guide curator attention, not replace it, and external evidence from gazetteers, maps or expert review remains necessary before any coordinate is treated as verified.

Subject of Research: AI-assisted georeferencing of museum specimen locality data using large language models

Article Title: PlaceBot: A Python tool for AI-assisted georeferencing of museum specimen locality data, with cross-vendor model evaluation

Article References: Hollister, J. D., & Price, B. W. (2026). PlaceBot: A Python tool for AI-assisted georeferencing of museum specimen locality data, with cross-vendor model evaluation. Natural History Collections and Museomics, 3, Article e203521. https://doi.org/10.3897/nhcm.3.203521

Image Credits: AI Generated

DOI: 10.3897/nhcm.3.203521

Keywords: georeferencing, museum specimens, large language models, biodiversity informatics, Natural History Museum, PlaceBot, Gemini 2.5 Pro, GPT-4.1, locality data, digitisation, coordinate accuracy, ensemble agreement