Scanned Floras and Web Data Fuse Into a 345,000-Species Text Collection for Plant Biodiversity AI

Botany has always been a science of books. For more than a century and a half, the world’s plant species have been described in printed Floras, monographs and taxonomic volumes — dense, telegraphic texts written by generations of botanists, scattered across libraries and archives in dozens of languages. That accumulated knowledge is exactly what modern artificial intelligence needs to extract plant traits at scale, yet it has remained locked in unstructured prose. Now, a team writing in the Biodiversity Data Journal has built one of the largest text collections of plant taxonomic descriptions ever assembled: roughly 390,000 pages, 113 million words and 345,260 species-level files covering 197,000 accepted plant species from every corner of the globe.

The dataset, released under a Creative Commons CC-Zero public domain waiver and deposited on Zenodo, merges two very different worlds. Thirty-five percent of the corpus derives from scanned PDFs — more than 550 monographs spanning 36 regions, most published between 1845 and 2000. Sixty-five percent comes from web scraping, including 122,000 Wikipedia pages, one for each distinct plant species, alongside specialised botanical websites chosen for their deeper local expertise. Each species file averages 290 words of natural language description, and the entire collection occupies just 1.41 GiB of disc space, accompanied by a metadata file compliant with the RO-Crate specification and a per-file quality assessment report.

Why does this matter? Plant taxonomic descriptions are a fundamental source of knowledge for botany, ecology, conservation and environmental science. They encode what botanists call plant traits — morphology, life cycle, habitat, geographic distribution — the raw material behind large trait databases such as TRY and AusTraits. But this information is scattered across printed volumes and rarely structured in a way automated systems can directly use. Large language models have shown promise in extracting morphological traits from unstructured text, achieving F1-scores over 75 percent when replicating manually curated species-trait matrices, yet they still struggle with factual accuracy and hallucination. Training such models — or building more reliable extraction pipelines — demands large, clean, precise textual corpora. That is precisely the gap this data text collection fills.

The technical challenges were formidable. Botanical descriptions vary enormously in style, vocabulary and structure depending on the author, historical period and language — Latin, French, English, Spanish, German, Russian, Chinese. The taxonomic lexicon is highly specialised, often polysemous or subject to synonymy. Descriptions follow implicit conventions, such as the standard morphological order from root to stem to leaf to flower to fruit, with syntactic ellipses, abbreviations and unstated units. Worst of all, botanical prose is telegraphic: verbs are routinely omitted, which confounds standard natural language processing tools built for ordinary syntax.

For the scanned volumes, the team used Tesseract OCR version 5.5.0, an open-source engine combining classical algorithms with long short-term memory neural networks, capable of handling historical and low-quality scans. Each PDF was processed page by page, the individual page texts concatenated into a single OCR file, and that text then segmented into family-, genus- and species-level files. The hierarchy matters: species descriptions are often incomplete in isolation, with characters shared by all species within a genus described at the genus level, and characters common to all genera within a family described at the family level. On the web side, a dedicated scraper was developed for each source, since every website has its own HTML structure and CSS styling. Data collection was rate-limited to avoid overloading infrastructure — a respectful, ethical approach that made the process time-consuming, with scraping time totalling 1,370 hours.

The heart of the pipeline is WordGen, a name-matching algorithm the team developed to identify species names even in texts riddled with OCR and typographical errors. WordGen merges three curated resources — the World Checklist of Vascular Plants, World Flora Online and TRY — into a reference dictionary of over 1.6 million entries, including species, subspecies and official synonyms, compiled exclusively from trusted taxonomic backbones. World Flora Online serves as the primary reference, with all names from WCVP retained alongside their status as accepted names or synonyms. After normalising subspecies and synonyms to their accepted parent species, roughly 1.2 million unique species-level names remain — far more than the accepted nomenclature, because historical documents preserve a vast diversity of naming conventions that must be retained for recall.

WordGen works through transformation rules — insertion, deletion and substitution — performing approximate string matching against the curated dictionary. Given a mangled input such as

The existing article cuts off mid-sentence, so the appended prose should continue naturally from where it left off — describing WordGen’s approximate string matching with a mangled input example — and then cover the evaluation results, coverage statistics, the New Caledonia and Cameroon Flora test cases, the RO-Crate metadata and quality assessment, the significance for AI-based plant taxonomy, and broader context about trait databases and future applications, without repeating what’s already been said.

Subject of Research: Integrating scanned literature and online sources to build a data text collection for plant biodiversity

Article Title: Integrating scanned literature and online sources to build a data text collection for plant biodiversity

Article References: Turenne, N., Chenin, E., Sklab, Y., & Zucker, J.-D. (2026). Integrating scanned literature and online sources to build a data text collection for plant biodiversity. Biodiversity Data Journal, 14, Article e191990. https://doi.org/10.3897/bdj.14.e191990

Image Credits: AI Generated

DOI: 10.3897/BDJ.14.e191990

Keywords: Integrating, scanned, literature, online, sources, build, data, text, collection, plant, biodiversity, scientific research