AI Turns Museum Drawer Photos Into Instant Insect Inventories

Deep inside the Senckenberg Research Institute and Natural History Museum Frankfurt lies one of the world’s great insect collections: roughly 1.9 million beetle specimens pinned into more than 5,000 drawers, assembled over more than 150 years through acquisitions, exchanges and successive rounds of curatorial reorganisation. Around 6,600 primary and secondary type specimens sit among them. Yet for all its scientific value, much of this collection has remained effectively invisible to researchers and curators alike, because the knowledge it contains is encoded not in databases but in the physical arrangement of pins, labels and rows within drawers. A new open-source workflow, published in Natural History Collections and Museomics, now shows how artificial intelligence can read that arrangement directly from photographs, converting whole-drawer images into structured, machine-readable inventory data.

The problem the researchers set out to solve is one that plagues large historical collections everywhere. Taxonomic concepts change over time, but physical arrangements often reflect outdated classifications, so specimens belonging to a single taxon may be scattered across multiple drawers under different historical names. Labels are mostly handwritten, heterogeneous in format and only minimally standardised. Without a comprehensive digital overview, assessing taxonomic coverage, identifying gaps or planning digitisation requires extensive manual inspection, which is slow and delays decisions about conservation, pest management, loans and budget allocation. Specimen-level digitisation, in which each insect is individually imaged and transcribed, is far too labour-intensive to solve this quickly for millions of pinned insects.

Whole-drawer imaging has emerged as an efficient alternative, capturing thousands of specimens in a single photograph while preserving their spatial context. But existing workflows, such as the semi-automated segmentation tool Inselect and the more recent AI-based DrawerDissect system, treat the drawer mainly as a source image from which individual specimens are extracted. The curatorial structure encoded in the drawer’s organisation, the taxonomic series that generations of curators built into the layout, is typically lost. The new pipeline takes the opposite view: the drawer itself becomes the primary unit of analysis, and its internal structure becomes the data.

The technical foundation is a low-cost imaging setup. Each drawer, measuring approximately 555 by 572 millimetres, was photographed in a single exposure using a Reprostand “Book 2 net Reprosystem” archive scanner mounted on a Keiser RSD camera stand and illuminated by two NANLITE Compac 100 LED panels. The resulting images, recorded as high-resolution TIFF files at 9992 by 7088 pixels with an average file size of about 203 megabytes, capture the complete drawer layout, a colour calibration scale and the drawer front label. Preparation and imaging took roughly five minutes per drawer, yielding an average throughput of about 46 drawers per day under standard operating conditions.

From there, a Python-based pipeline takes over. Each image is linked to a unique identifier combining the collection code with sequential numbering, such as SMFCOLD000001, which is overlaid on the image to guarantee persistent linkage between the digital file and the physical drawer. The embedded colour card, containing a white 16 by 16 millimetre square, is automatically detected using OpenCV thresholding to calculate a pixel-to-centimetre ratio and generate a scale bar. White balancing, applied via the colour calibration scale, reduces lighting variation. Images are then cropped to the drawer region and padded onto a square black background for neural network input.

The heart of the system is a deep learning object detection model based on the YOLOv.11m architecture, initialised with weights pre-trained on the COCO dataset and trained for 300 epochs at an input size of 640 by 640 pixels. Four classes were annotated in the training images: drawer columns, genus-group name labels, species-group name labels and individual specimens. In the Senckenberg arrangement, specimens sit in rows within columns beneath handwritten red-outlined genus labels and black-outlined species labels, a visual convention the model learned to exploit. Trained exclusively on Carabidae, the ground beetle family, the model achieved striking results on the validation set: overall precision of 99.2 percent, recall of 97.9 percent and a mean average precision at IoU threshold 0.50 of 98.5 percent. Genus labels, species labels and drawer columns were classified with near-perfect accuracy, while specimens reached 93 percent correct predictions, with most errors involving confusion with the background class.

Notably, the team found that shrinking the training set from 513 to 257 images produced negligible performance losses, suggesting that even a modest annotation effort is sufficient to capture the detection task. K-fold cross-validation with four folds confirmed the stability of the results, with per-class metrics showing low variance across partitions. The annotated Carabidae drawers represented approximately 144,187 specimens, spanning considerable morphological diversity from small, slender taxa to larger, more robust forms, all sharing the broadly consistent beetle body plan and structured row arrangement that made the learning problem tractable.

The real test was transferability. Applied to 30 drawers each from two other beetle families, Buprestidae and the longhorn beetle subfamily Lamiinae, the model held up remarkably well despite differences in morphology and arrangement. Buprestidae specimen detection achieved a precision of 0.922 and recall of 0.959, while Lamiinae reached a precision of 0.975. Performance declined further on 30 drawers of Hymenoptera, an entirely different insect order with elongated bodies, prominent wings and antennae and less uniform packing: specimen precision fell to 0.829 and recall to 0.69, with 28 percent of true specimens predicted as background. Yet even there, detection of genus labels, species labels and columns remained high, indicating that the model had captured layout regularities independent of taxonomic identity. The researchers attribute the cross-domain pattern to the interplay of morphology and spatial organisation, and note that the model can be fine-tuned with additional annotated data to broaden its reach.

Once objects are detected, a custom script reconstructs the taxonomic series. Detected objects are sorted to mirror the physical drawer: columns ordered left to right, objects within each column sorted top to bottom. Genus and species labels are cropped from the image and read with the PP-OCRv.4 pipeline implemented in PaddleOCR, and the resulting text strings are appended to the detection records. Specimen counts per taxon are then derived by attributing each specimen to the label preceding it in the spatial sequence, preserving the taxonomic hierarchy expressed in the drawer’s arrangement. Specimens without an associated label receive placeholder designations, ensuring nothing is lost. The output for each drawer is a CSV file listing the drawer identifier, total specimen count, number of genus and species labels and a per-taxon breakdown.

Optical character recognition proved the weakest link, which is hardly surprising given that every label analysed was handwritten. Accuracy, measured as Levenshtein similarity between OCR output and ground truth, ranged from 11 to 100 percent at the individual label level, with a drawer-level mean of 83.8 percent. Lower scores traced back to ambiguous character shapes, historical abbreviations such as “Pal.” or “Neotr.” and degraded or partially unreadable labels. Yet the crucial finding is a decoupling between OCR performance and counting accuracy: specimen counting achieved an overall accuracy of 92 percent across the 68 Carabidae test drawers, even where text recognition was imperfect. Because the counting relies on spatial structure rather than full transcription, reliable quantitative information can be extracted without perfectly reading every handwritten name. Errors in counting clustered around densely packed specimens, partial overlap and irregular spatial arrangement, known limitations of two-dimensional object detection.

The implications reach beyond one museum. The authors argue that drawer-level reconstruction should not be seen as a rival to specimen-level digitisation but as a complementary intermediate layer, providing rapid collection-scale visibility while preserving historical and curatorial context. It gives institutions actionable information for prioritising digitisation, planning conservation and managing loans, and it complements specimen-centric pipelines by supplying the collection-wide context those pipelines lack. The team also raises a broader concern about biodiversity informatics: current standards such as Darwin Core, GBIF and iDigBio are built around specimen-level occurrence records, leaving little room for the drawer images, annotation layers, OCR outputs, inferred taxonomic hierarchies and spatial relationships that whole-drawer approaches generate. Annotated whole-drawer datasets are rarely made openly available in standardised, machine-readable formats, which limits interoperability, reproducibility and the benchmarking of AI workflows that depend on diverse training material. The authors call for these datasets to be disseminated under the same principles of openness, interoperability and long-term accessibility that guide specimen-level digitisation, warning that restricting access risks fragmentation, duplicated effort and long-term data loss.

Limitations remain, and the authors are candid about them. Overlapping individuals cannot be reliably separated in two-dimensional images. The workflow depends partly on collection-specific conventions such as the red and black label colour coding, and accuracy may drop where those visual cues are absent. Size estimates from drawer images are approximate without depth information, and irregular column alignment or variable specimen spacing can degrade performance, reflecting the physical realities of legacy collections rather than purely algorithmic constraints. Reconstructed taxonomy also reflects the historically curated state of the collection, so integration with current taxonomic frameworks requires an additional reconciliation step. Still, the demonstration that low-cost imaging and open-source tools can transform whole-drawer photographs into structured inventory data marks a practical milestone. For the millions of specimens sitting in historical drawers worldwide, waiting to be counted, the pixels are finally becoming data.

Subject of Research: Automated reconstruction of taxonomic series from whole-drawer images in historical insect collections using deep learning and OCR

Article Title: From pixels to data: automated reconstruction of taxonomic series from whole-drawer images in historical insect collections

Article References: Bhansali, S., Ignatev, N., & Simões, M. (2026). From pixels to data: automated reconstruction of taxonomic series from whole-drawer images in historical insect collections. Natural History Collections and Museomics, 3, Article e196081. https://doi.org/10.3897/nhcm.3.196081

Image Credits: AI Generated

DOI: 10.3897/nhcm.3.196081

Keywords: natural history collections, digitisation, whole-drawer imaging, deep learning, YOLO object detection, optical character recognition, entomology, Coleoptera, Carabidae, biodiversity informatics, museum collections, Senckenberg