Chromosome Measurements Differ Between Observers Even With Modern Software

When fifteen trained botanists sat down to measure the very same microscopic image of a plant’s chromosomes, they did not get the same numbers. That simple experiment, carried out during a practical course in cytotaxonomy held in Pisa, Italy, between 6 and 9 February 2024, has revealed that observer bias persists even in the era of digital image analysis software, and that it affects some of the most widely used karyotype statistics in surprising and selective ways. The findings, published in the journal Comparative Cytogenetics, suggest that decades-old warnings about human measurement error still apply to modern cytogenetics, and that the field urgently needs a standardized measurement protocol.

Karyomorphology, the study of the physical form and structure of chromosome sets, is one of the cheapest and most accessible tools in systematic botany. Researchers spread metaphase plates on a microscope slide, photograph them, and measure the length of the long arm (L) and short arm (S) of each chromosome. From these basic measurements they derive a suite of quantitative traits that describe the karyotype: the total haploid chromosome length (THL), which serves as a rough proxy for genome size; the mean centromeric asymmetry (MCA), which captures how unevenly genetic material is distributed between the two arms of a chromosome; the coefficient of variation of chromosome length (CVCL), which expresses differences in size between chromosomes; and the coefficient of variation of centromeric index (CVCI), which reflects how heterogeneous the centromere positions are across the complement. Together with chromosome number, these parameters underpin countless comparative studies of plant evolution and systematics.

But how reliable are these numbers when different people produce them? The question is far from new. As early as 1959, the cytogeneticist Sybenga distinguished two main sources of variation in karyomorphological data: biological variation in actual chromosome length, driven by differences in the degree of chromosome condensation, and artificial variation caused by inaccuracy of measurement. In 1971, Bentzer and colleagues explicitly tested whether the same measurements made by different people produce consistent data, and showed that they did not, especially when researchers relied on camera lucida drawings of metaphase plates. Since the early 2000s, however, a new era of chromosome measurement through image analysis software has transformed the field. Programs such as ImageJ and dedicated tools like MATO have made measurements far more accurate than the printed microphotographs and hand-drawn plates of the twentieth century. What remained unknown was whether, given the very same microphotograph and the very same software, different observers still introduce measurable biases.

The Pisa course, organized by the Group for Floristics, Systematics and Evolution of the Italian Botanical Society, provided the perfect setting to find out. Lorenzo Peruzzi of the University of Pisa and his colleagues distributed a single metaphase plate of the diploid angiosperm Santolina decumbens subsp. diversifolia, a member of the daisy family with 2n = 18 chromosomes, to all participants. The image had been assembled by pasting together several microphotographs taken at different focal planes, so that all eighteen chromosomes could be seen clearly in one picture. Each participant then independently measured the plate using the software MATO, without conferring with anyone else. Fifteen independent sets of measurements were collected in total.

The results were striking. When the team calculated the coefficient of variation of each karyomorphological trait across the fifteen observers, the highest values concerned CVCI, at 17.3 percent, and CVCL, at 13.4 percent. The lowest values were found for MCA, at 4.7 percent, and THL, at 9.4 percent. In other words, the parameters most sensitive to observer error were exactly those describing the variation of centromere position and chromosome length within the karyotype, while the grosser measures of total chromosome length and average asymmetry were comparatively more stable. For a field that routinely compares karyotypes across species and populations to infer evolutionary relationships, a 17 percent disagreement among trained observers measuring the identical image is not negligible.

The statistical analysis uncovered something even more interesting: the observer bias is selective, not random. The only highly significant correlation among the traits was a negative one between THL and MCA, with a Pearson correlation coefficient of -0.72. The team’s interpretation points towards a systematic tendency among observers to overestimate the length of the short arms of chromosomes. Such an overestimation would simultaneously inflate the total haploid chromosome length, because the sum of all chromosome arms grows, and deflate the mean centromeric asymmetry, because the difference between the proportions L/(L+S) and S/(L+S) shrinks as the two arms appear more equal. This mechanism neatly explains why the two parameters move in opposite directions across observers.

Why, then, does CVCI show the greatest variation of all, yet no significant correlation with THL? The answer lies in the mathematics of the centromeric index, which is calculated as S/(L+S). An overestimation of the short arm affects both the numerator and the denominator of this ratio. On one side, this cancels out any correlation between CVCI and THL; on the other, it injects a great deal of additional variation into the parameter, making it the most vulnerable to observer bias in the entire experiment. The authors suggest that these errors may stem from the different decisions observers make when locating the centromere, the constricted region that divides a chromosome into its two arms, a judgment call for which no standardized protocol has ever been formally proposed.

The study’s conclusions carry practical weight for anyone working in comparative cytogenetics. The parameters most sensitive to observer error are CVCI and CVCL, and to a lesser degree MCA and THL. The authors recommend special attention to recognizing and measuring the short arms of chromosomes correctly, since these are the main source of observer bias in cytogenetics. A homogeneous approach among observers, they argue, could help stabilize results, and the development of a standardized measurement protocol, which the field still lacks, would be a valuable step forward. Until such a protocol exists, researchers comparing karyotypes produced by different laboratories should treat small differences in centromere-related statistics with caution, since these may reflect the hands that made the measurements rather than the biology of the organisms.

Perhaps the most enduring lesson, however, is one that was already articulated more than fifty years ago. Bentzer and colleagues famously advised that, in the course of an investigation, all the measurements should be made by the same person. The Pisa experiment demonstrates that this motto fully applies to the era of image analysis software. Digital tools have eliminated the crudest errors of the camera lucida age, but the human eye and the human judgment of where a centromere lies remain sources of systematic disagreement. As karyomorphological data continue to feed into phylogenetic and taxonomic decisions across the plant sciences, the study serves as a reminder that even the most routine measurements deserve scrutiny, and that the observers behind the numbers are themselves a variable that science must account for.

Subject of Research: Observer bias in karyomorphological measurements using image analysis software

Article Title: Different observers introduce not negligible biases in comparative karyomorphological studies

Article References: Peruzzi, L., Franzoni, J., Tiburtini, M., Abidi, E., Alù, E., Barone, G., Bianchi, E., Cataudella, C., Di Iorio, E., Guerrina, M., Mondello, F., Paino, L., Pentassuglia, M., Porrovecchio, M., Rivieccio, G., Siccardi, E., Stinca, A., Tei, A., Volanti, V., & Giacò, A. (2024). Different observers introduce not negligible biases in comparative karyomorphological studies. Comparative Cytogenetics, 18, 175-182. https://doi.org/10.3897/compcytogen.18.135172

Image Credits: AI Generated

DOI: 10.3897/compcytogen.18.135172

Keywords: cytogenetics, karyomorphology, observer bias, chromosome measurement, image analysis, Santolina decumbens, karyotype asymmetry, centromeric index, plant systematics, measurement standardization, Different, observers