The research, led by Josef Škola and colleagues from Bulovka University Hospital, Charles University, and collaborating Czech institutions, set out to answer a deceptively simple question. When different clinicians examine the same stretch of physiological waveform, do they agree on what counts as an anomaly? The answer matters enormously because, despite rapid progress in automated signal cleaning, human annotations remain the de facto reference standard against which machine learning detectors are trained and judged. If the humans disagree, the algorithm inherits their confusion, and the ceiling of achievable model performance is set not by computing power but by the consistency of the labels themselves.

To measure that consistency, the team assembled a dataset of extraordinary granularity. From a single-center neurocritical care database of 39 patients with acute brain injury, they selected 100 hours of synchronized ICP and ABP recordings, slicing them into 32,760 ten-second segments. The ten-second window was no arbitrary choice: it matches the standard averaging interval used to compute the pressure reactivity index (PRx), a cornerstone measure of cerebrovascular autoregulation that is notoriously sensitive to waveform contamination. Six annotators spanning the full clinical spectrum, three senior neurocritical care consultants with more than a decade of experience, two junior residents, and one final-year medical student, independently labeled each segment using custom-built software that displayed the target segment within a minute of surrounding context, in fully randomized order.

The scale of the effort is worth pausing on. Each of the 32,760 segments was reviewed by all six annotators for both signals, yielding roughly 393,000 individual judgments. Annotators worked in isolation, unable to communicate about their progress or impressions, with every action logged to preserve procedural integrity. Crucially, the team adopted a deliberately broad definition of “anomaly” rather than the narrower, wildly variable “artifact” definitions that have fragmented the literature. An anomaly was any segment in which the signal departed from the expected physiological-technical pattern in a way that masked or distorted clinically relevant information, a framing borrowed from data science that the authors argue will generalize better across centers and event types.

The headline finding is reassuring at first glance. Anomalies were rare, just 3.99 percent of ICP segments and 5.83 percent of ABP segments, yet overall agreement was superb, with Gwet’s AC2, a statistic robust to rare events, exceeding 0.98 in every configuration. But the devil, as so often, lives in the details. When the researchers separated agreement on anomalies specifically, rather than agreement on the abundant normal segments, cracks appeared. For ICP waveforms, senior clinicians achieved substantially higher anomaly-specific agreement than the mixed-experience team: positive agreement rose by 12.8 percentage points, and Fleiss’ kappa by 13.5. For ABP, by contrast, experience barely mattered at all once the analysis accounted for the fact that segments from the same patient are statistically entangled.

That statistical subtlety deserves attention, because it is where much of the study’s methodological rigor lies. Ten-second segments are not independent observations; each of the 16 contributing patients supplied many consecutive segments, and anomalies cluster within individuals. Treating segments as independent would artificially inflate confidence. The team therefore reported every estimate twice, once at the segment level and once with patient-level cluster-robust confidence intervals derived from bootstrap resampling of whole patients, and anchored all inferential claims to the more conservative patient-level analysis. Under that stricter lens, the experience effect for ICP remained robustly significant, while most nominal differences for ABP dissolved.

Why would expertise matter so much more for intracranial pressure than for arterial blood pressure? The authors point to the nature of the distortions themselves. ABP anomalies tend to be loud and stereotyped: flushing artifacts, damped traces, sudden dropouts that even a medical student can spot instantly. ICP anomalies, by contrast, often hide in the fine structure of the pulsatile waveform, shifts in the relative heights of the P1 and P2 peaks, subtle changes in pulse morphology that blur the line between technical glitch and genuine physiological change in cerebral compliance. Reading those nuances fluently requires years of exposure to both the monitoring technology and the underlying physiology, exposure that routine clinical practice concentrates in specialized neurocritical care units.

The study also delivers a cautionary tale about statistics. Fleiss’ kappa, one of the most widely used reliability measures, suggested only moderate agreement (0.61 to 0.74) even as Gwet’s AC2 reported near-perfect consistency. This is the well-known kappa paradox: when the event of interest is rare, chance agreement inflates and kappa values sink, systematically understating reliability. No single metric tells the whole story, the authors argue, which is why they triangulated across Gwet’s AC2, Fleiss’ kappa, prevalence- and bias-adjusted kappa, positive and negative agreement, and the intraclass correlation coefficient. Only the combination reveals that the high overall agreement coexists with meaningful, modality-specific differences in how anomalies are actually identified.

The implications for the AI pipeline are direct and practical. Because human inconsistency injects noise into training labels, the reliability figures reported here effectively define the performance ceiling any detector trained on such data can reach. For ABP, that ceiling is high and reachable by mixed teams, meaning datasets can be expanded cheaply with junior annotators under light oversight. For ICP, the authors recommend expert consensus or multi-rater agreement to suppress label noise, higher confidence thresholds for ICP-based clinical predictions, and active-learning strategies that route genuinely ambiguous segments to senior review. They also note a subtle trade-off: senior annotators flagged fewer segments overall, suggesting stricter criteria that produce fewer false positives but may discard borderline events a broader pool would capture.

Limitations remain, and the authors are candid about them. The 100 hours of data, though vast in segment count, came from a single center with standardized equipment; the cohort was dominated by physiology within recommended target ranges, leaving extreme states underexplored; and the binary labeling scheme ignores anomaly severity and mechanism. Future work, they suggest, should purposively sample unstable physiology, dissect subtype-specific reliability, and pursue consensus methods such as Delphi studies to sharpen anomaly definitions. But the core message stands as a milestone for the field: reliable human annotation of brain and blood pressure waveforms is achievable at scale in routine neurocritical care, yet it is not a one-size-fits-all enterprise. As hospitals race to build AI on top of their monitoring streams, this study is a reminder that the most important component of any machine learning system may still be the trained human eye, and knowing exactly when that eye needs a decade of experience behind it.

Subject of Research: Interrater reliability of human anomaly annotation in intracranial pressure and arterial blood pressure waveforms for training machine learning models in neurocritical care

Article Title: Data for Machine Learning Models: Clinical Experience Determines Interrater Reliability in Waveform Anomaly Detection

Article References: Škola, J., Posel, Z., Waldauf, P., Falta, P., Soukup, M., Horáková, L., & Rožánek, M. (2026). Data for Machine Learning Models: Clinical Experience Determines Interrater Reliability in Waveform Anomaly Detection. Neurocritical Care. https://doi.org/10.1007/s12028-026-02655-4

Image Credits: AI Generated

DOI: 10.1007/s12028-026-02655-4

Keywords: neurocritical care, intracranial pressure, arterial blood pressure, waveform anomaly detection, interrater reliability, machine learning, artifact detection, clinical expertise, ground truth annotation, pressure reactivity index, acute brain injury, Gwet's AC2

Now, a team of researchers at the U.S. Food and Drug Administration’s Division of Imaging, Diagnostics, and Software Reliability has developed a new tool designed to catch these flaws before they can do damage. The system, called HistoART, is described in a study published in the journal Neural Computing and Applications. Led by Seyed Kahaki, with contributions from Alexander Webber, Ghada Zamzmi, Adarsh Subbaswamy, Rucha Deshpande, and Aldo Badano, the work compares three fundamentally different strategies for detecting artifacts in whole slide images, and finds that a large foundation model originally trained for general pathology tasks outperforms both a conventional deep learning network and a classical, hand-engineered approach.

The stakes are higher than they might first appear. As hospitals adopt digital pathology workflows, the reliability of downstream image analysis tasks, from tumor detection to grading, depends on the quality of the images feeding into them. An algorithm trained to recognize cancer cells may misinterpret a tissue fold as a suspicious structure, or an air bubble as a region of lost tissue. Previous studies have documented how quality control, or the lack of it, directly affects the accuracy of computational pathology systems. Yet many existing quality control tools rely on fixed rules and handcrafted measurements that may not generalize across the wide variety of scanners, staining protocols, and tissue types encountered in real-world laboratories.

HistoART tackles the problem with a three-way comparison. The first approach, the foundation model-based approach, fine-tunes UNI, a general-purpose foundation model for computational pathology that has been trained on massive collections of pathology images. Foundation models of this kind learn rich, general-purpose visual representations that can be adapted to many downstream tasks with relatively little additional training. The second approach, the deep learning approach, is built on a ResNet50 backbone, a widely used convolutional neural network architecture originally developed for general image recognition. The third, the knowledge-based approach, dispenses with learned representations altogether and instead relies on handcrafted features derived from texture, color, and frequency-based metrics, drawing on decades of classical image analysis research.

All three approaches were trained and evaluated to detect six of the most prevalent artifact types in whole slide images: tissue folds, out-of-focus regions, air bubbles, tissue damage, marker traces, and blood contamination. Each of these artifacts arises at a different stage of the workflow. Tissue folds occur when sections of tissue wrinkle during mounting on the slide. Out-of-focus regions result from imperfect autofocus during scanning. Air bubbles become trapped under the coverslip, marker traces are left by pens used to label slides, and blood contamination can obscure underlying tissue architecture. Detecting such a heterogeneous set of defects demands a system capable of recognizing very different visual patterns, from the sharp linear creases of a fold to the diffuse haze of an out-of-focus patch.

The evaluation was deliberately broad. The researchers assembled a dataset of more than 50,000 image patches sourced from diverse whole slide imaging scanners, including instruments from Hamamatsu, Philips, and Leica Aperio AT2, and drawn from multiple imaging sites. This diversity matters because artifacts can look different depending on the scanner that produced the image, and a detection system that works well on one manufacturer’s output may fail on another’s. By testing across scanners and sites, the team aimed to assess how well each approach would generalize beyond a single laboratory environment, a persistent weakness of many published machine learning studies in pathology.

The results were striking. The foundation model-based approach achieved a patch-wise area under the receiver operating characteristic curve, or AUROC, of 0.995, with a 95 percent confidence interval of 0.994 to 0.995, a level of performance that approaches the theoretical ceiling of a perfect classifier. The ResNet50-based deep learning approach reached an AUROC of 0.977, with a confidence interval of 0.977 to 0.978, while the knowledge-based approach, built on handcrafted features, trailed at 0.940, with a confidence interval of 0.933 to 0.946. The AUROC metric measures a classifier’s ability to distinguish between positive and negative cases across all possible decision thresholds, so the gap between 0.995 and 0.940 represents a substantial difference in real-world reliability, particularly for the subtle or ambiguous artifacts that matter most in clinical practice.

Why would a foundation model so decisively outperform a purpose-built convolutional network? The likely answer lies in the breadth of its pretraining. UNI was trained on enormous and varied collections of pathology imagery, meaning it has already encountered an extraordinary range of tissue appearances, staining variations, and imaging conditions before any artifact-specific fine-tuning begins. When subsequently adapted to the artifact detection task, it brings that prior knowledge to bear, allowing it to distinguish, say, a genuine tissue fold from a naturally occurring tissue edge with greater confidence than a network learning visual features from scratch. The handcrafted approach, meanwhile, is limited by the imagination of its designers: texture descriptors, color statistics, and frequency-domain measures capture certain artifact signatures well but may miss patterns that do not fit predefined mathematical descriptions.

Detection alone, however, is not enough. A laboratory needs to know what to do with the information. To bridge the gap between detection and actionable conclusions, the team developed a quality report scorecard that quantifies the number of high-quality image patches in a dataset and visualizes the distribution of artifact subgroups. This reporting layer transforms raw per-patch predictions into a summary that laboratory staff and algorithm developers can act upon: a slide riddled with folds can be rescanned or excluded, a scanner producing systematic out-of-focus regions can be serviced, and a dataset destined for training a diagnostic model can be filtered to remove compromised patches. In this way, HistoART functions not merely as a classifier but as a quality assurance pipeline for the entire digital pathology workflow.

The researchers describe the resulting system as a multi-branch pipeline that may enhance the reliability of whole slide image analysis by providing a scalable and interpretable approach for improving digital pathology workflows. Interpretability is a key selling point of the scorecard design, since regulators and clinicians are often wary of black-box systems that flag images without explaining why. By visualizing which artifact types dominate a slide or dataset, the tool offers a transparent window into the failure modes of the imaging process. Consistent with the team’s commitment to open science, all the data and methods from the work are available for download on GitHub, allowing other laboratories to adopt, scrutinize, and extend the system. As whole slide imaging continues its march toward becoming the default medium of diagnostic pathology, tools like HistoART suggest that the quality of the digital slide, long treated as an afterthought, is finally being engineered with the same rigor as the diagnoses that depend on it.

Subject of Research: Artifact detection in whole slide histopathology images using foundation models, deep learning, and handcrafted features

Article Title: HistoART: Histopathology artifact detection based on large foundation model

Article References: Kahaki, S., Webber, A., Zamzmi, G., Subbaswamy, A., Deshpande, R., & Badano, A. (2026). HistoART: Histopathology artifact detection based on large foundation model. Neural Computing and Applications, 38(18), Article 753. https://doi.org/10.1007/s00521-026-12454-9

Image Credits: AI Generated

DOI: 10.1007/s00521-026-12454-9

Keywords: histopathology, whole slide imaging, artifact detection, foundation model, UNI, ResNet50, digital pathology, quality control, machine learning, AUROC, cancer diagnostics, image analysis