AI Reads Chest X-Rays With Near-Perfect Accuracy, Then Stumbles in the Real World

Artificial intelligence systems that read chest X-rays have spent the past half-decade posting eye-watering performance numbers, routinely claiming accuracy figures above 96 percent and, in some studies, pushing toward 99. A new systematic review has now taken a hard, quantitative look at that entire literature, and its verdict is a fascinating mix of triumph and caution. The review, published in the journal Multimedia Tools and Applications by Yosra Didi, Ahlem Walha, and Ali Wali of the National Engineering School of Sfax in Tunisia, synthesizes 121 peer-reviewed studies published between 2020 and 2025, all aimed at automating the detection and classification of four major pulmonary threats: pneumonia, COVID-19, tuberculosis, and lung cancer. The authors followed the PRISMA reporting guidelines for systematic reviews, screening an initial pool of 300 candidate studies and filtering them down through rigid inclusion and exclusion criteria drawn from nine major electronic databases, including IEEE Xplore, ScienceDirect, SpringerLink, PubMed, the ACM Digital Library, Elsevier, MDPI, Wiley, and Google Scholar. What emerges is the most detailed map yet of where machine learning in chest radiography genuinely stands, and where its laboratory sheen begins to crack.

The headline technical finding concerns architecture. Across the 121 studies, convolutional neural network backbones remain the undisputed workhorses of chest X-ray analysis. ResNet variants, with their residual skip connections that let gradients flow through very deep stacks of layers, appear again and again as the feature extractor of choice. DenseNet, which connects each layer to every subsequent layer so that features are reused rather than relearned, and EfficientNet, which scales network depth, width, and input resolution in a balanced, compound fashion, round out the most widely adopted trio. These designs, largely inherited from the ImageNet natural-image lineage that stretches back through VGG and GoogleNet to AlexNet in 2012, excel at converting a grayscale radiograph into a hierarchy of visual features, from edges and textures to the diffuse opacities and consolidations that radiologists associate with infection or malignancy. The review shows that when these backbones are coupled with transfer learning, the practice of initializing a network with weights pre-trained on millions of natural images before fine-tuning it on medical data, the resulting models dominate the standard benchmarks used in the field.

Ensemble methods add a second layer of dominance. Rather than trusting a single network, many of the highest-performing studies combine predictions from multiple architectures, for example fusing Xception with ResNet50V2, or blending parallel Visual Geometry Group networks with classical machine learning classifiers such as support vector machines and random forests. The logic is statistical: individual networks make partly uncorrelated errors, and averaging or voting across them cancels out idiosyncratic mistakes. The review’s quantitative synthesis confirms that transfer learning paradigms and multi-model ensemble configurations consistently top the leaderboards, with reported classification accuracies clustering in the 96 to 99 percent range for pneumonia, COVID-19, and tuberculosis screening tasks. Attention-based designs have also entered the mainstream, including dense attention mechanisms, masked neural networks, and, more recently, vision transformers, whose self-attention blocks model long-range dependencies across the entire radiograph rather than being confined to a local receptive field.

Beneath the algorithmic layer, the review devotes careful attention to preprocessing pipelines, which it treats as a first-class variable in the literature it categorizes. Chest radiographs are notoriously variable in contrast and exposure, and techniques such as contrast-limited adaptive histogram equalization, known as CLAHE, appear repeatedly as a standard remedy, redistributing pixel intensities locally to reveal subtle opacities without amplifying noise. Other studies apply morphological contrast enhancement with optimized structuring elements, unsharp masking derived from anisotropic diffusion models, multi-level enhancement operations, and denoising schemes built on multi-resolution parallel residual CNNs. Data augmentation plays an equally prominent role: synthetic minority oversampling, or SMOTE, to counteract imbalanced class distributions, Gabor-filter-based augmentation to inject texture diversity, three-dimensional rotational augmentation to mimic projection variability, and generative adversarial networks that synthesize additional training images. Several COVID-era studies combined deep convolutional features with discrete wavelet transforms before handing the vectors to classical classifiers, illustrating the hybrid pipelines that the field increasingly favors.

Yet the review’s most consequential contribution is its cross-reading of the evaluation literature, and here the picture darkens considerably. When the authors examined how these high internal metrics behave under rigorous external validation, testing a model trained at one institution on data from a different hospital, scanner, or population, they found systematic performance degradation. Models that appear virtually perfect on their own held-out test sets routinely lose several points of accuracy, and sometimes far more, when confronted with images from an unfamiliar workflow. The culprit is domain shift: differences in acquisition devices, dose levels, patient positioning, demographic composition, and even the subtle visual fingerprint of a hospital’s imaging protocol. A convolutional network, optimizing purely for pixel statistics, can latch onto shortcuts, such as lateral markers, text annotations, or institution-specific artifacts, that vanish or change outside the training domain. The review identifies this as a severe domain generalization bottleneck and ties it to deep-seated dataset biases running through the field’s standard benchmarks, from ChestX-ray8 and CheXpert to MIMIC-CXR, PadChest, VinDr-CXR, and the many COVID-specific collections assembled in haste during the pandemic.

Compounding the problem is the sheer unevenness of available data. The review isolates acute data scarcity and demographic imbalance as the first of three mutually reinforcing obstacles to clinical translation. Rare pathologies, pediatric cases, and underrepresented populations are thinly sampled in public datasets, which skews models toward the majority distribution and hides clinically meaningful failures, a phenomenon researchers have described as hidden stratification. The second obstacle is the uninterpretable black-box nature of deep neural networks. A model may flag a region as pneumonia-like, but without a mechanism that connects the decision to radiologically coherent evidence, clinicians cannot verify its reasoning, and regulators and patients cannot trust it. Explainable AI methods, including segmentation-based heatmaps and attention visualizations embedded directly into classification pipelines, have grown rapidly in response, and the review catalogues this trend, while noting that saliency maps themselves can be misleading if not anchored in causal understanding.

The third obstacle is performance degradation across heterogeneous institutional workflows, the operational face of the domain-shift problem. Even a model validated externally may fail silently when embedded into a live clinical pipeline whose image sizes, label conventions, comorbidity profiles, and referral patterns differ from anything seen in training. A 2022 study cited in the review documented a measurable generalization gap for convolutional networks on COVID-19 X-ray classification, and the broader literature on external validation of radiologic deep learning, also surveyed by the authors, echoes the same pattern across modalities. The review is explicit that the gap between laboratory performance and real-world reliability is not a rounding error but a structural feature of how these systems are currently built and evaluated.

In response, the authors chart an actionable research roadmap organized around three converging technologies. Federated learning, demonstrated experimentally in the literature with COVID-19 chest X-ray data, allows hospitals to train a shared model by exchanging gradient updates rather than patient images, preserving privacy while widening the demographic and institutional diversity of the training distribution. Vision-language models, which couple image encoders with text encoders trained on paired radiology reports, offer a path to richer supervision: free-text reports contain information that categorical labels discard, and alignment between the two modalities may teach models radiological semantics rather than dataset-specific shortcuts. Causality-driven explainable AI completes the triad, aiming to move explanation from post-hoc visualizations to models that learn features with a genuine causal relationship to disease, which should, in principle, survive the shift between domains far better than correlational features do.

The stakes of getting this right are enormous. Chest X-ray is one of the most common diagnostic examinations on Earth, cheap, fast, and often the first line of defense against diseases that kill millions annually, from tuberculosis to lung cancer. An automated reader that genuinely generalizes could bring expert-level triage to clinics without radiologists, accelerate screening in high-burden regions, and serve as a tireless second opinion in emergency departments. The review’s synthesis suggests the raw predictive power already exists, captured in networks trained by transfer learning and hardened by ensembles, and the remaining challenge is less about squeezing out another decimal point of benchmark accuracy than about engineering for robustness: honest external validation, bias-aware dataset curation, uncertainty quantification, and explanations that a clinician can interrogate. That reframing, from leaderboard competition to trustworthy deployment, is arguably the review’s most valuable contribution, and it arrives at exactly the moment the field needs to make it.

For readers watching the broader trajectory of AI in medicine, the study lands as a sober but ultimately optimistic data point. It confirms that the algorithmic toolkit, from residual CNNs and EfficientNets to transformers, attention ensembles, and GAN-based augmentation, is mature enough to match expert performance on curated benchmarks. It simultaneously demonstrates, with systematic evidence rather than anecdote, why 99 percent on a Kaggle challenge does not translate into a 99 percent clinical tool. The three-way prescription of federated training, vision-language understanding, and causal explainability gives researchers a concrete agenda for the next five years. If the field follows it, the promise that has energized medical AI since CheXNet first claimed radiologist-level pneumonia detection in 2017 may finally survive contact with the messy, heterogeneous, and profoundly human reality of the clinic, and the chest X-ray, a technology that has barely changed in a century, could become the proving ground for trustworthy diagnostic intelligence.

Subject of Research: Deep learning and machine learning approaches for classifying lung diseases from chest X-ray images

Article Title: A comprehensive review of AI-based approaches for lung disease classification using chest X-ray images

Article References: Didi, Y., Walha, A., & Wali, A. (2026). A comprehensive review of AI-based approaches for lung disease classification using chest X-ray images. Multimedia Tools and Applications, 85(10), Article 774. https://doi.org/10.1007/s11042-026-21915-1

Image Credits: AI Generated

DOI: 10.1007/s11042-026-21915-1

Keywords: artificial intelligence, deep learning, chest X-ray, pneumonia detection, COVID-19, tuberculosis, lung cancer, convolutional neural networks, transfer learning, ensemble learning, domain generalization, explainable AI