Multimodal AI maps the path to personalized cancer immunotherapy

Cancer immunotherapy has redefined the treatment of many malignancies by activating or restoring antitumor immune responses, and in some patients it produces deep and durable remission. Yet the same therapies fail in many others: some show primary resistance from the outset, some progress after an initial response, and others develop immune-related adverse events that complicate care. Response patterns can be deceptively complex, including pseudoprogression, in which tumors appear to grow before later shrinking, and hyperprogression, in which disease accelerates on treatment.

A new review published in Science Bulletin argues that no single biomarker can capture these dynamic interactions among tumor cells, the immune system, the tumor microenvironment, and therapeutic interventions, and that multimodal artificial intelligence offers a framework for doing so. The review, titled "Emerging advances in multimodal AI in cancer immunotherapy: from multi-scale data integration to clinical decision support," was led by Professor Bin Wang of Army Medical University in collaboration with researchers from multiple institutions. It systematically examines how artificial intelligence can integrate information across molecular, cellular, tissue, and clinical scales.

The relevant data modalities span genomics, transcriptomics, proteomics, epigenomics, single-cell and spatial omics, digital pathology, radiological imaging, and longitudinal electronic health records. These datasets differ substantially in acquisition time, spatial resolution, technical platform, and biological meaning, which is precisely why conventional analytical approaches struggle to combine them. Multimodal AI, the authors contend, provides a means of learning non-linear relationships and higher-order interactions that traditional statistics cannot easily capture.

Conventional biomarkers such as programmed death-ligand 1 expression, microsatellite instability, and tumor mutational burden are already used to support patient selection for immunotherapy. However, their predictive accuracy and generalizability across cancer types remain limited. A single PD-L1 measurement, for example, reflects only one snapshot of one axis of an immune system that changes continuously under therapeutic pressure.

To organize a rapidly expanding literature, the authors trace the methodological evolution of AI in cancer immunotherapy through six interconnected stages. The first is early machine-learning classification, in which predefined clinical, imaging, or molecular features are fed into models for patient classification and risk prediction. The second is unsupervised representation learning, including autoencoders, variational autoencoders, and contrastive learning, which can extract latent features directly from high-dimensional biomedical data and reduce reliance on manual feature engineering.

The third is graph-based modeling, in which graph neural networks represent cells, genes, proteins, and their interactions as structured networks, enabling computational analysis of communication among tumor, immune, and stromal cells. The fourth stage comprises multimodal foundation models and biomedical language models, which are beginning to connect omics profiles, medical images, biological sequences, and clinical text within unified representation spaces. The fifth involves reinforcement learning and active learning paradigms, which may support dynamic treatment optimization and the selection of the most informative samples for further study.

At the frontier sits the sixth stage: AI virtual cells, computational systems designed to learn representations of cellular states and predict how cells respond to genetic, pharmacological, or immunological perturbations. Such systems could eventually complement mechanistic research and help prioritize candidate interventions, although the authors stress that extensive experimental validation remains essential before any such predictions can be trusted. At the molecular and cellular levels, the review describes how multimodal AI can integrate genomic, transcriptomic, proteomic, epigenomic, single-cell, and spatial data to characterize the cellular composition, functional states, spatial neighborhoods, and communication networks of the tumor immune microenvironment.

These approaches may help identify molecular programs associated with immune response, treatment resistance, and disease progression, while revealing rare cell populations and spatial niches that bulk-tissue analyses routinely miss. Emerging techniques such as AI-assisted virtual staining, cellular phenotyping, and spatial multi-omics integration may also allow researchers to extract more information from limited tissue samples, an important consideration when biopsy material is scarce. At the clinical interface, the review highlights applications in digital pathology and radiological imaging.

Models applied to hematoxylin and eosin-stained whole-slide images and immunohistochemistry images have been investigated for predicting PD-L1 expression, microsatellite instability, tumor mutational burden, and immune-cell infiltration, as well as for assessing immunotherapy response and prognosis. On the imaging side, radiomics and deep-learning models built on computed tomography, positron emission tomography, and magnetic resonance imaging can extract phenotypic information from tumors and surrounding tissues to predict treatment response, progression, recurrence, survival, and immune-related adverse events. Each of these modalities, on its own, offers a partial view; combined, they promise a far more complete portrait of an individual patient's disease.

Combining digital pathology, radiological imaging, multi-omics profiles, clinical variables, and longitudinal health records could generate more comprehensive patient representations than any single modality alone, improving patient stratification and helping to distinguish complex response patterns. In particular, longitudinal multimodal analysis may offer new opportunities to differentiate pseudoprogression from true progression or hyperprogression by jointly evaluating imaging changes, pathological features, molecular states, and clinical trajectories over time. The authors also argue that efficacy and toxicity should not be treated as separate prediction tasks: future systems will need to jointly estimate the likelihood of therapeutic benefit and the risk of immune-related adverse events, an integrated approach that could better support treatment selection, follow-up scheduling, and the timing of preventive or therapeutic interventions.

Despite substantial progress, the review is notably candid about the gap between published models and routine clinical use. Data modalities are often collected at different times, paired samples may be unavailable, and individual modalities may be missing entirely. Differences among institutions, devices, platforms, and patient populations can produce distribution shifts that degrade model performance when a model trained at one center is deployed at another.

Models may also learn shortcut features unrelated to disease biology, or fail to use all available modalities effectively. Many published studies remain limited by retrospective single-center designs, insufficient external validation, inconsistent endpoint definitions, and inadequate assessment of calibration and clinical net benefit. Additional concerns include the opacity of foundation models, hallucination and overconfidence in generative systems, limited causal interpretation, and insufficient biological validation of model-derived predictions.

The authors therefore call for multicenter, temporal, and real-world validation, together with systematic evaluation of missing modalities, data drift, calibration, and predictive uncertainty. Decision-curve analysis and net-benefit assessment, they argue, will be needed to determine whether an AI system can meaningfully improve clinical decisions rather than merely achieve high statistical performance. Clinical translation will also require transparent data provenance, interoperability across healthcare systems, privacy protection, and lifecycle monitoring after deployment.

Functional experiments, prospective studies, and clinical trials should be used to assess the biological plausibility and clinical value of AI predictions, and AI virtual cells will similarly require high-quality perturbation datasets, standardized benchmarks, and closed-loop collaboration between computational prediction and experimental validation. The review ultimately positions AI as an assistive technology for clinicians and researchers rather than a replacement for clinical judgment, mechanistic investigation, or prospective trials, offering a roadmap for moving cancer immunotherapy beyond isolated biomarkers toward decision-support frameworks that jointly consider therapeutic efficacy, toxicity, and dynamic disease evolution.

Subject of Research: Multimodal artificial intelligence for integrating multi-scale data in cancer immunotherapy decision support

Article Title: Multimodal AI roadmaps the future of personalized cancer immunotherapy

Article References: Multimodal AI roadmaps the future of personalized cancer immunotherapy. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: cancer immunotherapy, multimodal AI, biomarkers, digital pathology, radiomics, AI virtual cells, tumor microenvironment, foundation models, immune-related adverse events, pseudoprogression, clinical decision support, Science Bulletin