PolyU’s AI “Virtual Patient” Builds a Living Digital Twin to Personalise Cancer Treatment
A research team at The Hong Kong Polytechnic University (PolyU) has developed an artificial intelligence system designed to transform how doctors monitor disease and evaluate cancer treatments. Known as the AI Virtual Patient Simulation System, the platform brings together genomic information, medical images, pathology reports, laboratory results and clinical records to create a continuously updated digital representation of an individual patient. Rather than treating diagnosis as a single event based on one scan or report, the system is designed to model a patient’s changing condition over time and simulate how that person might respond to different therapeutic strategies.
The approach addresses a central challenge in modern medicine: patients generate large amounts of clinically important data, but those data are frequently stored in separate systems and interpreted independently. A CT scan may reveal the physical structure of a tumour, while genomic testing may indicate the molecular pathways driving its growth and clinical notes may document symptoms, treatment history and adverse reactions. Conventional artificial intelligence tools often analyse only one of these sources, limiting their ability to capture the biological complexity of cancer. PolyU’s system instead uses multimodal data integration, allowing algorithms to examine relationships among imaging features, biomarkers, genomic profiles and treatment outcomes.
At the centre of the platform is a patient-centric “digital twin”—a computational model that is updated as new information becomes available. In principle, this model can reflect changes in disease status, laboratory measurements, symptoms and other clinically relevant signals. Its purpose is not simply to archive information, but to support predictive analysis. When a patient’s condition changes, the system can help healthcare professionals assess possible future trajectories and compare the potential effects of alternative treatment plans. Such simulations could be particularly valuable in cancer and critical care, where disease progression may be rapid, therapeutic options may carry serious risks and decisions often require input from multiple medical specialties.
The platform includes tools for healthcare professionals as well as a mobile application for patients. Doctors can use the clinical interface to assemble a more comprehensive view of a patient’s history and to support diagnosis, monitoring and treatment assessment. The system is also intended to facilitate multidisciplinary consultations and referrals by making relevant information easier to review across clinical teams. Through the patient-facing application, individuals can upload medical records, record daily symptoms and follow changes in their health status. This design aims to shift patients from passive recipients of care toward active participants in managing their conditions, while giving clinicians access to a more continuous stream of patient-reported information.
Data exchange is supported by an encrypted Deep Feature QR code, which the team says can help transfer medical information securely across clinics, hospitals and devices. The technology is intended to improve interoperability without abandoning privacy protections, a critical requirement for any system handling genomic and clinical data. However, secure transmission is only one part of responsible medical AI. Systems used in healthcare must also address data quality, consent, cybersecurity, algorithmic bias and the interpretability of predictions. PolyU’s platform is being developed as a decision-support tool rather than a replacement for clinical judgment, with healthcare professionals remaining responsible for interpreting results in the context of each patient’s circumstances.
To demonstrate the system’s potential in oncology, the researchers introduced a clinical, data-driven, multiscale framework for predicting responses to immunotherapy in people with non-small cell lung cancer (NSCLC). The framework, called the Visual-Global Relation Fusion Network, or ViGNet, is designed for digital pathology and combines histopathological image features with clinical information. Its inputs include gene-expression profiles and cancer-related text, allowing the model to connect microscopic tissue patterns with molecular and clinical factors associated with treatment response. This is technically important because immunotherapy outcomes can depend on interactions between tumour biology, the surrounding immune environment and patient-specific clinical characteristics.
ViGNet uses a multiscale visual encoder to examine pathology images at different levels of detail. At a fine scale, the model can identify cellular and tissue-level patterns; at broader scales, it can assess the organisation of tumour regions and their surrounding microenvironment. A separate gene-driven encoder processes molecular information, while the fusion architecture seeks to establish relationships between visual and genomic representations. By combining these data streams, the model is intended to identify features that may be difficult to recognise when pathology images or molecular profiles are analysed in isolation. The result is a prediction framework aimed at distinguishing patients more likely or less likely to benefit from immunotherapy.
In qualitative and quantitative evaluations, the researchers reported that ViGNet outperformed baseline approaches in response classification, achieving a reported discrimination performance of 82.55% in predicting immunotherapy response. The result suggests that multimodal integration may improve the ability of machine-learning systems to extract clinically relevant signals from complex cancer datasets. At the same time, a performance figure alone does not establish that a model is ready for routine clinical use. Independent validation across hospitals, patient populations and imaging platforms will be necessary, along with prospective studies examining whether AI-supported predictions actually improve treatment decisions and patient outcomes. The reliability of such systems also depends on the quality and representativeness of the data used to train them.
Prof. Lawrence Chan, associate professor in PolyU’s Department of Health Technology and Informatics and leader of the research team, described the AI Virtual Patient Simulation System as a platform combining diagnosis, monitoring and treatment assessment. According to Chan, the system may help identify subtle pathological relationships across multimodal datasets and act as a monitoring “sentinel” by alerting care teams when biomarkers or symptoms become abnormal. Earlier recognition of clinically significant changes could help shorten assessment times and support more precise treatment planning, although the practical value of these alerts will depend on careful clinical validation and integration into existing workflows.
The project was recently showcased at Mobile World Congress 2026 in Barcelona, where it was shortlisted as a finalist for the 2026 Global Mobile Awards in the category of Best Mobile Innovation for Connected Health and Wellbeing. PolyU reports that the work has received support from its Micro Fund and Seed Fund, as well as the Greater Bay Area Innovation and Entrepreneurship Incubation Programme. The project has also been conditionally accepted into the Hong Kong Science and Technology Park’s Incubation Programme and is moving toward commercialisation and industrialisation. If clinical deployment expands, the resulting real-world data could support drug development, clinical trials and treatment optimisation. The research team’s ViGNet study has been published in Medical Image Analysis, adding a peer-reviewed foundation to a broader effort to make personalised, continuously updated cancer intelligence part of future healthcare.
Subject of Research: Artificial intelligence, digital twins, multimodal medical data integration, digital pathology and personalised cancer treatment
Article Title: ViGNet: A clinical data-supported deep learning approach for NSCLC immunotherapy response prediction in digital pathology
Web References: https://www.sciencedirect.com/science/article/pii/S1361841526002239
References: Medical Image Analysis
Image Credits: PolyU
Keywords
Artificial intelligence, virtual patient, digital twin, cancer, non-small cell lung cancer, immunotherapy, digital pathology, medical imaging, genomics, biomarkers, personalised medicine, machine learning, multimodal data, clinical decision support
