assessing model performance with cross-validation in epidemiology – Science https://scienmag.com Sat, 24 Oct 2026 16:09:17 +0000 en-US hourly 1 https://wordpress.org/?v=7.1.3 https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg assessing model performance with cross-validation in epidemiology – Science https://scienmag.com 32 32 73899611 Mobility patterns dominate explainable machine learning forecasts of COVID-19 cases in India https://scienmag.com/mobility-patterns-dominate-explainable-machine-learning-forecasts-of-covid-19-cases-in-india/ Sat, 24 Oct 2026 16:09:17 +0000 https://scienmag.com/?p=337835 A machine learning study forecasting district-level COVID-19 cases in two Indian states found that Long Short-Term Memory (LSTM) networks outperformed three other models, achieving an R² of nearly 0.8 with a normalized root mean square error (NRMSE) of 0.07 in Maharashtra and an R² of about 0.6 with an NRMSE of 0.15 in Kerala. The study covered 33 districts in Maharashtra and 14 districts in Kerala between April 26, 2020 and June 20, 2021, producing one-day-ahead case forecasts. Four models were compared: Multiple Linear Regression (MLR), Random Forest (RF), Extreme Gradient Boosting (XGB), and LSTM networks.

Each model drew on 17 explanatory variables spanning human mobility, meteorological conditions, vaccination, social gatherings, lockdown restrictions, and population density. The pandemic in India showed substantial spatial and temporal variation in transmission dynamics, influenced by these factors. The researchers built separate models for each state and for two periods: a pre-vaccination phase in 2020 and a vaccination phase in 2021. Model performance was assessed with 10-fold walk-forward cross-validation, and SHAP (SHapley Additive exPlanations) analysis was applied to the best-performing model to quantify the magnitude and direction of each variable’s contribution to predictions.

SHAP analysis identified mobility-related variables, particularly workplace, retail, and residential mobility, as the dominant explanatory variables in most models. Their importance and direction of association shifted between states and pandemic phases. Reduced workplace mobility was associated with higher predicted cases in Maharashtra in 2020, while higher workplace mobility was associated with higher predicted cases in 2021. Increased grocery and retail activity was associated with higher case counts in 2020, whereas higher residential mobility was associated with higher cases in 2021.

Population density remained an important and comparatively stable variable in Kerala across both years, but was influential only in 2020 in Maharashtra. Higher vaccination instances were consistently associated with higher predicted case counts in 2021 in both states. Lockdown restrictions showed opposite associations between the two states in 2021, coinciding with higher predicted cases in Maharashtra but lower predicted cases in Kerala. Meteorological variables showed comparatively modest but more consistent effects across states and years.

The study’s authors conclude that mobility patterns and population density were the variables most consistently associated with district-level COVID-19 case trajectories in Maharashtra and Kerala, while vaccination and lockdown measures carried associations that varied with timing and state context. The findings, which reflect statistical associations in retrospective forecasting data rather than causal effects, highlight the importance of context-specific models that account for changes in transmission patterns over time when understanding and forecasting epidemic spread.

Subject of Research: Explainable machine learning forecasting of district-level COVID-19 cases in Maharashtra and Kerala, India

Article Title: Explainable machine learning-based forecasting of COVID-19 cases: a comparative analysis of maharashtra and Kerala, India

Article References: Patel, A., Wathore, R., Jadhao, D., Wanjari, S., Dafare, A., & Lalwani, A. (2026). Explainable machine learning-based forecasting of COVID-19 cases: a comparative analysis of maharashtra and Kerala, India. BMC Infectious Diseases. https://doi.org/10.1186/s12879-026-14674-9

Image Credits: AI Generated

DOI: 10.1186/s12879-026-14674-9

Keywords: COVID-19 forecasting, machine learning, LSTM networks, SHAP analysis, human mobility, population density, vaccination, lockdown restrictions, Maharashtra, Kerala, district-level modelling, epidemic spread

]]> 337835