AI Picks the Weather Clues That Matter Most for Solar Power Forecasts

Solar power lives and dies by the accuracy of its forecasts. Grid operators, investors, and plant designers all depend on knowing how much sunlight will strike a panel array hours or days from now, and even small errors in predicted solar irradiance can ripple into mismatched supply, wasted reserve capacity, and lost revenue. Yet the meteorological data that feeds those forecasts is messy. Ground-based sensors record a dozen or more variables at once—temperature, humidity, wind speed and direction, rainfall, pressure—many of which are entangled with one another, distributed in ways that defy statistical assumptions, and vulnerable to instrument drift or failure. A research team led by Abubakar Danjuma Maiwada and Mardiana Idayu Ahmad of Universiti Sains Malaysia, together with Sani I. Abba of Prince Mohammad Bin Fahd University, has now tackled this problem head-on, publishing in the journal Discover Sustainability a hybrid framework designed to decide, with unusual rigor, which weather measurements actually deserve a place in a solar irradiance prediction model.

The central question the researchers posed is deceptively simple: when every weather sensor on a station is reporting, which of those readings genuinely carry information about incoming solar radiation, and which are noise, redundancy, or artifacts of imperfect instrumentation? Conventional practice often throws every available variable into a machine learning model and lets the algorithm sort it out. That approach can work, but it carries a cost. Models trained on irrelevant or collinear predictors tend to be fragile, harder to interpret, and more likely to break down when a sensor malfunctions or conditions shift outside the training range. In renewable energy applications, where forecasts feed directly into grid scheduling decisions, that fragility is not an academic concern—it is an operational liability.

To address the challenge, the team built a multi-stage evaluation pipeline that deliberately combines three complementary families of analysis. The first stage probes linear structure in the data. The researchers applied three rank- and product-moment correlation methods—Pearson, Spearman, and Kendall—to quantify how tightly each meteorological predictor co-varies with solar irradiance, and they deployed principal component analysis (PCA) to map the dominant axes of variance running through the whole dataset. PCA is a classical dimensionality reduction technique that rotates the data into new coordinates ordered by the amount of variance each captures; when a handful of components account for most of the spread, it signals that many original variables are telling overlapping stories. In this study, the first three principal components together explained 62.88 percent of the total variance, a clear indication of substantial multicollinearity among the sensor readings.

Linear analysis alone, however, cannot capture the full picture. Atmospheric variables interact in ways that are stubbornly nonlinear: humidity modulates cloud formation, which alters irradiance, but the relationship bends and saturates depending on temperature, season, and pressure. To test nonlinear sensitivity, the team trained a bilayer neural network—a shallow architecture with two hidden processing layers—capable of registering interactions that straight-line correlation would miss. Crucially, the neural network served not as a final forecasting product but as a probe: by measuring how prediction error responds when each input is perturbed, the researchers could rank variables by their functional importance rather than their statistical association alone. This dual-track strategy, pairing classical statistics with neural sensitivity testing, forms the hybrid core of the framework.

The third and arguably most novel stage brought explainable artificial intelligence into the mix. The researchers applied SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), two techniques borrowed from the machine learning interpretability literature that decompose a model’s prediction into contributions from each input variable. SHAP, grounded in cooperative game theory, allocates to each feature a Shapley value representing its average marginal contribution across all possible coalitions of predictors, yielding a global picture of which variables drive the model overall. LIME, by contrast, fits simple surrogate models around individual predictions to explain why a specific forecast took the value it did. Combining the two gives both the bird’s-eye view and the close-up, allowing the team to verify that the variables flagged by correlation and neural sensitivity are the same ones the trained model actually leans on when making its estimates.

The results delivered a clear hierarchy of predictors. The most influential single variable turned out to be average battery voltage, abbreviated ABV, which achieved a root mean square error of 0.203 when used to predict solar irradiance. The finding may look surprising at first glance—an electrical quantity appearing to outperform classic atmospheric measurements—but it makes physical sense in context: at a ground-based monitoring station, battery voltage tracks the charging cycle driven by the panel’s own exposure to sunlight, making it a sensitive, integrated proxy for the very quantity being forecast. Relative humidity followed as the second-strongest predictor, with an RMSE of 0.239, consistent with its well-documented role in cloud and aerosol processes that scatter and absorb incoming radiation. Wind direction ranked third at 0.244, plausibly reflecting its association with air mass origin and the transport of moisture and particulates.

The explainability analysis reinforced this ranking from an independent angle. Global SHAP assessment confirmed that ABV and relative humidity contributed most significantly to the model’s predictions, while rainfall emerged as a weak global influence—a result the researchers attribute to its sparse occurrence in the record. Rainfall events, though meteorologically dramatic, occupy so few rows in a typical time series that the model has little opportunity to learn a robust mapping from rainfall to irradiance, and the feature’s Shapley values remain correspondingly small. This kind of diagnostic insight is precisely what the framework is designed to surface: rather than silently retaining a predictor that adds little, reliability-aware selection recognizes that a rarely active sensor contributes maintenance cost and potential failure modes without commensurate predictive benefit.

Beyond the rankings themselves, the study’s methodological contribution lies in its explicit handling of the data-quality hazards that plague real-world meteorological records. The authors frame the problem as one of identifying physically meaningful predictors under conditions that are simultaneously non-normal, multicollinear, and sensor-sensitive—three threats that conventional single-method feature selection handles poorly. Non-normal distributions undermine parametric tests; multicollinearity inflates the apparent importance of redundant variables and destabilizes coefficient estimates; and sensor sensitivity means that measured values may reflect instrument behavior as much as atmospheric reality. By triangulating across correlation families, PCA, neural sensitivity, and interpretable machine learning, the framework forces a candidate predictor to earn its place through convergent evidence before it is accepted into a forecasting model.

The practical implications extend across the renewable energy value chain. More reliable irradiance forecasts translate directly into better solar farm siting, more accurate energy yield assessments for investors, and smoother integration of photovoltaic capacity into smart grids, where dispatch decisions must anticipate the variable output of solar assets. The framework’s emphasis on lean, interpretable predictor sets also lowers computational and maintenance burdens, since models built on a handful of well-justified inputs are cheaper to run and easier to audit than sprawling ones. For operators in data-scarce regions, the finding that a simple, routinely logged variable such as battery voltage carries exceptional predictive power suggests that meaningful forecasting capability may be achievable with minimal instrumentation—a potentially significant result for expanding solar infrastructure across developing regions where dense sensor networks are not yet in place.

The study, which received funding support from a Universiti Sains Malaysia Research University Grant, arrives at a moment when the global build-out of solar capacity is accelerating and the marginal value of forecast accuracy is rising accordingly. It also joins a broader movement in environmental machine learning toward transparency: rather than treating predictive models as black boxes, the authors demonstrate how explanation tools can be woven into the model-building process itself, serving as a quality check on the science as much as a window on the model. For a field where forecasting errors carry real economic and grid-stability consequences, the message is clear—the path to trustworthy solar prediction runs not through bigger models with more inputs, but through disciplined, reliability-aware scrutiny of the measurements that feed them.

Subject of Research: Hybrid feature selection for ground-based solar irradiance prediction using meteorological data and explainable machine learning

Article Title: Hybrid feature selection framework for reliable solar irradiance prediction using ground-based meteorological data

Article References: Hybrid feature selection framework for reliable solar irradiance prediction using ground-based meteorological data. (n.d.). https://doi.org/10.1007/s43621-026-04977-x

Image Credits: AI Generated

DOI: 10.1007/s43621-026-04977-x

Keywords: solar irradiance, feature selection, machine learning, SHAP, LIME, PCA, neural networks, renewable energy, meteorology, forecasting, multicollinearity, smart grids