AI Can Classify National CO2 Emission Levels, But Only If It Never Sees the Country Before

Machine learning models that appear to classify countries’ carbon dioxide emission levels with high accuracy may be doing something far less impressive than it looks: recognizing countries they have already seen. That is the uncomfortable, methodologically important finding of a new study published in Frontiers in Artificial Intelligence, in which researchers built a global framework for classifying national CO2 emission levels using trade openness, governance quality, environmental taxation and macroeconomic indicators, and then deliberately tested whether their classifiers could generalize to countries and periods the models had never encountered during training. The answer, quantified with unusual rigor, is that a substantial share of the celebrated accuracy of emissions classification studies reflects within-country persistence rather than genuine predictive skill on unseen nations.

The study, led by Muhammad Ali Khawaja of the Applied Economics Research Centre at the University of Karachi and Muhammad Farooq Shaikh of the School of Engineering at the University of Warwick, assembled a country-level panel dataset covering 135 countries from 2000 to 2023, drawing on the World Bank’s World Development Indicators and the OECD’s environmental taxation database. Rather than treating CO2 emissions per capita as a continuous number to be predicted, the researchers framed the problem as an ordinal classification task: each country-year observation was assigned to one of three emission levels, low, medium or high, using tertile cut points estimated only on training data within every validation split. Under the primary, country-grouped design, those cut points fell at 1.32 and 5.04 tons of CO2 per capita, values close to but not identical with the pooled-sample thresholds of 1.35 and 5.00 tons, a small discrepancy that illustrates precisely the information leakage the training-only procedure is designed to prevent.

Six classifiers were tuned and compared: Random Forest, XGBoost, a support vector machine with a radial basis function kernel, a multinomial logistic regression, and two models designed specifically for ordinal outcomes, an ordered logistic regression and an ordinal decomposition of XGBoost that reduces the three-class problem to two nested binary classifiers, one for the probability that emissions exceed the low level and one for the probability that they exceed the medium level. Because the outcome is substantively ordered, every model was evaluated not only with accuracy and macro F1 but also with quadratic weighted and linear weighted Cohen’s kappa and mean absolute error on the class labels, so that a high-emission country misclassified as low, a severe two-category error, was scored as substantively worse than a one-category slip. Very few such severe errors occurred under any design, confirming that the ordinal structure of the outcome was respected even by models that imposed no ordinal loss.

The heart of the study is its validation design, and this is where the viral lesson for the machine learning community lies. Four designs were run side by side. In the primary country-grouped holdout and country-grouped five-fold cross validation, entire countries, not country-year observations, were allocated to training or test sets, so no model was ever evaluated on a country it had seen during training. In a chronological holdout, models trained on data through 2017 were tested on 2018 to 2023. In a one-year-ahead forecasting task, predictors measured in year t were used to classify the emission level in year t plus 1. And a random stratified country-year split, the design typical of much of the existing literature, was retained only as a reference point to quantify how much apparent accuracy reflects within-country persistence.

The results are striking. Under the random split, XGBoost achieved a test accuracy of 0.749 with a quadratic weighted kappa of 0.788, and Random Forest reached 0.766. Under the country-grouped design, accuracy fell by 12.5 percentage points for XGBoost, to 0.624, and to 0.629 for Random Forest, with the support vector machine at 0.650, the multinomial logit at 0.622, the ordered logit at 0.631 and the ordinal XGBoost model at 0.640. Nested country-grouped cross validation, in which hyperparameters were re-tuned within each fold, gave mean accuracies of 0.644 for XGBoost, 0.648 for Random Forest, 0.671 for the support vector machine, 0.674 for the multinomial logit, 0.676 for the ordered logit and 0.637 for the ordinal XGBoost model, with overlapping confidence intervals throughout. No classifier achieved a decisively higher accuracy than the others once the panel structure of the data was accounted for, a humbling result for anyone hoping that a particular ensemble algorithm holds the key to emissions classification.

Accuracy fell less sharply under the chronological holdout and the one-year-ahead forecast, landing in the region of 0.65 to 0.74: Random Forest reached 0.740 under the chronological holdout and XGBoost 0.710 under the one-year-ahead task. Because these designs still involve some overlap of countries between training and test periods, the pattern indicates that the classifiers generalize noticeably better across time within a country than across previously unseen countries. For a framework intended to support monitoring of countries in general, the authors argue, the country-grouped design is the more conservative and the more relevant test of generalization, and the gap between the designs is itself a diagnostic: it directly measures how much of a model’s apparent skill is recognition of persistent national profiles rather than prediction.

Confusion matrices under the country-grouped design reveal a consistent pattern across all six models: the low and high categories are classified reasonably well, while the medium category is the hardest for every model. For XGBoost, 235 of 253 low observations and 119 of 259 high observations were classified correctly, but only 140 of 280 medium observations were, with 94 misclassified as low. The explainability analyses explain why. SHAP summary plots for the medium category are visibly compressed, with values clustered near zero for most variables, indicating that the medium category has no distinctive profile of its own and is identified residually, as the absence of the clear signatures that characterize the low and high categories. LIME case studies, applied to 30 randomly sampled correctly classified and misclassified observations rather than hand-picked cases, confirmed this: misclassified cases showed markedly smaller and more evenly spread feature weights than correctly classified ones, meaning they carry no strong single-feature signal in either direction.

The feature importance results, confirmed jointly across permutation importance, SHAP and LIME and stable across cross-validation folds, identify government effectiveness, urban population growth and environmental tax revenue as the predictors most strongly and consistently associated with the classification. Permutation importance on the country-grouped test set gave mean importances of 0.123 for government effectiveness under XGBoost and 0.081 under Random Forest, followed by urban population growth at 0.081 and 0.101 respectively, and environmental tax revenue at 0.039 and 0.033. The remaining five predictors, rule of law, trade openness, gross fixed capital formation, labor force participation and GDP growth, all had importances close to zero, with GDP growth the least informative of all. The Spearman rank correlation between the SHAP rankings of the two tree-based ensembles was 0.952, and between permutation and SHAP rankings within XGBoost 0.881, a degree of cross-method agreement that considerably strengthens the central finding. Accumulated local effects and partial dependence plots add direction: the probability of the high category rises sharply once government effectiveness exceeds roughly 65 to 70 on the 0-to-100 scale, urban population growth above roughly 2 to 3 percent a year is associated with a sharply higher probability of the low category, and the probability of the high category rises steeply with environmental tax revenue up to about 1.5 percent of GDP before plateauing.

The authors are careful, throughout, to frame these associations as predictive rather than causal. The higher environmental tax revenue observed among high-emission countries should be read as a marker of the fiscal and administrative capacity that accompanies high-income, high-emission development stages, since countries that collect substantial environmental tax revenue are predominantly high-income economies with large per capita emission bases, not as evidence that environmental taxation itself increases emissions. Similarly, the strong association of governance with the classification suggests governance indicators are useful inputs to climate monitoring systems, but translating any of these associations into a specific governance, urbanization or tax policy recommendation would require causal evidence that this predictive design cannot provide. The results are best used to prioritize which countries and which indicators merit closer monitoring and further, causally identified evaluation.

The study also reports honestly where its design is most exposed to bias and unmodeled uncertainty. Classification accuracy is lower for non-OECD members, 0.605, than for OECD members, 0.708, consistent with the OECD-sourced environmental tax variable being more completely measured for OECD members, and region-level figures vary considerably, from 0.923 for Sub-Saharan Africa to 0.417 for the Middle East and North Africa, though the small number of test countries in some regions makes those estimates indicative rather than precise. The mean predictive entropy of the models’ class probabilities is highest for the medium emission category and for lower-coverage regions, a signal that the model’s explanations should be weighted less heavily for those observations. Sensitivity analyses show that excluding the ten highest-emitting countries, including Trinidad and Tobago, Brunei Darussalam, Luxembourg, Australia, the United States, Canada, Estonia, the Russian Federation, Kazakhstan and Czechia, raises accuracy to 0.707 for Random Forest and 0.690 for XGBoost, confirming that high-emitting, often small or resource-intensive economies are harder to classify correctly than the rest of the sample. Direct energy variables such as fossil fuel consumption or the energy mix were not available in the underlying dataset and could not be included; the authors recommend their addition as a priority extension, since energy use is a close economic and physical correlate of several of the predictors already in the model. In an era when artificial intelligence is increasingly deployed in climate governance, the study’s most durable contribution may be its demonstration that responsible AI in climate research means reporting where apparent accuracy is recognition rather than generalization, and that explainability techniques like SHAP, LIME and permutation importance are most valuable when they are used to verify, rather than assert, the stability of a model’s signal.

Subject of Research: Machine learning classifiers for national CO2 emission levels generalize better across time than across countries, with government effectiveness, urban population growth and environmental tax revenue the most stable predictors.

Article Title: A global machine learning framework for classifying national CO2 emission levels: responsible and explainable artificial intelligence for sustainable climate governance

Article References: Khawaja, M. A., & Shaikh, M. F. (2026). A global machine learning framework for classifying national CO2 emission levels: responsible and explainable artificial intelligence for sustainable climate governance. Frontiers in Artificial Intelligence, 9, Article 1943572. https://doi.org/10.3389/frai.2026.1943572

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1943572

Keywords: machine learning, CO2 emissions, explainable AI, climate governance, SHAP, XGBoost, environmental taxation, governance, global, machine, learning, framework