Simple Models Beat Fancy AI in Test of Fair Credit Scoring for the Underbanked

Some 1.4 billion adults worldwide remain unbanked, and most of them live exactly where formal credit histories do not exist. In the Middle East and North Africa, microfinance institutions fill part of that gap, lending small amounts to borrowers whose financial lives unfold in cash, mobile money, and community savings groups rather than in bank statements. A new open-access study in Discover Artificial Intelligence asks a deceptively simple question about this world: when algorithms decide who is likely to repay a microloan, does the most sophisticated artificial intelligence actually do a better job than a transparent, decades-old statistical method? The answer, backed by unusually rigorous testing, is no—and that finding could reshape how lenders in emerging markets think about deploying AI.

The research team, led by Yazan Taher Shawabkeh of Middle East University in Amman, Jordan, together with colleagues at the National Agriculture Research Center, analyzed a de-identified dataset of 2,500 resolved microloan applications from participating microfinance institutions in the MENA region, covering loans originated between January 2022 and December 2023. Default was defined as being 90 or more days past due, and 682 of the loans—27.28 percent of the sample—ended in default. Because the sample was deliberately outcome-stratified to ensure enough defaults for modeling, that figure reflects the study design rather than true portfolio default rates, a caveat the authors state plainly. The dataset included 15 predictors spanning demographics, loan terms, institutional risk ratings, and three so-called non-traditional indicators: mobile money transaction frequency, utility payment behavior, and savings group membership.

Against this data, the researchers pitted nine supervised classifiers. The lineup ranged from classical benchmarks—logistic regression, decision tree, random forest, gradient boosting, support vector machine, and k-nearest neighbors—to the modern gradient-boosting frameworks that dominate contemporary credit-scoring research: XGBoost, LightGBM, and CatBoost. Each model was tuned by grid search with five-fold stratified cross-validation on a training set of 2,000 loans, then evaluated on a held-out test set of 500. Performance was measured with ROC-AUC and precision-recall AUC, with 95 percent confidence intervals estimated from 1,000 bootstrap resamples, and differences between models were formally tested using DeLong tests rather than eyeballed from point estimates.

The headline result is a statistical dead heat. CatBoost achieved the highest test-set ROC-AUC at 0.783, with a confidence interval of 0.741 to 0.825, but regularized logistic regression sat essentially on top of it at 0.780—a difference the DeLong test put at p = 0.59, indistinguishable from chance. No gradient-boosting model significantly outperformed the logistic benchmark, and in repeated cross-validation on the training data, logistic regression actually posted the numerically best mean AUC at 0.799. Only the humble decision tree was significantly beaten. The pattern echoes what large benchmarking studies in consumer lending have long hinted at: the ensemble advantage shrinks or vanishes on small, low-dimensional tabular problems, which is precisely the regime a microfinance portfolio with thin predictor sets occupies.

Interpretability was not treated as an afterthought but as an object of study in its own right. The team used SHAP—Shapley additive explanations, a technique grounded in cooperative game theory that attributes each prediction to individual features—to open up the black box. Three variables dominated: the institution’s internal Credit Score Category, the presence or absence of a utility payment record, and the borrower’s Repayment History on previous loans. Crucially, the researchers validated the explanations themselves. The global importance ranking proved extraordinarily stable under 30 bootstrap resamples of the test set, with a mean Spearman rank correlation of 0.999, and the rankings were strongly consistent across model families, with correlations of 0.95 to 0.98 among the tree ensembles. In other words, the explanations reflect genuine signal in the data, not quirks of one particular algorithm.

The study’s most methodologically pointed contribution concerns alternative data. Advocates of fintech-driven financial inclusion often cite high feature-importance scores as proof that mobile money records and savings-group membership expand credit access. The authors instead ran an explicit ablation experiment: they re-estimated the two best model families on a traditional-only feature set and compared the results on held-out data. Adding the three non-traditional indicators lifted CatBoost’s ROC-AUC from 0.775 to 0.783—a gain of just 0.008, with a confidence interval spanning −0.010 to 0.027 and a DeLong p-value of 0.36. The indicators carry real signal, as their strong SHAP contributions and bivariate associations show, but that signal overlaps heavily with what institutional risk variables already capture. Within-model importance, the study argues, is simply not evidence of incremental value.

Fairness auditing revealed the study’s most consequential nuance. On selection-rate criteria, the final CatBoost model looked exemplary: disparate impact ratios exceeded the 0.80 four-fifths heuristic for gender (0.956), geographic region (0.961), and all four gender-by-region intersections (minimum ratio 0.881), while equal-opportunity and predictive-parity gaps were small and group-level calibration was reasonable. But the false positive rate—the share of actual defaulters the model wrongly predicted would repay—was 0.680 for rural applicants against 0.492 for urban applicants, a gap of 0.188. That asymmetry means the model’s errors concentrate as excess credit extended to rural borrowers who subsequently default, a pattern with implications for institutional risk exposure and potentially for over-indebtedness among rural clients. A single-metric fairness audit would have certified the model without qualification; the multi-metric audit surfaced a deployment-relevant risk that parity statistics alone conceal.

The authors are careful about scope. Because the dataset contains only granted loans with observed outcomes, the fairness analysis characterizes model behavior on the approved-borrower population and cannot speak to approval decisions across the full applicant pool—the well-known selective-labels problem. The 0.80 disparate-impact benchmark, they note, originates in U.S. employment-selection guidance and functions as a screening heuristic, not a universal regulatory threshold. A sensitivity analysis excluding the two institutionally derived risk variables showed performance dropping from 0.783 to 0.732 ROC-AUC, bounding but not eliminating concerns about residual leakage from variables whose internal construction could not be externally audited. All classification metrics were computed at a fixed 0.50 threshold without cost-sensitive optimization, and the authors stress that operational deployment would require threshold selection with fairness metrics re-audited at the chosen operating point.

The practical implications are strikingly counterintuitive for an era of AI maximalism. When a transparent, well-calibrated logistic regression matches the discrimination of a state-of-the-art ensemble—and in this study even achieved the best calibration, with a Brier score of 0.162 and expected calibration error of 0.040—the argument for deploying black-box models in high-stakes lending collapses. The findings support what interpretability researchers have long argued: in data-scarce regimes, model selection should be governed by explainability, calibration, and governance rather than by leaderboard chasing. For microfinance institutions and their supervisors, the recommended path is a disciplined one—transparent models with SHAP-style reporting, multi-metric and intersectional fairness audits repeated at every threshold change, and alternative-data initiatives justified by incremental-contribution evidence in the target population rather than by importance rankings. The genuine frontier for financial inclusion, the authors suggest, lies not in more complex models but in richer data for genuinely thin-file borrowers, especially first-time applicants for whom repayment history and internal scores say nothing at all.

Subject of Research: Explainable machine learning models for AI enabled credit scoring and financial inclusion in emerging markets

Article Title: Explainable machine learning models for AI enabled credit scoring and financial inclusion in emerging markets

Article References: Shawabkeh, Y. T., Bani Atta, A. A., Aldarabah, K. A., Marei, A., Al-Qur’an, A. B., & Alofishat, R. (2026). Explainable machine learning models for AI enabled credit scoring and financial inclusion in emerging markets. Discover Artificial Intelligence, 6(1), Article 1418. https://doi.org/10.1007/s44163-026-02392-9

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02392-9

Keywords: Explainable, machine, learning, models, enabled, credit, scoring, financial, inclusion, emerging, markets, scientific research