Machine Learning Models Classify University Faculty Self-Reported Digital Competence in Ecuador

A quantitative investigation conducted across two public universities in Ecuador has evaluated the capacity of supervised machine learning algorithms to classify faculty members into sample-defined tiers of self-reported digital competence. The research, carried out by Jenniffer Sobeida Moreira-Choez and an interdisciplinary team, examined whether algorithms representing diverse computational paradigms could reliably distinguish between low, medium, and high levels of digital proficiency based on contextual, demographic, and professional attributes. Rather than relying on standard linear regressions alone, the investigators developed a structured analytical workflow to determine how non-academic variables, ranging from institutional affiliation to teaching experience, contribute to self-perceptions of technological skill across higher education environments. The study focused specifically on academic personnel at the Technical University of Manabí and the State University of Milagro, establishing a predictive framework that highlights internal classification accuracy while deliberately accounting for the methodological boundaries of survey-based measurement.

The study design was cross-sectional, quantitative, and non-experimental, relying on an analytical sample of 315 faculty members who completed a comprehensive 300-item questionnaire. This instrument was structured around four core domains of higher education practice, allocating 75 items each to teaching, research, outreach, and academic management. The conceptual organization was grounded in established educational frameworks, namely the European Framework for the Digital Competence of Educators and the Technological Pedagogical Content Knowledge model. However, the researchers explicitly treated these areas as theoretically defined content categories rather than empirically confirmed latent dimensions, noting that confirmatory factor analysis and structural equation evaluations were not conducted. Faculty responses were recorded using Likert-type scales, yielding an overall internal consistency coefficient of 0.997 as measured by Cronbach’s alpha. Within specific domains, reliability estimates remained similarly high, ranging from 0.975 to 0.998 across both institutions, though the investigators emphasized that such elevated values can partly reflect item redundancy resulting from the substantial length of the survey.

To establish the classification target, the composite scores from the self-report instrument were converted into an ordinal outcome comprising low, medium, and high competence categories. This stratification was executed using percentile thresholds calculated directly from the complete analytical sample, producing a pronounced class distribution where 230 faculty members were categorized as high, 65 as medium, and only 20 as low. The authors underlined that these categories represent relative rankings within the specific observed sample rather than externally validated, criterion-referenced, or performance-tested benchmarks. A acknowledged reproducibility limitation of this outcome-construction process is that the exact numerical percentile cutoffs were not retained in the archived study materials, precluding independent researchers from re-creating the target boundaries directly from raw score files. Furthermore, because threshold creation occurred prior to partitioning the data, the resulting labels were treated as fixed parameters during downstream modeling rather than being re-estimated dynamically within validation subsets.

A critical component of the methodology involved preventing information leakage between the feature matrix and the target variable. To maintain predictive validity, all four domain scores, individual questionnaire responses, and derived cluster metrics were strictly excluded from the set of candidate predictors. Instead, the final input space was restricted to seven variables that operated independently of the outcome’s mathematical formulation: participant age, gender, highest academic degree attained, total teaching experience in years, employment contract type, disciplinary area of knowledge, and institutional affiliation. Within this configuration, institutional affiliation was integrated solely as a contextual predictor rather than an analytical target. Data preprocessing procedures, including mean imputation for continuous missing values and mode imputation for categorical gaps, were executed strictly within the training partitions of each cross-validation fold to guarantee that information from held-out validation cases did not influence training parameters.

Model training and evaluation were carried out using a stratified 10-fold cross-validation scheme implemented in the Orange 3 computational environment with a fixed random seed of 42. Hyperparameter optimization was conducted exclusively on training folds using embedded grid-search strategies, preventing validation observations from contaminating model tuning. The analytical framework benchmarked nine distinct supervised learning algorithms: k-nearest neighbors, logistic regression, decision trees, random forests, naive Bayes, gradient boosting, support vector machines, stochastic gradient descent, and multilayer neural networks. Predictions generated across the ten held-out validation folds were subsequently pooled, ensuring that each of the 315 individuals contributed exactly one out-of-fold prediction to the primary confusion matrices. The authors documented that these pooled outputs reflect internal resampling assessments across shared institutional sources and do not constitute an independent external test cohort.

Across the pooled out-of-fold evaluations, eight of the nine algorithms achieved an identical classification pattern, correctly assigning 314 of the 315 faculty records. This outcome corresponded to an overall accuracy of 0.9968, accompanied by a Wilson 95 percent confidence interval spanning from 0.982 to 0.999, and a Matthews correlation coefficient of 0.9924. For these eight models—comprising k-nearest neighbors, logistic regression, decision trees, random forests, gradient boosting, support vector machines, stochastic gradient descent, and neural networks—the sole classification error occurred when a single participant belonging to the medium category was assigned to the high category. In contrast, naive Bayes exhibited lower discriminative capability, correctly classifying 294 of 315 participants to achieve an overall accuracy of 0.9333 with a Wilson confidence interval between 0.900 and 0.956, and a Matthews correlation coefficient of 0.8382. Naive Bayes recorded 14 medium-competence observations classified as high, alongside seven high-competence observations classified as medium.

An examination of class-specific performance metrics provided deeper insight into algorithmic behavior under conditions of severe sample imbalance. For the eight top-performing architectures, precision reached 1.000 for the medium category with a 95 percent confidence interval of 0.943 to 1.000, and recall stood at 0.985 with an interval of 0.918 to 0.997, yielding an F1 score of 0.992 across 65 cases. For the high category, precision was 0.996 with a confidence interval of 0.976 to 0.999, while recall reached 1.000 with an interval of 0.984 to 1.000, producing an F1 score of 0.998 across 230 cases. Naive Bayes registered lower precision and recall within the medium class, recording 0.879 and 0.785 respectively, and an F1 score of 0.829. While all nine evaluated models correctly identified all 20 participants in the low-competence tier, achieving nominal precision and recall values of 1.000, the corresponding Wilson 95 percent confidence intervals were wide, spanning from 0.839 to 1.000, demonstrating substantial uncertainty surrounding minority-class predictions.

Complementary inferential analyses revealed distinct descriptive patterns between the two institutions when digital competence domains were evaluated individually. Contingency table assessments demonstrated statistically significant differences in self-reported competence distributions across all four operational areas, with Pearson chi-square tests yielding p-values below 0.001. The magnitude of these associations varied notably, with Cramér’s V reaching 0.301 for academic management, 0.428 for research, 0.512 for teaching, and 0.676 for outreach. In the teaching domain, 90.1 percent of faculty at the Technical University of Manabí placed in the high category compared with 42.9 percent at the State University of Milagro. Similarly, 58.6 percent of Manabí faculty reported high research competence compared to 28.6 percent at Milagro. Bivariate correlation analyses showed positive associations between research and teaching competence scores within both institutions, registering r equals 0.51 at Manabí and r equals 0.65 at Milagro, though a Fisher r-to-z transformation confirmed that this institutional difference in correlation strength was not statistically significant.

To interpret how the models processed input attributes without relying on opaque decision mechanisms, the investigators applied Shapley Additive Explanations across the leakage-controlled feature set. The global analysis indicated that institutional affiliation exerted the largest overall mean absolute contribution within the fitted model, followed by academic degree, teaching experience, and chronological age, whereas gender and contract type demonstrated comparatively lower predictive importance. Class-specific explanations further highlighted structural variation in feature reliance depending on the target level being predicted. When assigning low competence classifications, age emerged as the most influential feature, whereas academic degree predominated in assignments to the medium category, and institutional affiliation was the primary driver for high competence classifications. The researchers cautioned that these SHAP metrics describe internal model mechanics and predictive associations rather than verified causal mechanisms or independent structural determinants of competence.

In detailing the overall findings, the authors stressed several fundamental methodological constraints that govern how the results must be applied. Foremost among these is the self-reported character of the survey instrument, which captures faculty perceptions rather than directly observed digital performance, practical problem solving, or artifact evaluation. Consequently, the high internal discrimination achieved by the models cannot be interpreted as evidence of demonstrated pedagogical competence or generalizable prospective prediction. Furthermore, the absence of retained fold-level probability arrays precluded formal calibration testing and multiclass area-under-the-curve reconstruction, while the unrecorded percentile cutoffs restrict reproducibility. The investigators concluded that future research must incorporate objective performance metrics, evaluate measurement invariance across institutions, and implement external validation in independent academic cohorts to determine whether machine learning frameworks can reliably inform faculty development initiatives.

Subject of Research: Explainable machine learning prediction of faculty digital competence

Article Title: Explainable machine learning for predicting self-reported digital competence levels among university faculty

Article References: Moreira-Choez, J. S., Bueno-Fernández, M. M., Quirino-Rodríguez, L. G., Tostado-Ramírez, M. I., Ultreras-Rodríguez, A., Sabando-García, Á. R., Nieto-Sánchez, Z. C., Zambrano-Alcívar, M. V., & García-Saltos, C. D. (2026). Explainable machine learning for predicting self-reported digital competence levels among university faculty. Frontiers in Education, 11, Article 1877791. https://doi.org/10.3389/feduc.2026.1877791

Image Credits: AI Generated

DOI: 10.3389/feduc.2026.1877791

Keywords: machine learning, digital competence, higher education, faculty development, explainable AI, educational analytics, cross-validation, SHAP analysis, Explainable, machine, learning, predicting