The study was a cross-sectional model-development and external-validation project using questionnaire data from two independent university samples in China. The development sample came from a single university, where raw questionnaire submissions numbered 1,392 records; after consent screening and removal of 118 duplicates, the cleaned dataset included 1,267 students. Outcome-specific analyses required complete symptom scores: depression-risk status was available for 1,096 students, of whom 102 (9.31%) were classified as positive cases, and anxiety-risk status was available for 1,095 students, with 66 positive cases (6.03%). Risk was defined by established questionnaire cutoffs rather than clinical diagnoses—a Patient Health Questionnaire-9 (PHQ-9) total of 10 or higher for depression and a Generalized Anxiety Disorder-7 (GAD-7) total of 10 or higher for anxiety, thresholds the authors note identify at least moderate symptom severity and should not be read as diagnoses.

The framework’s distinguishing feature is its construct-level design. Candidate predictors spanned demographic, anthropometric, lifestyle, digital-behavior, sleep-rhythm, dietary, body-image, social-support, and general health-status domains, selected a priori based on prior literature linking perceived social support, problematic smartphone use, circadian misalignment, body dissatisfaction, dietary habits, and physical activity with student mental health. These were organized into nested feature sets: Feature set A contained demographic and basic socioeconomic variables; set B added traditional health behaviors such as physical activity, sedentary time, sleep duration, smoking, alcohol, and diet; set C added emerging behavioral and psychosocial indicators and served as the primary model; and set D held more granular, correlated variables as a sensitivity model. Variables derived directly from the PHQ-9 and GAD-7 themselves, including self-harm items, were excluded from all predictor sets to avoid outcome leakage.

The primary benchmark compared logistic regression, random forest, and support vector machine classifiers, with the random forest selected for its discrimination profile and its support for descriptive model-level explanation. Internal validation used repeated stratified 5-fold cross-validation with 5 repeats, yielding 25 validation folds, with all preprocessing—including median imputation, standardization, and percentile-derived binary indicators—estimated within each training fold and applied to the held-out fold. The corrected pooled cross-validated results showed a clear gradient: for depression risk, feature sets A through D achieved AUROC values of 0.506, 0.657, 0.773, and 0.812, respectively; for anxiety risk, the corresponding values were 0.602, 0.634, 0.796, and 0.849.

After internal validation, the final Feature set C random-forest pipelines were refitted on the full development samples and applied to the external cohort—the Shanghai sample provided through the Shanghai Institute of Technology—without retraining, predictor reselection, hyperparameter optimization, recalibration, or threshold adjustment. Of the 1,983 raw external records, 1,530 remained after duplicate review and 1,468 complete records formed the validation sample, including 148 depression-risk cases (10.1%) and 79 anxiety-risk cases (5.4%). External discrimination held up well: the depression model reached an AUROC of 0.813 and a precision–recall area under the curve (PR-AUC) of 0.402 (95% CI, 0.325–0.491), while the anxiety model reached an AUROC of 0.809 and a PR-AUC of 0.205 (95% CI, 0.144–0.300). Because both outcomes were imbalanced, the authors emphasize that PR-AUC provides information beyond ROC analysis in low-prevalence settings.

Discrimination and calibration, however, told different stories. The depression model’s external Brier score of 0.090 (95% CI, 0.083–0.098) sat close to the prevalence-only baseline of 0.091, with a calibration intercept of −0.668 and slope of 1.401. Anxiety showed meaningful external miscalibration: its Brier score of 0.063 exceeded the prevalence-only baseline of 0.051, with an intercept of −1.227 and a slope of 1.102. The authors are explicit about what this means: the models support external discrimination and risk ranking, but they did not validate individual absolute-risk probabilities or clinical decision thresholds. Differences between the two university samples, including case-mix shift in age and several behavioral distributions, may partly explain the calibration drift, and local recalibration would be needed before any broader use.

Threshold behavior also mattered. At the default 0.5 threshold, the locked external models showed high specificity but limited recall—0.318 recall and 0.961 specificity for depression risk, and 0.241 recall and 0.970 specificity for anxiety risk—meaning many positive cases would be missed. At the fixed development-derived Youden thresholds of 0.226 and 0.251, which were not re-estimated in the Shanghai cohort, recall rose to 0.791 (specificity 0.689) for depression and 0.608 (specificity 0.828) for anxiety. The authors frame threshold choice as context-specific: prevalence-adjusted or utility-based cutoffs may be preferable depending on the relative costs of false negatives and false positives and on referral capacity.

Construct-level permutation importance, based on AUROC and summed across related variable representations, ranked perceived social support, body-image concern, and smartphone dependency as the top three constructs for depression risk, with breakfast skipping and social jetlag also prominent. For anxiety risk, body-image concern, perceived social support, and smartphone dependency led the ranking. Directionality summaries showed directionally consistent differences in predicted probabilities across categories of these constructs, including patterns involving lower social support and shorter sleep duration. The authors caution that because several dichotomized predictors were derived from continuous scores, these results should be read as construct-level signals rather than independent effects, and that the analyses characterize model behavior without establishing causal effects.

Sensitivity analyses reinforced the main findings. A cleaned variant of Feature set C that removed dichotomized indicators duplicating continuous scores produced similar external results for depression (AUROC 0.820) and anxiety (AUROC 0.807), and logistic-regression robustness models showed lower discrimination than the random forests. The more complex Feature set D model achieved even higher external AUROC values—0.845 for depression and 0.854 for anxiety—but was retained as a sensitivity model because of its complexity. A reciprocal validation, in which the roles of the two samples were reversed, showed broadly consistent discrimination (AUROC 0.760 for depression and 0.823 for anxiety).

The authors are careful about limits. The design was cross-sectional, all predictors and outcomes were self-reported within the same survey, raising possible common-method bias; the study was restricted to Chinese university settings; and fewer than 100 anxiety-risk cases make anxiety findings preliminary and potentially unstable, particularly for PR-AUC, calibration, and threshold metrics. Sleep-related predictors had roughly 27.5% missingness in the development cohort, and a missing-not-at-random mechanism cannot be excluded, since students with more irregular lifestyles may have been more likely to omit those items. Participants with complete outcome data also differed in age and perceived social support from those excluded, so selection bias cannot be ruled out. The team concludes that the framework is a research-oriented risk-stratification tool for prevention and hypothesis generation—not a diagnostic system, automatic classifier, or implementation-ready decision-support tool—and that further evaluation, including local recalibration, is needed before wider implementation.

Subject of Research: External validation of machine-learning risk stratification for depression and anxiety among college students using behavioral and psychosocial indicators

Article Title: Behavioral and psychosocial indicators for depression and anxiety risk stratification among college students: development and external validation of a machine-learning-assisted framework

Article References: Mao, K., He, Y., Yin, X., Lin, L., Zheng, Y., & Hu, G. (2026). Behavioral and psychosocial indicators for depression and anxiety risk stratification among college students: development and external validation of a machine-learning-assisted framework. Frontiers in Psychology, 17, 1972318. https://doi.org/10.3389/fpsyg.2026.1972318

Image Credits: AI Generated

DOI: 10.3389/fpsyg.2026.1972318

Keywords: college students, depression, anxiety, machine learning, risk stratification, external validation, random forest, PHQ-9, GAD-7, calibration, smartphone dependency, social support