Two-stage model predicts developers’ perceptions of AI capability across survey years

A theory-anchored, two-stage statistical model predicted how professional developers rated the ability of their AI tools to handle complex programming tasks in a later survey year, according to a repeated cross-sectional study of the official 2024 and 2025 Stack Overflow Developer Surveys. The analysis included 22,757 employed professional developers in 2024 and 17,021 in 2025, and the source-only evaluation trained every preprocessing rule and model on 2024 data before making predictions for 2025. Called Theory-Anchored Residual Surface learning, or TARS, the method first modeled the relationship between trust in AI accuracy, favorability toward AI tools, and perceived capability. It then held those coefficients fixed while using work and demographic information to predict the error remaining in the first stage. Under this prospective protocol, TARS explained 44.5% of the observed 2025 outcome variance, with a 95% confidence interval from 0.434 to 0.456, and achieved a mean absolute error of 0.742 on the five-category outcome scale.

The central challenge was not simply to predict an answer, but to preserve the psychological meaning of the model while improving its forecasts. Trust in output accuracy and favorability toward AI represent related but distinct appraisals: a developer might believe a tool’s output is accurate without welcoming its use, or might like using AI while remaining uncertain about its reliability. In a conventional one-stage regression, adding occupation, age, country, AI exposure, or perceived job threat can change the estimated coefficients for trust and favorability. Retuning the entire model for a different error measure can also alter those coefficients. TARS separates the two functions. Its Stage 1 is a polynomial ridge regression containing trust, favorability, and their interaction and quadratic terms; Stage 2 is another ridge regression that predicts Stage 1 residuals from 66 encoded context variables. Because the context matrix and Stage 2 tuning choices do not enter the Stage 1 equation, the theoretical relationship remains fixed even when the contextual predictors or prediction loss change.

The study used a single survey question as its outcome: “How well do the AI tools you use in your development workflow handle complex tasks?” Responses ranged from 1, meaning “very poor,” to 5, meaning “very well.” The authors analyzed the ordered item directly and supplemented continuous error measures with quadratic weighted kappa, which assesses agreement between predicted and observed categories after predictions are rounded to the scale. Trust in AI output accuracy and favorability toward using AI tools were also measured with matching five-category items in both years. The analysis identified itself as observational: it estimated associations and predictive performance, not causal effects or changes within individual developers. The samples consisted of different people in each survey wave, and the reported cross-year differences in outcome distributions cannot establish a population trend over time.

Sample construction produced an important boundary on the findings. Starting from 65,437 retained records in the official 2024 file and 49,191 in 2025, the researchers selected respondents who identified as developers by profession, met the respective year-specific employment criterion, and answered all three required items with evaluable responses. The 2024 rule required full-time employment, whereas the redesigned 2025 item required employment more generally. Missing responses, an “I don’t know” outcome category in 2025, and an “Unsure” favorability response were excluded. The final samples represented 34.78% of the official 2024 records and 34.60% of the 2025 records. No respondents were removed as multivariate outliers. Complete-item selection may create selection or collider bias if AI attitudes, tool use, occupation, or perceived capability influence which questions people answer, so the results apply most directly to eligible complete responders rather than to all developers or workers.

Across both years, higher trust in AI accuracy and greater favorability toward AI were associated with higher perceived capability. Trust had the stronger association. The response-surface analysis quantified this pattern: the line-of-congruence slope, describing the change when trust and favorability rise together, was 0.702 in 2024, with a 95% confidence interval of 0.684 to 0.720, and 0.792 in 2025, with an interval of 0.775 to 0.809. The line-of-incongruence slopes were also positive, 0.249 in 2024 and 0.210 in 2025, indicating that trust carried greater weight when the two ratings differed. The corresponding curvature intervals included zero in both years, so the analysis did not provide evidence of a general mismatch penalty. The directional comparison between trust and favorability was not preregistered and was therefore treated as exploratory, while the jointly increasing relationship was the principal theoretical component carried into TARS.

The strict 2024-to-2025 evaluation showed small but consistent advantages for the two-stage design. TARS produced the highest R² point estimate and the lowest RMSE and MAE point estimates among the source-only models tested. Its RMSE was 0.911, with a 95% confidence interval of 0.902 to 0.921, while quadratic weighted kappa was 0.584, with an interval of 0.574 to 0.592. Compared with one-stage response-surface ridge regression, the paired R² gain was 0.00179, with an interval of 0.00131 to 0.00221; MAE fell by 0.00643, and kappa rose from 0.573 to 0.584. The model also outperformed a trust-only ridge, a context-only ridge, elastic net, random forest, histogram gradient boosting, and a multilayer perceptron on the main point-estimate comparison. These were modest differences rather than a dramatic separation from the strongest baselines.

Model order provided further evidence that the two stages did different jobs. A one-stage model with separate penalties for the trust–favorability and context terms reached an R² of 0.444 and an MAE of 0.748, compared with 0.445 and 0.742 for TARS. A context-first two-stage model, which fitted context before modeling the remaining trust and favorability relationships, was weaker, with an R² of 0.440 and MAE of 0.749. Removing the context correction entirely reduced R² to 0.435 and increased MAE to 0.757, indicating that contextual information improved prediction beyond the theoretical terms alone. Changing the set of context variables produced exactly identical TARS Stage 1 coefficients across four configurations: work and AI-use context, demographic context, all non-geographic context, and the complete set. A comparable one-stage model changed a Stage 1 coefficient by as much as 0.0202 when context changed, while all 16 paired performance intervals across the tested configurations favored TARS.

The researchers also tested whether the second stage could target absolute category error without disturbing the first. TARS-RMSE and TARS-MAE reused the same polynomial Stage 1 model, trust–favorability coefficients, and Stage 1 penalty; only the Stage 2 ridge penalty and its contribution to the final score were retuned. The maximum absolute difference among the Stage 1 coefficients was zero. On the 2025 target, the MAE-tuned version reduced MAE by 0.00134 relative to the RMSE-tuned version, with a 95% confidence interval of 0.00102 to 0.00168, while differences in R² and RMSE were small and their intervals included zero. This demonstrates the separation of concerns directly: an organization can emphasize squared error or mean absolute error in the contextual correction while retaining the same estimated trust and favorability relationship for interpretation.

Generalization checks and computational results strengthened the practical picture, although they did not eliminate the study’s observational limits. TARS had lower MAE point estimates than one-stage response-surface ridge regression in all 13 audited target groups, comprising five role families and eight country groupings, and every paired MAE interval favored TARS. In a tougher leave-one-group-out analysis, the evaluated role or country group was removed before source tuning and fitting, so it contributed no labeled members to model development. Across 13 such tasks, TARS-MAE still produced lower MAE than an independently MAE-tuned one-stage model, with all paired intervals below zero; the median reduction was 0.00793. Its RMSE was lower in 11 tasks, with 10 intervals excluding zero. The full preprocessing, fitting, and prediction pipeline for 19,961 target rows had a median runtime of 0.312 seconds in the audited environment, compared with 0.294 seconds for one-stage ridge regression and substantially longer times for the evaluated nonlinear ensembles.

The result offers a compact method for repeated monitoring surveys, but it does not establish that AI tools objectively perform better or worse, nor that trust causes favorable attitudes or perceived capability. The measures are single self-reports collected at the same time, their temporal order is unknown, and the developers were self-selected through Stack Overflow’s recruitment channels. The two years also differed in some response formats, including the employment item and a non-evaluable 2025 outcome category, so the lower mean capability rating in 2025—2.666 compared with 2.798 in 2024—may reflect respondent composition, selection, or instrument changes as well as different appraisals. The strongest supported conclusion is narrower and still notable: when an interpretable trust–favorability relationship must remain stable, a source-only two-stage model can use contextual information to improve prediction of a future survey wave while preserving that relationship. For organizations monitoring developer attitudes toward AI, TARS provides a transparent, fast, and auditable option, while its aggregate appraisal scores should not be repurposed for employee selection or performance management.

Subject of Research: Source-only cross-year prediction of developers’ perceived AI capability using trust, favorability, and contextual variables.

Article Title: Source-only cross-year prediction of perceived AI capability with a theory-anchored, loss-specific two-stage model

Article References: Wang, D., & Song, Y. (2026). Source-only cross-year prediction of perceived AI capability with a theory-anchored, loss-specific two-stage model. Frontiers in Psychology, 17, Article 1867910. https://doi.org/10.3389/fpsyg.2026.1867910

Image Credits: AI Generated

DOI: 10.3389/fpsyg.2026.1867910

Keywords: artificial intelligence, developer surveys, prediction, trust in AI, favorability, cross-year transfer, ridge regression, response surface, Stack Overflow, Source-only, cross-year, perceived