Questionnaire Model Ranks HIV Testing Encounters for Extra Support

Every HIV testing encounter among men who have sex with men carries a different probability of returning a positive result, yet most services treat each visit identically. A team at Frontiers in Public Health has now shown that a simple pre-test questionnaire, processed through a machine-learning model, can rank those encounters so that scarce additional support reaches the visits most likely to need it. The study, published in October 2026, subjected the model to two of the harshest tests a prediction model can face: validation on data from later years and validation in an entirely different city in Shandong Province. The model held up remarkably well in both settings, suggesting that routine surveillance data already collected before an HIV result is known could help services plan optional counseling, prevention assessment, peer support, or follow-up when capacity is limited.

The research team drew on de-identified records from an MSM sentinel surveillance program in Nantong, a coastal city in Jiangsu Province, China. Sentinel surveillance programs routinely collect structured demographic and behavioral information from men who have sex with men through questionnaires, capturing items such as age, education, sexual behavior, condom use, and service-use history. These data have traditionally been used for aggregate reporting rather than for individual-level decision support. The investigators saw an opportunity: if the questionnaire responses collected before an HIV result is known could predict which encounters would return a confirmed positive result, services could use that ranking to prioritize optional additional support without changing who gets tested or how.

The cohort construction followed strict rules to prevent information leakage. From 2,460 testing encounters recorded in Nantong between 2020 and 2023, the team excluded 34 encounters because the current outcome or calendar year could not be resolved, leaving 2,426 encounters. Records belonging to the same person were identified and only the first retained testing encounter was kept, so each participant contributed exactly one record. The final Nantong dataset contained 2,426 encounters with 100 confirmed HIV-positive events. These were split by calendar year: 2020 to 2022 formed the development cohort, and 2023 formed the temporal validation cohort. A separate surveillance dataset from an unnamed city in Shandong Province, spanning 2022 to 2023, served as the geographic validation cohort. The external city and data provider were not identified because the data were used under an inter-institutional agreement.

The outcome was a confirmed HIV-positive result during the same encounter, representing the final result after the complete testing process. A screening-positive result alone was never coded as an event unless confirmatory evidence was available. The predictors were restricted to questionnaire items collected before the result was known, including demographic characteristics, sexual behavior, condom use, and service-use history. Items collected after the result, such as post-test counseling records, were excluded. The team also applied a manual feature-review step to guard against proxy leakage: one questionnaire item was flagged as a potential proxy for the outcome and removed from the primary model, because its coding pattern suggested it might indirectly encode result information rather than genuine pre-test behavior.

The final model was a gradient-boosted decision tree ensemble trained with XGBoost, a widely used implementation of gradient boosting that builds trees sequentially, each new tree fitting the residual errors of the ensemble so far. Gradient boosting works by fitting trees to the negative gradient of a loss function, typically logistic loss for binary classification, so that each addition reduces the ensemble’s overall prediction error. The team tuned hyperparameters through five-fold cross-validation within the development cohort, selecting the configuration that maximized the area under the receiver operating characteristic curve. Preprocessing parameters, including imputation values and encoding categories, were estimated only from training records within each cross-validation fold, a discipline that prevents held-out records from influencing preprocessing. The locked model, including its preprocessing pipeline and hyperparameters, was then frozen and applied unchanged to both validation cohorts.

Validation results exceeded expectations. In the temporal cohort, the locked model achieved an area under the precision-recall curve of 0.437, compared with 0.318 in development cross-validation. In the geographic cohort, the precision-recall area rose further to 0.522. The area under the receiver operating characteristic curve reached 0.855 in temporal validation and 0.877 in geographic validation, up from 0.821 in development. The Brier score, which measures the mean squared difference between predicted probabilities and actual outcomes, was 0.061 in temporal validation and 0.055 in geographic validation, both better than the development value of 0.068. These improvements likely reflect the later cohorts’ higher event prevalence, which was 4.6 percent in the temporal cohort and 5.7 percent in the geographic cohort, compared with 3.9 percent in development.

Calibration, the agreement between predicted probabilities and observed event rates, showed a more nuanced picture. The calibration slope, which assesses whether the model over- or underestimates risk across the full range of predictions, was 1.08 in temporal validation, close to the ideal value of 1. In geographic validation, the slope was 0.92, indicating mild overestimation of risk in that setting. The calibration-in-the-large statistic, which measures whether the model’s average prediction matches the observed event rate, was 0.03 in temporal validation and -0.11 in geographic validation. The geographic overestimation likely reflects differences in questionnaire implementation, event prevalence, or population characteristics between the two cities. The team emphasized that the model was designed for ranking encounters rather than for producing exact individual probabilities, so mild miscalculation does not undermine its intended use.

The most practically relevant analyses examined how many confirmed events the model would capture if services prioritized the highest-scoring encounters under different workload constraints. At a 10 percent prioritization level, meaning the top-scoring 10 percent of encounters were flagged for optional additional support, the model captured 46.7 percent of temporal events and 52.0 percent of geographic events. At 20 percent prioritization, the capture rates rose to 63.3 percent and 68.0 percent, respectively. At 30 percent, the rates reached 76.7 percent and 80.0 percent. The observed positivity among flagged encounters was consistently higher than the overall cohort positivity: at 10 percent prioritization, the flagged groups had event rates of 9.3 percent in temporal validation and 10.4 percent in geographic validation, compared with 4.6 percent and 5.7 percent overall. This concentration means that prioritizing the highest-scoring encounters would direct additional support toward visits with roughly double the average event rate.

The study’s design choices reflect current best practice for prediction model research. Reporting followed the TRIPOD+AI statement, an international guideline for transparent reporting of clinical prediction models using artificial intelligence. Risk of bias was assessed with PROBAST+AI, the companion tool for evaluating prediction model risk of bias and applicability. The team explicitly framed the model as a decision-support ranking tool rather than a diagnostic instrument, and they stated that it should never restrict access to HIV testing or standard services. The model is intended to help services plan optional counseling, prevention assessment, peer support, or follow-up when capacity is limited, not to determine who receives testing.

Several limitations qualify the findings. The development and validation data came from only two Chinese cities, so generalizability to other regions or service models is unknown. The geographic validation city was not identified, which limits independent scrutiny of that cohort. The temporal validation covered a single later year, and the geographic validation covered two years. The analysis used only the first retained encounter per participant, so the model’s performance on repeat visits is untested. The mild geographic miscalculation suggests that the model would benefit from local recalibration before deployment in a new setting. The team also noted that the later cohorts’ higher event prevalence contributed to the improved validation metrics, so the model’s apparent gains partly reflect changing epidemiology rather than pure model quality. Despite these constraints, the study demonstrates that structured questionnaire data already collected before an HIV result is known can support encounter-level prioritization across time and geography, offering a template for other services seeking to allocate optional additional support more efficiently.

Subject of Research: Development and external validation of a questionnaire-based machine-learning model to prioritize additional support during HIV testing encounters among men who have sex with men.

Article Title: A questionnaire-based model to prioritize additional support during HIV testing encounters among men who have sex with men: temporal and geographic external validation

Article References: Zhao, J., Yang, Y., Zhu, P., Zheng, S., Xu, X., & Zhou, X. (2026). A questionnaire-based model to prioritize additional support during HIV testing encounters among men who have sex with men: temporal and geographic external validation. Frontiers in Public Health, 14, Article 1944057. https://doi.org/10.3389/fpubh.2026.1944057

Image Credits: AI Generated

DOI: 10.3389/fpubh.2026.1944057

Keywords: HIV, machine learning, men who have sex with men, sentinel surveillance, external validation, model calibration, prediction model, public health, HIV testing, questionnaire, questionnaire-based, model