The impact of missing data on the validity of study estimates depends heavily on why the data are missing. Pregnant women from lower socioeconomic backgrounds tend to access antenatal care later in pregnancy and enroll later in research studies, which means they accumulate more missing data early in pregnancy. This is particularly problematic because socioeconomic status is itself an important determinant of drinking behavior during pregnancy. Women also modify their consumption after recognizing they are pregnant, and the timing of that recognition varies widely. Some stop or reduce drinking immediately upon learning of the pregnancy, while some heavy drinkers continue to binge in the first trimester or maintain heavy drinking throughout gestation. Standard approaches to handling missing values, such as last value carry forward or mean imputation, are poorly suited to this setting because drinking behavior during pregnancy varies by gestational timing and by day of the week, and because the pattern of missingness is itself linked to drinking recency.

A team of researchers led by Ayesha Sania of Columbia University Irving Medical Center and the New York State Psychiatric Institute, working with colleagues at NYU Grossman School of Medicine, Avera Health, the University of South Dakota, and Stellenbosch University in Cape Town, has now demonstrated a machine learning solution to this problem. Writing in Advances in Drug and Alcohol Research, the team applied the k-nearest neighbor algorithm, or k-NN, to impute missing daily alcohol consumption data from the Safe Passage Study, a prospective cohort investigation conducted by the Prenatal Alcohol and SIDS and Stillbirth Network. The Safe Passage Study enrolled 11,892 pregnant women between August 2007 and January 2015 from antenatal clinics in the Northern Plains of the United States and in Cape Town, South Africa, with 11,083 women included in the imputation analysis. Participants were eligible if they were pregnant with one or two fetuses, aged 16 years or older, at least six weeks gestational age at recruitment, and spoke English or Afrikaans. Women were followed throughout pregnancy and for one year postnatally, with ethical approval obtained from institutional review boards including Stellenbosch University, Sanford Health, the Indian Health Service, and participating Tribal Nations.

Alcohol exposure data in the Safe Passage Study were collected using a modified version of the Timeline Followback method, a validated self-report instrument in which participants report drinking on their last known drinking day and then for the 30 days prior. The instrument captured remarkable detail: types and number of drinks, container sizes, the amount of ice in each drink, how many people shared a drink, and the duration of drinking episodes. These data were converted into estimates of total alcohol consumed and the number of standard drinks on each reported drinking day. Yet the design of the method inherently generates a variable amount of missing data per participant. Women who never drank, or whose last drinking day preceded their last menstrual period, had complete records. Women who drank but quit within 30 days of a collection period had few or no gaps. But women who continued to drink and reported their most recent 30 days of intake left unrecorded intervals stretching back toward the start of pregnancy. Frequent drinkers consequently accumulated more missing days than infrequent drinkers, meaning that simple summation of reported drinks would systematically underestimate true consumption and bias any analysis built on that metric.

The k-NN algorithm addresses this challenge through pattern recognition rather than statistical modeling. As a non-parametric method, it makes no distributional assumptions about the underlying data, an advantage given that alcohol consumption during pregnancy is highly skewed, with most drinking concentrated in the first trimester before pregnancy recognition. The algorithm works by measuring the similarity between the drinking pattern of a participant with missing data and the patterns of other participants in the database. Similarity is quantified using cosine distance, which captures the shape of a drinking pattern regardless of its absolute magnitude. Once the k most similar participants are identified, the weighted average of their drinking values for the missing day serves as the best estimate, with nearer neighbors exerting greater influence than distant ones. The team also scaled imputed values to each individual’s consumption level, ensuring that a participant with a similar pattern but heavier overall intake was not matched to a lighter drinker’s absolute values.

Data preparation required careful engineering. The team converted the records so that each participant occupied a single row, with drinking values as separate variables for each day from 15 days before the last menstrual period through day 310 of pregnancy. Because drinking behavior varies by day of the week, participants were temporally aligned so that, for example, a participant whose pregnancy began on a Wednesday was compared with others whose corresponding days also fell on Wednesdays. Outlier drinking values were Winsorized, or capped, at three standard deviations to limit their influence on imputed values. Participants lacking any first-trimester data or missing more than 200 days were excluded, since their drinking patterns could not be reliably established. A crucial methodological insight shaped the reference pool: women with completely missing-free records were more likely to be nondrinkers or light drinkers, so using only them as neighbors would underestimate true drinking. Instead, the team included segments of non-missing data from all participants as reference material.

Validation proceeded by randomly deleting data for 5 to 15 consecutive days from the first-trimester records of participants with no missing data, then comparing imputed values against the actual deleted values. The first trimester was chosen because both the proportion of women drinking and the magnitude of their drinking are highest then, particularly before pregnancy recognition. The team examined root mean squared error across values of k from 1 to 10 and across reference segment lengths. They found that five nearest neighbors and reference segments of 55 days, roughly two months, provided imputed values with the least error. Shorter segments did not meaningfully increase the available reference pool, while the 55-day length preserved the algorithm’s ability to characterize specific drinking patterns. For nondrinking segments, a single neighbor produced the lowest error, but multiple neighbors improved prediction of drinking segments, making k equal to 5 a reasonable compromise for both.

The results were striking. Across 3.2 million person-days of observation, 0.36 million person-days, or 11.4 percent, were missing. About 45 percent of participants had alcohol data for every single day of pregnancy, while the remaining 55 percent had at least one day missing. Sixty-two percent of participants were drinkers, consuming at least one drink during pregnancy. After deleting data segments from complete first-trimester records, the algorithm produced no difference between actual and predicted values for 64 percent of deleted segments. For another 31 percent of segments, imputed values fell within plus or minus one drink per day of the actual values. The algorithm exactly predicted drinking status for 76 percent of drinking segments in the combined analysis. Accuracy varied by study site: nondrinking segments were predicted within one drink for 78.6 percent of deleted segments in South Africa and 67.6 percent in the Northern Plains, reflecting differences in the magnitude of drinking and the proportion of missing data between sites.

Imputation also shifted population-level estimates in informative ways. Following imputation, the mean number of drinks in the first trimester increased by an average of two drinks in South Africa, while the increase for the Northern Plains sites was just below one drink. The Northern Plains had fewer missing data points overall, with over half of participants having 30 or fewer missing days, and lower proportions of drinkers and drinking volumes. Most missing data at the South African site came from early trimesters, reflecting later enrollment there, whereas most missing data in the Northern Plains occurred in the third trimester. This distinction matters because early-pregnancy missing data are more likely to represent active drinking periods, given that many women reduce or stop drinking after learning of the pregnancy.

The study is, to the authors’ knowledge, the first to describe this method for imputing missing alcohol consumption data in a longitudinal study of pregnant women. Prior comparative work had found that multiple imputation and full information maximum likelihood yielded less biased estimates than complete case analysis or last observation carried forward, but even those methods underestimated actual drinking levels, and imputing irregularly spaced longitudinal data would require complex extensions of standard multiple imputation. The non-parametric nature of k-NN sidesteps the normality assumptions embedded in most standard software packages, making it well suited to skewed, high-volume daily data. The k-NN algorithm is increasingly used to impute missing data in high-volume research areas such as genetics and metabolomics, where it has often produced the smallest imputation error compared with mean and median imputation, Bayesian linear regression, and clustering approaches. The authors note that the algorithm cannot overcome bias introduced when participants misreport drinking days as nondrinking days, and that future studies should evaluate alternative distance measures and other machine learning approaches. The methodological details presented here hold relevance well beyond alcohol research, extending to substance use and nutrition studies that suffer from informative missing data in longitudinal designs.

Subject of Research: Machine learning imputation of missing prenatal alcohol exposure data

Article Title: K-nearest neighbor algorithm for imputing missing longitudinal prenatal alcohol data

Article References: Sania, A., Pini, N., Nelson, M. E., Myers, M. M., Shuffrey, L. C., Lucchini, M., Elliott, A. J., Odendaal, H. J., & Fifer, W. P. (2025). K-nearest neighbor algorithm for imputing missing longitudinal prenatal alcohol data. Advances in Drug and Alcohol Research, 4, Article 13449. https://doi.org/10.3389/adar.2024.13449

Image Credits: AI Generated

DOI: 10.3389/adar.2024.13449

Keywords: k-nearest neighbor, machine learning, data imputation, prenatal alcohol exposure, missing data, longitudinal studies, Safe Passage Study, Timeline Followback, epidemiology, alcohol research, K-nearest, neighbor