Fake Data, Real Findings: How Synthetic Records Could Fix Criminology’s Reproducibility Crisis

Criminology has a transparency problem, and a new study suggests an unexpectedly elegant solution: fabricated data. Not fabricated in the sense of fraud, but statistically engineered stand-ins for real records that are too sensitive to release. In a paper published in the American Journal of Criminal Justice, Ian T. Adams and Paige E. Vaughn of the University of South Carolina demonstrate that a carefully generated synthetic dataset can reproduce the findings of a published survey experiment almost exactly, offering the field a workable pathway around its most stubborn obstacle to open science.

The scale of the discipline’s transparency deficit is striking. Surveys of top criminology journals have found that fewer than five percent of articles include preregistration, and large-scale reproduction tests place criminology near the bottom of the social and behavioral sciences, with exact reproduction achieved in just five percent of sampled articles from flagship outlets. No criminology journal currently requires data and code sharing, and the field’s open data efforts have concentrated on rare events such as school shootings and terrorism incidents rather than the everyday studies that make up the bulk of published research. Researchers who avoid archiving data often report that they believe the practice is either impossible or professionally unrewarded, a perception reinforced by the absence of institutional requirements.

What makes criminology different from fields where open data has flourished is the nature of the raw material. Victimization surveys, correctional records, and law enforcement data carry strict privacy protections that preclude public release. Researchers navigating privacy laws, institutional review requirements, and data use agreements face real professional and ethical risks in sharing records that describe criminal victimization, offending, or police conduct. Protecting the people represented in these datasets is not optional. But as Adams and Vaughn argue, privacy concerns need not foreclose transparency entirely, provided that researchers can verify which statistical relationships survive the process of manufacturing artificial records.

Synthetic data are not a new idea; statisticians have proposed them as a disclosure limitation tool since the early 1990s. The concept is straightforward: rather than publishing real records, researchers fit statistical models to the original data and draw new, artificial records from those models. The synthetic dataset preserves the useful structure of the original, including correlations and group differences, while containing no actual participants. Whether such data are fit for purpose, however, depends entirely on how they are generated and what analyses they are meant to support. General fidelity to the original distributions does not guarantee that any particular analysis will yield the same conclusions.

To test this question rigorously, Adams and Vaughn turned to a preregistered factorial vignette experiment originally published in Police Quarterly, which examined how police and human resources executives evaluate workplace profanity. The original design was methodologically challenging for synthesis: 1,351 respondents each rated four of nine vignette conditions, producing 5,180 nested observations, with outcomes measured on five-point scales covering appropriateness, professionalism, public trust, and recommended sanctions. The experimental design manipulated two factors: the target of profanity, whether directed at oneself, a colleague, or the public, and the context in which it occurred, whether derogatory, celebratory, or neutral.

The researchers used sequential classification and regression tree synthesis, implemented in the widely used synthpop package for the R statistical language. Because the experiment involved repeated responses from the same people, they first restructured the data so each respondent’s observations occupied a single row, ordered by vignette condition, preserving the experimental assignment and completion patterns. The five outcomes in each slot were then synthesized conditional on the design variables and previously synthesized outcomes. The result was a partially synthetic dataset: experimental assignments remained unchanged while all outcome values were replaced with model-generated draws, with identifiers excluded throughout to maintain deidentification.

The validation strategy was unusually thorough, examining disclosure risk, general fidelity, and utility for the specific models the data were meant to support. Disclosure diagnostics counted distinct variable combinations, duplicated rows, and exact matches across datasets, including replicated uniques, combinations appearing exactly once in each dataset that nevertheless match. General fidelity was assessed with a propensity score mean squared error, a comparison of all ten Pearson correlations among the outcomes, and Mahalanobis distance analysis using the original data’s covariance structure. Specific utility was judged by re-estimating the five mixed-effects models from the original study on the synthetic data and comparing all 20 treatment coefficients, their standard errors, and their significance classifications.

The results were remarkably strong on the measures that matter most for reproduction. Ninety percent of the 20 treatment coefficients differed from the original estimates by less than two original standard errors, with a mean absolute standardized coefficient difference of 1.03. Standard-error ratios ranged from 0.989 to 1.085, meaning model-based precision was nearly identical to the original analysis. All 20 coefficient directions and all 95 percent confidence interval classifications agreed across the two datasets, and both supported the original substantive finding: profanity directed at the public drew harsher evaluations than self-directed profanity, with smaller contrasts for colleague-directed speech. The mean absolute difference across the ten outcome correlations was just 0.015, and the propensity score measure of 0.0020 indicated the two datasets were hard to distinguish statistically.

Yet the study is candid about the limits of the approach. The two coefficients outside the descriptive benchmark both concerned personal discipline judgments, departing in opposite directions. Residual intraclass correlations, which capture how strongly a respondent’s answers are clustered together, declined modestly in the synthetic data for some outcomes, dropping from 0.515 to 0.419 for personal discipline. Extreme multivariate observations diverged in the tails of the Mahalanobis distance distribution. And critically, synthesis is not automatic privacy protection: exact matches and distinctive respondent response patterns remained despite the replacement of all outcome values, meaning release decisions still require context-specific assessment of what external information could connect patterns to people.

The implications reach well beyond one dataset. Adams and Vaughn note that limited data access may inadvertently encourage questionable research practices: surveys of criminologists have found that 53 percent report selectively underreporting results and 43 percent admit omitting nonsignificant findings. Synthetic data releases paired with executable code can expose programming errors and make analytical choices inspectable, though they cannot authenticate original observations or prove selective reporting did not occur. The authors recommend that researchers document intended uses, software versions, random seeds, and disclosure assessments; that repositories clearly distinguish synthetic records from originals; and that journal editors request executable code alongside a justified data-access plan. With recent signals that the Journal of Quantitative Criminology will soon require reproducible code and data at submission, synthetic data may arrive just in time, giving a privacy-bound discipline the tools to open its doors without betraying the people behind its records.

Subject of Research: Synthetic data generation for reproducible and privacy-preserving open science in criminology

Article Title: Synthetic Data as a Pathway to Reproducible Criminology

Article References: Adams, I. T., & Vaughn, P. E. (2026). Synthetic Data as a Pathway to Reproducible Criminology. American Journal of Criminal Justice. https://doi.org/10.1007/s12103-026-09953-9

Image Credits: AI Generated

DOI: 10.1007/s12103-026-09953-9

Keywords: synthetic data, criminology, reproducibility, open science, data privacy, synthpop, CART synthesis, survey experiment, disclosure risk, mixed-effects models, data sharing, research methods