New R package turns messy school and clinic counts into honest proportions with uncertainty

Every year, education departments, health agencies, and public administrators publish tables of proportions: the share of children eligible for free meals at a school, the fraction of patients enrolled in a nutrition program at a clinic, the percentage of families receiving assistance in a district. These numbers look simple, but the uncertainty behind them is anything but. A proportion based on nine children carries far more statistical noise than one based on 134, and analysts who ignore that difference risk comparing sites on statistical artifacts rather than real differences. A new open-source tool called sitemix, described in the journal SoftwareX, aims to make those comparisons honest by computing site- and group-level proportions and their standard errors from whatever information is actually available, whether individual records or published aggregate counts.

The package, developed by JoonHo Lee and Alison Hooper and released under an MIT license, grew out of a practical evaluation problem in early childhood education. Analysts studying pre-kindergarten programs often want to know what proportion of enrolled children have families receiving nutrition assistance. Sometimes they hold individual student records; sometimes they have only published counts, often with different denominators across sites and sometimes with values deliberately suppressed by the publisher. Before sitemix, preparing proportions and their standard errors from these heterogeneous inputs required ad hoc calculations scattered across scripts, with assumptions rarely stated explicitly. The package replaces that improvisation with a common set of functions that return a standardized table of estimates, standard errors, and provenance information for every site and year.

The central technical insight of sitemix is that a standard error is meaningless until you specify what the proportion is intended to describe. Administrative records may cover an entire cohort, a selected subset, or a probability sample, and each interpretation demands a different variance calculation. The package’s default binomial model treats the observed count as arising from independent Bernoulli trials with a common underlying probability, so the variance of the observed proportion is the familiar theta times one minus theta divided by n. Under this model, uncertainty concerns repeated outcomes at a fixed sample size. Alternatively, analysts may want the proportion in a fixed finite population of N individuals, in which case uncertainty comes from which n individuals were sampled, and a finite-population correction adjusts the sampling variance. Crucially, observing the entire population gives zero design-based sampling variance, even when the population consists of a single person. The package forces users to choose between these targets rather than silently applying one formula to all data.

Once the target is defined, sitemix offers a menu of established calculations with their assumptions stated openly. For proportions of exactly zero or one, where the plug-in variance collapses to zero, the package supplies a Wilson-derived standard error surrogate by default, with options for the Agresti-Coull adjustment or a retained zero variance. For reporting on transformed scales, the default is the arcsine square-root transformation, whose first-order working standard error is simply one divided by two n, an approximation that depends only on the denominator. An optional Anscombe adjustment refines this transformation for small samples. The package also implements a bias-corrected binomial variance that replaces the denominator n with n minus one for interior cells. None of these boundary surrogates or working approximations guarantee confidence-interval coverage, and the documentation says so explicitly, a refreshing departure from software that presents approximations as exact.

Perhaps the most technically sophisticated part of the package concerns covariance between overlapping indicators. Suppose an analyst wants to know not just the share of children eligible for free and reduced-price meals and the share enrolled in SNAP, but how these indicators vary together. Separate totals for two assistance programs do not reveal how many children participate in both; only joint records or co-occurrence counts can estimate that covariance. When complete joint information is available, sitemix computes the plug-in covariance from the observed joint proportion minus the product of the marginal proportions, divided by n. For a single categorical variable partitioned into mutually exclusive categories, complete counts determine a multinomial covariance matrix whose rows sum to zero, preserving the constraint that category proportions total one. The package validates these matrices against the properties required by their construction rule, recording scale, indicator order, and sampling design metadata alongside every covariance carrier.

The package’s treatment of published aggregate counts reveals what such summaries actually retain. When numerator-denominator pairs match the record-based inputs, the aggregate route reproduces the record-based estimates and standard errors exactly, a property the authors demonstrate across 250 simulated site-year cells. But separate marginal counts for several indicators generally do not determine covariance between them. When only margins are available, sitemix returns a diagonal covariance matrix explicitly labeled as working independence, recording whether the margins describe the same, different, or unknown sampled units. Equal denominators alone do not establish that the units are the same. For complete, unsuppressed margins from the same units, the package computes Fréchet identification intervals, the mathematically sharp bounds on the joint proportion given the observed margins: the joint share must lie between the maximum of zero and the sum of the two proportions minus one, and the minimum of the two marginal proportions. These are identification intervals, not confidence intervals, and the simulated example shows how wide the joint-proportion range can remain without co-occurrence counts, falling on an endpoint at 13 of 50 sites.

Suppression handling is another distinctive feature. Publishers routinely withhold small or sensitive counts, and software that treats withheld values as observed zeros would silently corrupt the analysis. sitemix keeps suppressed values distinct from observed zeros throughout, requires users to explicitly acknowledge sensitivity values, stores them in separate columns, and excludes them from ordinary covariance and Fréchet calculations. A dedicated suppression report audits the data before estimation begins. The diagnostic function sm_diagnose checks objects, rows, and covariance matrices, returning results on a severity scale from error through warning, note, and ok, so analysts can review missing values, small denominators, and covariance compatibility before feeding the output into another model.

The validation effort behind the package is unusually thorough for statistical software. The authors checked all 263 scalar and finite-population configurations against reference values computed from closed-form identities, score-quadratic roots, and finite-population sample enumeration, with the largest absolute discrepancy at 8.88 times ten to the minus sixteenth, essentially machine precision. All 118 categorical and Fréchet assertions passed, covering simplex constraints, positive semidefiniteness, sparse categories, boundary surrogates, and census behavior. The full test suite passed 6,918 expectations across 53 files with no failures, and test-based line coverage reached 91.37 percent. The authors also examined the accuracy of the arcsine working variances by computing exact variances across a grid of sample sizes and probabilities, finding that the unadjusted working variance could be anywhere from 0.405 to 10.23 times the exact variance across interior grid points, a sobering reminder that transformed-scale approximations demand scrutiny at small samples.

To illustrate the workflow, the authors provide prek_sim, a simulated panel of 7,845 student-year rows from 50 sites over 2021 to 2025, containing four overlapping binary indicators: eligibility for free and reduced-price meals, enrollment in SNAP, participation in WIC, and receipt of TANF. Site-year cells contain between 6 and 134 children. The examples walk through estimating site-level proportions, computing covariance from jointly observed indicators, reproducing record-based estimates from matching counts, and examining what marginal counts alone cannot recover. A final empirical Bayes illustration shows how the estimate-standard error pairs can feed a normal-normal shrinkage analysis on the arcsine scale, where sites with small denominators move noticeably in rank after borrowing strength across sites, though the authors caution that rank uncertainty is not quantified and the normal approximation requires assessment for sparse cells.

The impact of sitemix lies in making uncertainty preparation repeatable and transparent. Each estimate retains its denominator, reporting scale, suppression status, and method information, so analysts can review how a table was produced when updating it or sharing it with colleagues. The count interfaces extend the workflow to settings where individual records are unavailable, while the Fréchet machinery helps analysts decide which additional counts they actually need. The package does not pool proportions across sites, fit hierarchical models, or handle complex survey designs; its only cross-site model is an optional experimental variance smoother. Instead, it focuses on preparing estimates and uncertainty summaries for subsequent analysis, leaving the modeling choices where they belong: with the analyst, justified separately. For a field drowning in published proportions of unknown provenance, that discipline may prove quietly transformative.

Subject of Research: Statistics; Software; Data Science

Article Title: sitemix: An R package for site- and group-level proportions and uncertainty from individual records and aggregate counts

Article References: Lee, J., & Hooper, A. (2026). sitemix: An R package for site- and group-level proportions and uncertainty from individual records and aggregate counts. SoftwareX, 36, Article 103131. https://doi.org/10.1016/j.softx.2026.103131

Image Credits: AI Generated

DOI: 10.1016/j.softx.2026.103131

Keywords: R package, statistics, uncertainty quantification, proportions, empirical Bayes, open-source software, education research, covariance, sitemix, package, site-, group-level