Research report: A tutorial on selecting, translating, and reporting variance priors in Bayesian multilevel models

A tutorial published in Behavior Research Methods shows that variance priors described as “weakly informative” can place substantially different amounts of probability near zero and allow very different degrees of variation, even when they are matched to the same benchmark. Drawing on a worked analysis of TIMSS 2019 data, the paper proposes a five-step workflow for selecting, translating, checking and reporting priors for scalar variance components in Bayesian multilevel models. The example involved 400 students nested within 40 schools, a deliberately small number of higher-level units for which a between-school variance parameter may be weakly identified.

Its main finding was that prior family and parameter scale can change the precision of variance estimates even though posterior medians and broad substantive conclusions remained fairly stable. The tutorial focuses on variance components, including residual variance and random-effect variances, rather than priors for covariance matrices or models with more complicated variance structures. It concentrates on models in which the number of higher-level units is limited, because higher-level variance components can be difficult to estimate when there are few schools, clinics or other clusters.

In such settings, maximum-likelihood estimation may produce unstable variance estimates or estimates near a boundary, while a Bayesian prior regularizes the parameter. That regularization is not automatically neutral: its influence depends on the prior family, hyperparameters, likelihood and the degree of alignment between the prior and substantive knowledge. The first step is to identify both the target variance component and the scale on which the prior must be implemented.

Variance, precision, defined as the reciprocal of variance, and standard deviation are mathematically related, but a prior does not retain an identical practical behavior when expressed on different scales. The tutorial reviews gamma and inverse-gamma distributions, which are commonly discussed on variance or precision scales, and half-t and half-Cauchy distributions, which can be placed directly on the standard deviation scale. It also notes that a bounded uniform prior is not a neutral default: its limits encode assumptions about which values are plausible.

For example, small-hyperparameter inverse-gamma specifications sometimes described as diffuse can place substantial probability near zero, while an excessively diffuse prior can assign appreciable probability to implausibly large values. Several distributional relationships provide a way to translate a benchmark into software-compatible parameters. A chi-square distribution is a special case of the gamma family, and a scaled chi-square can anchor a gamma prior to a benchmark variance while separating location from concentration.

Using the shape-rate parameterization, a benchmark variance s² and effective degrees of freedom ν give the specification Gamma(ν/2, ν/2s²), whose mean is s²; larger ν makes the prior more concentrated around that benchmark. A gamma prior on precision induces an inverse-gamma prior on variance through reciprocal transformation, but interpretation of the hyperparameters changes: the second parameter is a rate in the gamma formulation and a scale in the inverse-gamma formulation. These links make prior construction more transparent, yet they do not imply that mathematically related priors will regularize a fitted model identically.

For priors on the standard-deviation scale, the tutorial uses the half-t family to separate the rough magnitude of plausible standard deviations from tail behavior. The scale parameter determines magnitude, while the degrees of freedom determine how readily the prior accommodates larger values; smaller degrees of freedom produce heavier tails. The half-Cauchy is the zero-location half-t case with one degree of freedom, and its scale equals its median.

Median matching offers a common calibration target across families with different shapes: for a half-t prior, the desired median is the scale multiplied by the appropriate quantile of a standard t distribution, while for a half-Cauchy it is simply the scale. Matching medians aligns only one feature, however. Priors can share a central value while differing in near-zero mass, skewness, tail weight and resulting prior predictive behavior.

The TIMSS example used a two-level random-intercept model relating students’ science achievement to mathematics achievement at the student and school levels, including a cross-level interaction. The focal sample contained 400 students in 40 schools. A separate, non-overlapping reference sample contained 7,408 students in 233 schools, and a maximum-likelihood fit to that reference sample supplied benchmark standard deviations of 12.20 for the between-school random intercept and 46.08 for the residual variation.

This separation avoided using the same observations to construct the prior and fit the focal model, but it did not guarantee that the samples shared the same variance structure. The benchmark values were therefore empirical calibration inputs rather than ground truth, and a mismatch between reference and target populations could have pulled inference toward an inappropriate magnitude. For the fitted models, gamma, inverse-gamma, half-t and half-Cauchy priors were all calibrated so that their medians matched the relevant benchmark standard deviation, while their concentration and tail behavior remained distinct.

For the between-school component, the specifications were Gamma(shape 12, rate 0.9566), inverse-gamma(shape 12, scale 142.32), half-t(4, scale 16.47) and half-Cauchy(scale 12.20). For the residual component, they were Gamma(shape 30, rate 0.6438), inverse-gamma(shape 30, scale 1367.08), half-t(4, scale 62.21) and half-Cauchy(scale 46.08). A baseline half-t(3, location 0, scale 20) prior on both standard deviations provided a less structured comparison.

Fixed-effect priors were held constant, so the comparison focused on the variance specifications. The models were fitted in brms with Stan’s Hamiltonian Monte Carlo sampler using four chains and 2,000 iterations per chain, including 1,000 warm-up iterations. Inspection of the calibrated densities showed that median matching did not make the families interchangeable.

Gamma and inverse-gamma priors placed relatively more probability near the benchmark values, whereas half-t and especially half-Cauchy priors retained heavier tails. Prior predictive checks translated those differences to the outcome scale. The distributions for science achievement were centered in broadly similar regions, but heavier-tailed specifications implied greater dispersion, with the half-Cauchy condition producing the broadest upper range.

This step matters because a mathematically reasonable prior can still imply implausible variability when its consequences are expressed on the scale of the measured outcome. A useful workflow must therefore examine both parameter-scale densities and prior predictive implications rather than stopping at a list of hyperparameters. After fitting, posterior medians for the between-school random-intercept standard deviation ranged from approximately 12.74 under the inverse-gamma condition to 13.62 under the baseline.

Residual-standard-deviation medians were more stable, ranging from about 46.27 to 46.48. The clearest differences appeared in uncertainty rather than location: gamma and inverse-gamma conditions produced somewhat tighter 95% credible intervals for the between-school component, while the heavier-tailed half-t and half-Cauchy conditions produced wider intervals. Fixed-effect summaries were broadly stable, although this was not a general test of fixed-effect prior sensitivity because those priors were not varied.

The worked example therefore shows that variance-prior choice can shape posterior precision around the targeted components without reversing the model’s broad substantive pattern in this dataset. Sampler and convergence diagnostics were satisfactory across conditions. The fitted models showed no divergent transitions or maximum-tree-depth warnings, R-hat values were close to 1.00 and effective sample sizes were adequate overall, although they were somewhat smaller under the half-Cauchy condition than under more concentrated specifications.

Sensitivity analyses varied concentration within the gamma family and tail behavior within the half-t family while keeping benchmark locations fixed. Residual-variance results remained highly stable and between-school results changed only modestly. The tutorial stresses that this single worked example is not a general performance study: it provides no known population values with which to compare bias, root mean square error or credible-interval coverage, and it cannot establish universally superior priors across model structures, sample sizes or degrees of benchmark misalignment.

The paper’s reporting recommendations are correspondingly practical. An analysis should identify the target variance components and the scale on which each prior was imposed, disclose the benchmark and its source, distinguish hyperparameters that were fixed from those solved for, and explain the intended degree of concentration or tail flexibility. It should also report prior-density inspection, prior predictive checks, posterior comparisons, convergence information and sensitivity analyses.

The benchmark-informed route is not presented as the only coherent strategy; formal expert elicitation, hierarchical priors, prior predictive elicitation and fully external information may be preferable in other applications. The tutorial’s broader contribution is a practical framework for making the prior an inspectable scientific assumption rather than an invisible setting inherited from software.

Subject of Research: Benchmark-informed selection and reporting of scalar variance priors in Bayesian multilevel models

Article Title: A tutorial on selecting, translating, and reporting variance priors in Bayesian multilevel models

Article References: Liu, L. (2026). A tutorial on selecting, translating, and reporting variance priors in Bayesian multilevel models. Behavior Research Methods, 58(11), Article 328. https://doi.org/10.3758/s13428-026-03201-y

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03201-y

Keywords: Bayesian multilevel models, variance priors, prior calibration, parameterization, prior predictive checks, sensitivity analysis, TIMSS 2019, brms, small-cluster inference, Stan, tutorial, selecting