The framework builds on a practice that has been gaining ground in psychological methods: interval-based hypothesis testing. Instead of testing whether an effect differs from zero, researchers first specify a smallest effect size of interest, often abbreviated as SESOI, which is the boundary below which an effect is considered too small to care about. They then examine a confidence interval around the observed effect. If the entire interval falls outside the SESOI boundaries, the effect is deemed practically meaningful, a procedure known as a minimum-effect test. If the entire interval falls inside the boundaries, the effect is deemed negligible, the logic behind equivalence testing. These approaches, which trace back to work in pharmacology by Westlake in 1972 and to minimum-effect tests proposed by Murphy and Myors in 1999, force researchers to commit to what counts as a meaningful result before they see the data.

But there is a catch that the new paper puts front and center. Confidence intervals frequently straddle the SESOI boundaries, overlapping both the meaningful and the negligible regions at the same time. When that happens, the interval-based test is inconclusive: the data are compatible with an effect that matters and with an effect that does not. The authors illustrate the problem with a running example throughout the article. Suppose an observed effect of 5.5 on some scale, with a standard error of 0.51, is tested against SESOI boundaries of minus 5 and plus 5 using a two-tailed minimum-effect test. The 90 percent confidence interval around the observed effect extends beyond the upper boundary, so the researcher cannot confidently claim the effect is practically meaningful, yet the point estimate sits above the threshold. The result is neither a clean win nor a clean loss, and standard practice offers little guidance on what to say next.

This inconclusive zone is not a rare curiosity. Because confidence intervals widen with smaller samples and noisier measures, underpowered studies land in the uncertainty zone routinely. The authors argue that the problem has been underappreciated precisely because the methodological reform conversation has focused on getting researchers to adopt SESOI-based tests in the first place, without equipping them for the awkward outcomes those tests can produce. A researcher who runs an equivalence test and finds the confidence interval crossing the equivalence margin is left with a result that looks like failure but is really just ambiguity. Worse, the pressure to publish can tempt researchers to interpret ambiguous intervals in whichever direction favors their hypothesis, reintroducing the very flexibility that interval-based testing was meant to eliminate.

To address this, Riesthuis and colleagues propose a set of complementary decision tools, packaged in a freely available Web application that accompanies the paper. The tools are designed to be used after an interval-based test yields an inconclusive result, helping researchers characterize just how uncertain they are and what kinds of effects remain plausible given the data. Rather than forcing a binary verdict, the framework encourages researchers to describe the position of the confidence interval relative to the SESOI boundaries, quantify how much of the interval lies in the meaningful versus negligible region, and consider what additional data would be needed to reach a decisive conclusion. The goal, the authors write, is to support more informed decision-making under uncertainty and to help researchers draw more nuanced statistical and practical conclusions from their results.

The framework also engages with a broader debate about how uncertainty should be quantified. Notably, the authors use a 90 percent confidence interval for their interval-based tests rather than the conventional 95 percent interval, citing a rationale discussed by Alter and Counsell in 2023. Because a two-sided interval-based test at significance level alpha effectively involves checking whether the interval excludes the SESOI boundary, a 1 minus 2 alpha confidence level is the coherent choice, and this detail matters for researchers implementing the procedures correctly. The paper also distinguishes itself from magnitude-based inference, a controversial approach used in sports science that was systematically reviewed in 2020 and criticized in 2019 as neither Bayesian nor a valid method of inference. The new framework stays firmly within frequentist interval logic while adding structured tools for interpreting ambiguous outcomes.

Specifying the SESOI in the first place is itself a thorny problem, and the paper situates its contribution within an existing literature on how to do it well. Anchor-based methods, described by Anvari and Lakens in 2021, derive thresholds from external benchmarks, such as the minimal clinically important difference concept introduced in health-status measurement by Jaeschke, Singer and Guyatt in 1989. Other work has surveyed experts to elicit the smallest effect that would matter in a given field, including studies on false memory research and on expert witness testimony concerning alcohol and memory. A 2025 literature review by Riesthuis and Cribbie examined when scientific findings are deemed practically or theoretically relevant in minimum-effect testing applications. The new paper assumes such a threshold has been chosen and focuses on the downstream problem: what to conclude when the interval will not cooperate.

The practical implications reach beyond academic statistics. Recent work has documented how frequently nonsignificance is misinterpreted as evidence of an effect’s absence in psychology, with a 2025 analysis in Royal Society Open Science documenting the prevalence of this error. Interval-based testing was supposed to fix such misinterpretations by making the role of effect magnitude explicit. But if inconclusive results are handled ad hoc, the reform risks trading one form of confusion for another. The authors also point to connections with replication research, noting in prior work that statistical nonsignificance does not equal an unsuccessful replication when the smallest effect size of interest is taken into account. A replication that lands in the uncertainty zone may simply reflect insufficient precision, not a failed replication, and the framework’s tools help make that distinction visible.

For researchers who want to apply the framework, the entry points are concrete. The Web application, hosted at paulriesthuis.github.io/SESOI-Decisions/, implements the complementary tools, and a mirror is available on ShinyApps. The code is openly available on the Open Science Framework, and supplementary materials accompany the article, including an HTML file and a PDF. The authors even provide exact parameters for reproducing their leading example: an observed effect of 5.5, a standard error of 0.51, SESOI bounds of minus 5 and 5, and a two-tailed minimum-effect test. The manuscript was supported by a FWO post-doctoral fellowship grant awarded to the first author, and the authors declared no conflicts of interest.

The paper lands at a moment when quantitative psychology is actively rethinking inference, with parallel efforts ranging from unified null-region frameworks to practical guides for drawing statistical conclusions that bridge null hypothesis testing and estimation. What distinguishes this contribution is its honesty about the limits of the reform it supports. Specifying a smallest effect size of interest and testing against it is a genuine improvement over worshiping the p-value threshold against zero, but no statistical procedure eliminates uncertainty; it can only make uncertainty explicit and manageable. By giving researchers a structured way to reason through inconclusive interval-based tests, the framework aims to make the ambiguity itself informative, showing which conclusions the data can support, which they cannot, and what it would take to find out. In a field still learning to think in intervals rather than verdicts, that may be the most practical result of all.

Subject of Research: A statistical framework for evaluating practical relevance in interval-based hypothesis testing using a smallest effect size of interest

Article Title: Decisions under uncertainty: A statistical framework for evaluating practical relevance in interval-based hypothesis testing

Article References: Riesthuis, P., Cribbie, R. A., Celio, V., & Beribisky, N. (2026). Decisions under uncertainty: A statistical framework for evaluating practical relevance in interval-based hypothesis testing. Behavior Research Methods, 58(11), Article 323. https://doi.org/10.3758/s13428-026-03204-9

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03204-9

Keywords: interval-based hypothesis testing, smallest effect size of interest, equivalence testing, minimum-effect test, effect size, statistical significance, practical relevance, confidence intervals, quantitative psychology, research methods, decision making under uncertainty, Behavior Research Methods