Landmark Analysis Confirms the TPACK Scale Is a Highly Reliable Tool for Measuring Teachers’ Tech Knowledge

A sweeping new meta-analysis has delivered one of the most comprehensive verdicts yet on the trustworthiness of a questionnaire that thousands of education researchers have relied on for more than fifteen years. The Technological Pedagogical Content Knowledge scale, first published by Schmidt and colleagues in 2009, measures how well teachers integrate technology into their professional knowledge. According to a Reliability Generalization Meta-Analysis published in BMC Psychology by Ahsen Filiz of Biruni University and Cem Oktay Güzeller of Yıldız Technical University, the instrument shows exceptionally high internal consistency across the research literature, with a pooled Cronbach’s alpha of 0.944.

For readers unfamiliar with psychometrics, Cronbach’s alpha is a number between zero and one that expresses how coherently a set of questionnaire items hangs together. Values above 0.90 are conventionally considered excellent, and a pooled estimate of 0.944 sits comfortably in that range. But the Turkish research team went well beyond simply averaging numbers. They applied the Reliability Generalization Meta-Analysis framework, known as REGEMA, which treats each reported reliability coefficient as an effect size in its own right and then models how much those coefficients vary from study to study. This approach matters because reliability is not a fixed property of a questionnaire; it depends on the specific sample, the language version, the item count and the testing conditions of every individual administration.

The evidence base was assembled through a systematic search of the Web of Science and Scopus databases covering the years 2010 through 2025. The initial sweep returned 604 records, which the authors then screened and audited with unusual care. During an audit of their original Excel coding file and their JASP statistical dataset, they identified one duplicate study record and one study represented by two repeated reliability coefficients drawn from pre-test and post-test administrations of the same participants. After these corrections, the qualitative synthesis comprised 69 unique studies. The quantitative synthesis was narrower: it included 48 effect estimates drawn from 47 studies, because 22 of the unique studies could not contribute a poolable estimate. Twenty-one of those lacked a sample-specific alpha, and one reported an alpha but omitted the item count needed to calculate the sampling variance.

That last detail points to one of the study’s central messages, and one of its most quietly damning findings about the field. In a substantial share of published TPACK research, authors simply borrowed the reliability figure from the original 2009 validation paper instead of computing it for their own participants. When researchers do this, they implicitly assume that a questionnaire behaves identically in every language, culture and sample, an assumption that measurement specialists have repeatedly shown to be false. The new analysis demonstrates both the value of reporting sample-specific estimates and the scale’s resilience: even across wildly different contexts, the instrument’s internal consistency remained high.

Statistically, the primary analysis used a Bonett transformation of the alpha coefficients, a technique that stabilizes the sampling distribution of reliability estimates before they are pooled. Because one study contributed two dependent coefficients, the model was specified as a multilevel random-effects model with effect estimates nested within studies, a design that respects the non-independence of repeated measurements from the same source. The pooled result was an alpha of 0.944 with a 95 percent confidence interval running from 0.933 to 0.953, meaning the true population value almost certainly lies in a narrow band near the top of the reliability scale.

Perhaps more revealing was the 95 percent prediction interval, which stretched from 0.812 to 0.983. Unlike a confidence interval, a prediction interval estimates where a single new study’s reliability coefficient would plausibly fall. The lower bound of 0.812 still represents good internal consistency, suggesting that even in less favorable circumstances the TPACK scale tends to perform well. A sensitivity analysis using untransformed alphas produced a similar pooled estimate of 0.934, but its prediction interval ran from 0.845 to 1.024, a range that exceeds the theoretical maximum of any reliability coefficient. That mathematical impossibility was itself evidence in favor of the Bonett-transformed model as the more defensible primary analysis.

The analysis also quantified just how much reliability estimates differ across studies. The between-study variance, expressed as tau-squared, reached 0.376 on the transformed scale, which the authors describe as substantial heterogeneity. In plain terms, while the average reliability is excellent, individual studies have reported alphas ranging from merely acceptable to nearly perfect. Influence diagnostics singled out one observation as the most influential low-reliability data point: a study by Cao and colleagues involving 440 participants and a 23-item version of the instrument that produced an alpha of only 0.708. Removing or down-weighting such outliers in influence analyses helps reveal whether a pooled estimate depends heavily on a single anomalous result, and here the overall conclusion survived scrutiny.

The team also confronted a subtler methodological question: what counts as the TPACK scale? Over the years, researchers have produced numerous adaptations, translations and shortened versions of the original instrument. Four of the 48 pooled coefficients came from closely related but non-identical TPACK instruments rather than the original scale itself. To test whether these near-relatives distorted the picture, the authors ran a sensitivity analysis excluding them, and the pooled estimate did not materially change. That robustness check strengthens the claim that the high reliability is a genuine property of the instrument family rather than an artifact of mixing apples and oranges.

The TPACK framework itself, which stands for Technological Pedagogical Content Knowledge, grew out of earlier work on pedagogical content knowledge and describes the intersection of three knowledge domains: understanding technology, understanding teaching, and understanding subject matter. Since its introduction, it has become one of the dominant theoretical lenses in educational technology research worldwide, underpinning teacher training programs, national surveys and countless dissertations. A measurement tool this widely deployed deserves exactly this kind of periodic health check, because policy decisions and program evaluations built on an unreliable instrument would inherit its flaws.

The authors’ concluding recommendation is straightforward: future TPACK research must report sample-specific reliability estimates rather than recycling values from earlier publications. The study received no external funding, involved no new data collection from human participants, and was published open access, making its methods and coding materials transparent to any researcher who wishes to scrutinize or extend them. For the global community of educational researchers, the message is reassuring on one front and cautionary on another. The TPACK scale, by and large, measures what it holds together with remarkable consistency. But the practice of assuming reliability instead of demonstrating it remains widespread, and this meta-analysis provides both the evidence and the statistical toolkit to change that habit.

Subject of Research: Reliability generalization meta-analysis of the TPACK scale's internal consistency

Article Title: The reliability of the TPACK scale developed by Schmidt et al. (2009) and related adaptations: a Reliability Generalization Meta-Analysis (REGEMA)

Article References: Filiz, A., & Güzeller, C. O. (2026). The reliability of the TPACK scale developed by Schmidt et al. (2009) and related adaptations: a Reliability Generalization Meta-Analysis (REGEMA). BMC Psychology. https://doi.org/10.1186/s40359-026-05695-9

Image Credits: AI Generated

DOI: 10.1186/s40359-026-05695-9

Keywords: TPACK, Cronbach's alpha, reliability generalization, meta-analysis, psychometrics, teacher knowledge, educational technology, internal consistency, REGEMA, questionnaire validation, teacher professional development, BMC Psychology