New Validated Questionnaire Reveals Why Teachers Struggle to Use Scientific Evidence

For decades, education researchers have produced a steady stream of findings about what works in classrooms, and for just as long, teachers have largely failed to use them. The reasons are neither mysterious nor trivial: heavy workloads, inaccessible journal articles, poor initial training in research methods, and an educational system that lurches from one legislative reform to the next all conspire to keep scientific evidence out of daily teaching practice. Now, a team of Spanish researchers has taken a concrete step toward measuring that disconnect. Montse Guinovart-Pedescoll of the Departament d’Educació de la Generalitat de Catalunya and Ramon Palau of Universitat Rovira i Virgili have designed and validated a questionnaire that captures how teachers and future teachers perceive the barriers and facilitators that shape whether research findings ever reach their classrooms. The study, published in the Journal of New Approaches in Educational Research, offers a rigorous psychometric foundation for understanding one of education’s most stubborn gaps.

The rationale for the work rests on a well-documented paradox. Both individual educators and the institutions that govern them recognize the benefits of applying educational evidence to teaching, yet the gap between academic research and frontline practice has persisted since at least the early 1990s, when Charles Kaestle famously described the awful reputation of education research. Previous studies have traced the problem to multiple, overlapping causes. Research competence is poorly integrated into initial teacher education, so many graduates never learn to interpret studies or judge the quality of evidence. Few professionals are trained to distinguish reliable findings from unfounded claims. Time is another critical constraint: teachers are unwilling to invest scarce hours in engaging with research, and those in leadership positions are often unwilling to fund the resources that would make such engagement feasible.

The new study also highlights subtler obstacles that operate at the psychological and structural level. Teachers may resist change because new evidence conflicts with pre-existing beliefs and values, a phenomenon long noted in the literature. Research on teacher effectiveness suggests a kind of ossification: classroom effectiveness improves rapidly during the first five to seven years of teaching and then plateaus, a shift the authors of that earlier work attribute to the transition from deliberate, reflective practice to habitual routines at the neural level. Meanwhile, the way scientific articles are written poses an accessibility problem in itself, and the realities of a given classroom often differ so much from the conditions of the original research that teachers judge the findings irrelevant. Political pressures and the rush to publish significant results can further compromise research quality and ethics, leading society to confuse impact with quality and to adopt ineffective practices.

Against this backdrop, the researchers set out to build a measurement tool rather than another exhortation to collaborate. Their questionnaire follows a cross-sectional design and draws on Dillman’s Tailored Design Method, a framework for constructing surveys that produce valid and reliable results. Questions were drafted with clarity, impartiality, and conciseness in mind, each addressing a single element. Crucially, the instrument defines what it means by educational research and scientific evidence, ensuring that all respondents share a common conceptual framework, since definitions of research can vary widely across the teaching population. The questionnaire measures three main variables on five-point Likert scales: perception of barriers to accessing and transferring scientific evidence, perception of potential facilitators, and perception of the potential of new technologies to help teachers integrate evidence into the classroom.

Content validity was established through a formal expert judgment process. Eleven experts, nine from the university sector and two schoolteachers with university affiliations, evaluated every item for comprehension, relevance, and importance. Selection criteria required experience in education and educational technologies, training in scientific research, more than three years of professional experience, and membership of different educational bodies and institutions. The experts’ own competence was quantified using the K Coefficient, or Expert Competence Coefficient, which combines a knowledge coefficient with an argumentation coefficient. The results were reassuring: most experts scored above 0.8, indicating high competence, with only two in the medium range. Every item received an average score above 3.5 out of 4, and with the standard threshold of 80 percent expert agreement met, all items were retained for the pilot phase.

The pilot test involved 278 students enrolled in the Master’s Degree in Teacher Training at four Catalan universities: Universitat de Lleida, Universitat Ramon Llull, Universitat Rovira i Virgili, and Universitat de Vic. Drawn from a total population of 450 prospective secondary teachers through non-probabilistic convenience sampling, the sample exceeded the 259 participants calculated as necessary for a 99 percent confidence level with a 5 percent margin of error. The questionnaire was distributed by email and in person, with students able to respond via a QR code on their mobile devices. Notably, 37.05 percent of the sample combined their studies with active teaching work, a situation the authors describe as distinctive to the Catalan context.

The statistical analysis is where the study earns its technical credibility. Because Mardia’s multivariate kurtosis coefficient of 18.963 indicated non-normal data, the researchers selected the Weighted Least Squares estimation method, chosen for its robustness against departures from normality in exploratory factor analysis. Sampling adequacy was confirmed with a Kaiser-Meyer-Olkin value of 0.847, comfortably above the 0.80 threshold considered good, and Bartlett’s test of sphericity was highly significant. An oblique Promax rotation was applied to allow factors to correlate, and the resulting correlation matrix justified that choice. Applying the Kaiser criterion, a scree test, and a parallel analysis comparing observed eigenvalues against those from random data, the team retained seven factors, each represented by at least three items with loadings above the 0.32 significance threshold.

The seven factors tell a coherent story about what shapes evidence use. Factor 1 captures technology as a facilitator, addressing how digital tools can help integrate scientific evidence into classrooms. Factor 2 reflects institutional support, including time to implement evidence and the accessibility of research in terms of language and complexity; a negatively loading item within this factor signals that research lacking relevant conclusions for practice can undermine even improved access. Factor 3 concerns direct contact between teachers and research teams. Factor 4 covers educational resources, Factor 5 groups relevance, accessibility, and practicality, Factor 6 encompasses physical and cognitive barriers to accessing scientific articles, and Factor 7 measures the value of scientific literature itself as a professional resource. Reliability analysis showed Cronbach’s Alpha values above 0.7 for all factors except Factor 7, which scored 0.492; the authors retained it because its eigenvalue exceeded 1 and its low consistency likely reflects the small number of items, while also noting that merging it with Factor 4 would have produced an acceptable alpha of 0.764.

The practical implications extend well beyond psychometrics. By mapping the latent structure of teachers’ perceptions, the instrument gives policymakers, teacher educators, and researchers a diagnostic tool for identifying exactly where the pipeline from evidence to practice breaks down, whether at the point of access, interpretation, institutional backing, or technological support. The authors acknowledge limitations: convenience sampling limits representativeness, and the sample of Catalan master’s students may not generalize to the broader teaching population. They propose administering the questionnaire to more active teachers across experience levels and complementing future quantitative analyses with qualitative and mixed-methods designs to strengthen validity. Still, the study delivers what Kaestle argued was essential decades ago: a clear picture of the frontline of the system. With a validated instrument now available, the dialogue between educational research and classroom practice has, for the first time, a reliable way to measure itself, and a concrete pathway toward closing a gap that has persisted for more than thirty years.

Subject of Research: Design and validation of a self-perception questionnaire on teachers' use of scientific knowledge in teaching practice

Article Title: Design and validation of a self-perception instrument on the use of scientific knowledge in teaching practice

Article References: Guinovart-Pedescoll, M., & Palau, R. (2025). Design and validation of a self-perception instrument on the use of scientific knowledge in teaching practice. Journal of New Approaches in Educational Research, 14(1), Article 3. https://doi.org/10.1007/s44322-024-00026-5

Image Credits: AI Generated

DOI: 10.1007/s44322-024-00026-5

Keywords: educational research, scientific evidence, questionnaire validation, teacher training, exploratory factor analysis, evidence-based practice, teaching quality, research-practice gap, psychometrics, educational technology, Catalonia, teacher perceptions