New Chemistry Test Reveals a Bimodal Puzzle in Kazakhstani Teacher Trainees’ Critical Thinking

A new screening test designed to measure the receptive-analytical core of critical thinking among Kazakhstani pre-service chemistry teachers has passed its first psychometric trial, but the results come with an unexpected twist: the score distribution is sharply bimodal, and nearly three in ten test-takers produced results statistically indistinguishable from random guessing. The instrument, called TKM-Chem, was developed and calibrated by Mariya Kabysheva of the Kazakh National Women’s Teacher Training University in Almaty, in a study published in Frontiers in Education. The work addresses a genuine gap: Kazakhstan’s teacher-education system, despite listing critical thinking among its general graduate outcomes, has lacked validated, domain-specific measurement tools.

The regulatory backdrop helps explain why such an instrument is needed. Kazakhstan’s State Compulsory Standard of Higher Education, approved in 2022 and updated in 2025, names critical thinking as an expected outcome for all graduates but leaves its translation into program-level outcomes to individual universities. The professional standard for teachers, revised in February 2025, does not include critical thinking among mandatory competencies. A documentary analysis of 910 teacher-education bachelor’s programs found explicit references to critical thinking in only about 2.75 percent of learning outcomes, with three Application-level verbs from Bloom’s taxonomy—apply, use, demonstrate—accounting for 42.7 percent of all verb occurrences. Foreign instruments such as the Watson-Glaser appraisal are copyrighted, developed in an English-language context, and lack documented cross-cultural adaptation for Kazakhstan.

TKM-Chem deliberately restricts its target to what the author calls the receptive-analytical tier of critical thinking: interpretation, structural analysis, and evaluative appraisal of arguments. Generative skills such as hypothesis creation and self-regulation require constructed responses and rater judgment, which are incompatible with automated scoring. Instead, each of the test’s items presents a self-contained chemistry-education dilemma in a single-best-answer format with four options, with all premises embedded in the stimulus so that no prior chemistry knowledge or reading speed advantage is needed. One exemplar item asks students to evaluate a shampoo labeled “chemical-free” whose ingredient list includes aqua and glycerin; the correct answer recognizes the inherent contradiction, while the distractors operationalize naive chemophobia and the naturalistic fallacy.

The instrument was administered in parallel Kazakh and Russian forms to 253 bachelor’s students enrolled in chemistry teacher-education programs, recruited purposively from universities across 16 regions. Data were collected from March to June 2026 through an unproctored online questionnaire, with each participant freely choosing their preferred language version. The final form contains 26 items—8 interpretation, 9 analysis, 9 evaluation—after two items were pruned in a pre-pilot for poor fit and one item, Q02, was removed for severe differential item functioning between the language versions. The mean total score was 15.89 out of 26, with a standard deviation of 7.87.

The most striking finding is the shape of the distribution. Hartigan’s dip test decisively rejects unimodality (D = 0.087, p < 0.0001), and the bimodality coefficient of 0.700 far exceeds the uniform reference value of about 0.556. In the floor band of 8 points or fewer sit 73 respondents—28.9 percent of the sample—while 77 respondents, 30.4 percent, occupy the upper band of 24 or more, and 17.8 percent obtained a perfect score. For a 26-item test with four options each, pure guessing yields an expected score of 6.5, meaning the lower cluster coincides with chance-level performance. A mixture model assigning one class a fixed guessing probability of 0.25 on all items fit decisively better than a standard model, and assigned roughly three quarters of the floor stratum to the random-responding class.

Yet the floor stratum proved not to be homogeneous. Accuracy on a cluster of six items reached 0.409, well above the 0.25 chance baseline, while accuracy on the remaining twenty items stood at 0.207, at or below chance. The same six items had already been flagged by three independent structural procedures—exploratory factor analysis, exploratory structural equation modeling, and local dependence diagnostics—as forming a reproducible cluster cutting across all three declared subscales. What unites them is not subject matter but the type of operation: in each, the object of judgment is a statement to be characterized rather than a problem to be solved. The author suggests these items may share an alternative solution path, possibly formal cues in the option sets, though the data cannot confirm or exclude this.

Structurally, the analysis delivered one robust verdict: a single general factor dominates. The bifactor model in its item-response-theory specification converged cleanly and showed the general factor explaining 87.0 percent of common variance, with hierarchical omega estimates of 0.922 to 0.947 from two independent methods. The specific reliabilities of the three subscales, once the general factor was controlled, collapsed to near zero—0.044 to 0.144—and the disattenuated correlation between subscales reached 0.913. The practical implication is unambiguous: only the total score is a defensible unit of interpretation, and using subscale scores for comparisons or individual decisions would be incorrect. This mirrors earlier findings with the Watson-Glaser test, where reanalyses repeatedly recovered one factor instead of the five theoretically posited subscales.

The headline reliability figures demand a qualified reading. Cronbach’s alpha of 0.94 and ordinal alpha of 0.97 are partly dispersion-driven, inflated by the very bimodality the test detected, and further inflated by local dependence among the six hub items. When extreme scores were truncated, the coefficients fell monotonically—alpha dropping to 0.742 in the middle band of the scale. The tetrachoric correlation matrix was not positive definite in any subsample, an expected consequence of estimating such correlations under a violated normality assumption, and this propagated into all model-based fit indices, which the author therefore presents as comparative rather than absolute. Item-level calibration itself was successful: 25 of 26 items reached the high-discrimination threshold, mean discrimination was 2.25, and the distractors functioned as intended in both language forms.

The test’s working range is concentrated in the lower-to-middle portion of the latent trait, from about −2.00 to +1.05, where measurement precision is acceptable. That is appropriate for a baseline screening tool, but it means the 45 students who achieved perfect scores are not resolved by the instrument at all, and comparisons among higher-performing cohorts would lack precision. None of 25 exploratory group comparisons—by year of study, self-rated academic performance, conference participation, or region—reached significance, though the study was underpowered for the smaller effects observed. The null differential item functioning result across languages means bias was not detected, not that equivalence was demonstrated: the Russian-language subsample of 61 affords limited statistical power.

The author is explicit about the instrument’s proper use. TKM-Chem is intended for aggregated group-level diagnosis of baseline receptive-analytical critical thinking skills and for evaluating educational interventions among groups in the lower-to-middle ability range; individual-level high-stakes decisions such as assessment, certification, or admission are methodologically inadmissible, and broad population generalizations remain unwarranted given the voluntary, geographically skewed sample. Convergent validity was not established, since no adapted external criterion of critical thinking exists for Kazakhstan, and the declared construct cannot yet be separated from test-wiseness or subject knowledge. Future work includes a parallel Form B with harder items calibrated at positive logits, a fixed-guessing item-response model, item-level response-time recording to disentangle disengagement from genuine skill deficits, and data-integrity checks on the upper cluster before the next administration. For now, the study stands as a candid example of initial calibration: a workable item pool, a dominant general factor, and a distribution whose shape is itself the most informative result.

Subject of Research: Development and psychometric calibration of the TKM-Chem critical thinking screening instrument for Kazakhstani pre-service chemistry teachers

Article Title: Development and initial psychometric calibration of a screening instrument for receptive-analytical skills in critical thinking: TKM-Chem for Kazakhstani pre-service chemistry teachers

Article References: Kabysheva, M. (2026). Development and initial psychometric calibration of a screening instrument for receptive-analytical skills in critical thinking: TKM-Chem for Kazakhstani pre-service chemistry teachers. Frontiers in Education, 11, Article 1959270. https://doi.org/10.3389/feduc.2026.1959270

Image Credits: AI Generated

DOI: 10.3389/feduc.2026.1959270

Keywords: critical thinking, psychometrics, item response theory, teacher education, Kazakhstan, chemistry education, bifactor model, differential item functioning, test construction, bimodal distribution, screening instrument, classical test theory