<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>instrument validation &#8211; Science</title>
	<atom:link href="https://scienmag.com/tag/instrument-validation/feed/" rel="self" type="application/rss+xml" />
	<link>https://scienmag.com</link>
	<description></description>
	<lastBuildDate>Sun, 11 Oct 2026 15:09:51 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://scienmag.com/wp-content/uploads/2024/07/cropped-scienmag_ico-32x32.jpg</url>
	<title>instrument validation &#8211; Science</title>
	<link>https://scienmag.com</link>
	<width>32</width>
	<height>32</height>
</image> 
<site xmlns="com-wordpress:feed-additions:1">73899611</site>	<item>
		<title>New Digital Test Measures How Students Internalize Digital Citizenship Values</title>
		<link>https://scienmag.com/new-digital-test-measures-how-students-internalize-digital-citizenship-values/</link>
		
		<dc:creator><![CDATA[Courtney Benton]]></dc:creator>
		<pubDate>Sun, 11 Oct 2026 15:09:51 +0000</pubDate>
				<category><![CDATA[Science Education]]></category>
		<category><![CDATA[affective assessment]]></category>
		<category><![CDATA[affective domain in digital education]]></category>
		<category><![CDATA[civic education]]></category>
		<category><![CDATA[confirmatory factor analysis]]></category>
		<category><![CDATA[convergent validity]]></category>
		<category><![CDATA[digital assessment]]></category>
		<category><![CDATA[digital citizenship]]></category>
		<category><![CDATA[digital citizenship assessment]]></category>
		<category><![CDATA[digital literacy and ethical online behavior]]></category>
		<category><![CDATA[digital values internalization in students]]></category>
		<category><![CDATA[educational measurement]]></category>
		<category><![CDATA[educational psychology in digital learning]]></category>
		<category><![CDATA[high school digital citizenship attitudes]]></category>
		<category><![CDATA[Indonesia]]></category>
		<category><![CDATA[innovative digital learning assessment]]></category>
		<category><![CDATA[instrument validation]]></category>
		<category><![CDATA[Likert scale]]></category>
		<category><![CDATA[measuring digital citizenship value development]]></category>
		<category><![CDATA[online responsible behavior measurement]]></category>
		<category><![CDATA[psychometric evaluation of digital citizenship tools]]></category>
		<category><![CDATA[psychometrics]]></category>
		<category><![CDATA[reliability]]></category>
		<category><![CDATA[student engagement in digital norms]]></category>
		<category><![CDATA[validation of digital citizenship measurement instruments]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=262494</guid>

					<description><![CDATA[Indonesian researchers report the first psychometric evaluation of IDBEL, a digital questionnaire measuring students' digital citizenship attitudes across five levels of value internalisation, finding strong reliability but mixed model fit and ceiling effects.]]></description>
										<content:encoded><![CDATA[<p>What does it actually mean for a teenager to be a good digital citizen? A research team from Universitas Lambung Mangkurat in Banjarmasin, Indonesia, has taken a serious run at answering that question with numbers rather than slogans. In a study published in Discover Education, Rabiatul Adawiah and colleagues report the first psychometric evaluation of a new digital assessment tool called IDBEL, short for Innovative Digital-Based Learning Evaluation. The instrument is designed to measure not whether students know the rules of responsible online behaviour, but whether they have genuinely absorbed the values behind them. With 661 high school students, five expert reviewers, and a battery of statistical tests, the study offers one of the most transparent early-stage validations of a digital citizenship attitude instrument to date, and its findings are as instructive for what they reveal about measurement as for what they reveal about students.</p>
<p>The conceptual heart of IDBEL is a decades-old idea from educational psychology: the affective domain. Drawing on the classic taxonomy of Krathwohl, Bloom, and Masia, the researchers treat digital citizenship attitudes as a process of progressive value internalisation with five levels. Receiving captures a student&#8217;s initial awareness and acceptance of digital norms. Responding reflects active engagement with those norms. Valuing indicates genuine appreciation of responsible digital behaviour. Organization describes the integration of digital values into a coherent personal value system, and characterization represents consistent, value-driven conduct online. Each of the five dimensions was translated into five Likert-scale statements, producing a 25-item questionnaire delivered entirely through a digital application, so that the mode of assessment mirrors the digital construct being measured.</p>
<p>The technical groundwork began before any student saw the instrument. Five experts in educational assessment, language, civic education, and information technology rated the relevance of each of the 25 initial items on a five-point scale, and the researchers quantified agreement using Aiken&#8217;s V, a content-validity coefficient. The scores ranged from 0.70 to 0.95, with a mean of 0.83, and the team applied an operational threshold of 0.80. Twenty items cleared the bar; five, specifically Items 3, 5, 10, 13, and 20, fell short and were revised in response to expert feedback. Notably, the researchers acknowledge that the revised versions were never re-rated, so the content-validity evidence describes the initial drafts rather than a guarantee that every final item meets the criterion. That kind of candour is rare in instrument-development papers and sets the tone for the entire study.</p>
<p>Data collection ran from October 2025 to February 2026 through the IDBEL application, drawing Grade X and XI students from seven public senior secondary schools in Banjarmasin, selected purposively to span urban and peri-urban contexts. Grade XII students were excluded because of final examinations. The final analytical matrix contained 661 students, with school contributions ranging from 34 to 142 participants. The descriptive results told a familiar story with an uncomfortable twist: students overwhelmingly reported positive digital citizenship attitudes, with combined high and very high responses ranging from 66.0 percent to 97.4 percent across items. One item, Item 12, drew a maximum-category endorsement from 61.4 percent of respondents, while another, Item 23, drew only 14.5 percent. That spread matters, because when responses pile up at the top of a scale, the instrument loses its ability to distinguish between students who are good and students who are exceptional.</p>
<p>Before any structural modelling, the team tested whether the data were even suitable for factor analysis, and the answer was emphatically yes. The Kaiser-Meyer-Olkin measure of sampling adequacy reached 0.963, a value considered excellent, and item-level adequacy statistics ranged from 0.930 to 0.980. Bartlett&#8217;s test of sphericity was overwhelmingly significant, χ²(300) = 9,751.236, p &lt; 0.001, confirming that the correlation matrix contained enough shared variance to justify extracting factors. In plain terms, the items were related to one another in exactly the way a coherent attitude construct should be. The question then became whether the relationships followed the five-dimensional architecture the theorists had prescribed.</p>
<p>The answer, delivered through a prespecified second-order confirmatory factor analysis, was a qualified no. Standardized item loadings were respectable, ranging from 0.567 to 0.776, and the second-order loadings linking the five dimensions to the overarching attitude construct were extremely high, between 0.929 and 0.971. But the overall model fit fell short of accepted standards: the chi-square ratio of 5.567 was high, RMSEA came in at 0.083, just above the conventional 0.08 guideline, and the incremental fit indices CFI, TLI, and IFI all landed below 0.90, at 0.871, 0.857, and 0.872 respectively. Only SRMR, at 0.054, was comfortably within target. The researchers also flagged a methodological caveat: they used normal-theory maximum likelihood on Pearson correlations, which is suboptimal for ordinal items concentrated in upper response categories, and they call for an ordinal CFA using polychoric correlations and a weighted estimator in future work.</p>
<p>Perhaps the most intriguing technical finding concerns what those very high second-order loadings imply. Together with a McDonald&#8217;s omega hierarchical of 0.940, they suggest that a single dominant general factor accounts for most of the reliable variance in scores. In other words, the five affective dimensions may be so conceptually intertwined, progressive stages of the same internalisation process, that they cannot yet be interpreted as distinct subscales. Modification diagnostics reinforced this concern, identifying strong local dependencies between Items 1 and 2 and between Items 23 and 24, with additional cross-dimensional overlap for pairs such as Items 4 and 8 and Items 15 and 25. Crucially, the team refused the temptation to patch the model post hoc by correlating error terms or deleting items, arguing that such changes require content justification and independent-sample cross-validation.</p>
<p>Reliability, by contrast, was stellar. Cronbach&#8217;s alpha reached 0.953 for the total scale, McDonald&#8217;s omega total was 0.958, and corrected item-total correlations ranged from 0.489 to 0.727, with no single item&#8217;s removal materially improving consistency. Domain-level alpha coefficients ranged from 0.792 to 0.854 and composite reliability from 0.806 to 0.854, all comfortably above the 0.70 threshold. Yet the convergent validity picture was murkier. Average variance extracted, a measure of how much variance a latent dimension explains in its own indicators, ranged from 0.456 to 0.540 and crossed the conventional 0.50 criterion only for the Organization dimension, with Characterization sitting essentially at the threshold. The lesson the authors draw is an important one for anyone who reads assessment research: very high internal consistency does not by itself establish that an instrument measures what it claims, and it may partly reflect item similarity and response concentration rather than construct strength.</p>
<p>So where does IDBEL stand? The researchers position it deliberately as initial psychometric evidence, not a finished, validated scale. The study&#8217;s limitations are laid out with unusual precision: the sample covers one Indonesian city and excludes vocational schools and Grade XII students; the data are entirely self-reported and vulnerable to social desirability bias, since digital citizenship items describe morally desirable behaviour; and the validity argument remains incomplete, lacking evidence on relations to external variables, measurement invariance, test-retest reliability, and the consequences of test use. Future refinements they propose include scenario-based items presenting realistic digital dilemmas, carefully constructed reverse-coded items to counter acquiescence, and Item Response Theory or Rasch analysis to map item difficulty and discrimination, particularly at the upper end of the attitude continuum where current items blur together.</p>
<p>For educators, the practical message is twofold. First, digital citizenship can indeed be measured as an affective construct, not merely as technical skill, and a digital platform is a natural home for such assessment in civic education classrooms. Second, caution is warranted: IDBEL is suitable as a formative, low-stakes tool and a foundation for further development, but it should be paired with teacher observations, student reflections, and scenario-based tasks rather than used for high-stakes judgements. In an era when schools everywhere are being asked to certify not just what students know but who they are online, this study offers both a promising template and a sobering reminder that building a trustworthy measure of character is harder, and more honest, work than it looks.</p>
<p><strong>Subject of Research:</strong> Psychometric validation of a digital instrument assessing students&#x27; digital citizenship attitudes</p>
<p><strong>Article Title:</strong> Initial psychometric evidence of an innovative digital-based learning evaluation instrument for assessing students’ digital citizenship attitudes</p>
<p><strong>Article References:</strong> Adawiah, R., Ruchliyadi, D. A., &amp; Nugroho, D. A. (2026). Initial psychometric evidence of an innovative digital-based learning evaluation instrument for assessing students’ digital citizenship attitudes. <em>Discover Education, 5</em>(1), Article 997. <a href="https://doi.org/10.1007/s44217-026-02201-6" rel="noopener noreferrer">https://doi.org/10.1007/s44217-026-02201-6</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44217-026-02201-6" rel="noopener noreferrer">10.1007/s44217-026-02201-6</a></p>
<p><strong>Keywords:</strong> digital citizenship, psychometrics, affective assessment, instrument validation, confirmatory factor analysis, digital assessment, civic education, reliability, convergent validity, Likert scale, Indonesia, educational measurement</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">262494</post-id>	</item>
		<item>
		<title>New Scale Measures How Ready Future Teachers Are to Embrace Group Dynamics</title>
		<link>https://scienmag.com/new-scale-measures-how-ready-future-teachers-are-to-embrace-group-dynamics/</link>
		
		<dc:creator><![CDATA[Courtney Benton]]></dc:creator>
		<pubDate>Fri, 02 Oct 2026 02:11:37 +0000</pubDate>
				<category><![CDATA[Social Science]]></category>
		<category><![CDATA[active learning]]></category>
		<category><![CDATA[case studies in education]]></category>
		<category><![CDATA[confirmatory factor analysis]]></category>
		<category><![CDATA[cooperative learning techniques]]></category>
		<category><![CDATA[educational research]]></category>
		<category><![CDATA[evaluation of teacher training programs]]></category>
		<category><![CDATA[factorial invariance]]></category>
		<category><![CDATA[group dynamics]]></category>
		<category><![CDATA[group dynamics in secondary education]]></category>
		<category><![CDATA[impact of group interaction on student learning]]></category>
		<category><![CDATA[instrument validation]]></category>
		<category><![CDATA[measurement of pre-service teachers' readiness]]></category>
		<category><![CDATA[modern classroom teaching methods]]></category>
		<category><![CDATA[pre-service teachers]]></category>
		<category><![CDATA[psychometrics]]></category>
		<category><![CDATA[role-playing in teacher training]]></category>
		<category><![CDATA[secondary education]]></category>
		<category><![CDATA[Spain]]></category>
		<category><![CDATA[teacher training]]></category>
		<category><![CDATA[teacher training assessment]]></category>
		<category><![CDATA[teamwork skills for future teachers]]></category>
		<category><![CDATA[tripartite model of attitudes]]></category>
		<category><![CDATA[University of Zaragoza educational research]]></category>
		<category><![CDATA[validation of educational questionnaires]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=225058</guid>

					<description><![CDATA[Spanish researchers have designed and validated a fifteen-item questionnaire that reliably measures whether trainee secondary school teachers are willing to use group dynamics techniques in their future classrooms.]]></description>
										<content:encoded><![CDATA[<p>Every year, thousands of graduates across Spain enrol on a one-year master&#8217;s programme that will decide whether they stand in front of a secondary school classroom. The Master&#8217;s Degree in Secondary School Teaching, launched in the 2009/2010 academic year, is the compulsory gateway into the teaching profession for subject specialists in fields ranging from history to engineering. Yet a persistent question has haunted teacher educators: how can we measure whether these trainees are actually willing to use the group-based, interactive methods that modern classrooms demand? A new study from the University of Zaragoza offers an answer in the form of a rigorously validated questionnaire, and its findings could reshape how teacher training programmes are designed and evaluated.</p>
<p>The research, published in the Journal of New Approaches in Educational Research, was conducted by Pablo Palomero-Fernández and Eva Vicente-Sánchez, who set out to build and validate an instrument measuring the willingness of pre-service secondary teachers to use group dynamics techniques. These techniques, rooted in the classic work of Kurt Lewin and later codified by Cartwright and Zander, encompass methods such as cooperative learning, role-playing, case studies and teamwork, all of which harness group interaction to improve both learning and interpersonal relationships. Although such methods are widely promoted across European higher education, no standardised tool existed to assess whether trainee teachers genuinely intend to apply them once they enter the profession.</p>
<p>The theoretical foundation of the new instrument is the tripartite model of attitudes proposed by Rosenberg and Hovland in 1960, a framework that remains one of the most influential structures in social psychology. According to this model, any attitude can be decomposed into three components: cognitive beliefs, emotional responses and behavioural tendencies. The researchers translated this structure into three subscales. The first, Perceived Usefulness, captures what trainees believe about the value of group techniques for their students. The second, Emotional Experience, probes the feelings these techniques arouse during initial training. The third, Behavioural Engagement, measures the interest and intention to actually programme and deploy group methods in future professional practice.</p>
<p>Building a psychometrically sound instrument is a painstaking process, and the researchers followed the standardised ten-step procedure for test construction recommended by Muñiz and Fonseca. The first version of the questionnaire contained twelve items, four for each subscale, each rated on a five-point Likert scale from complete disagreement to complete agreement. To establish content validity, the draft was scrutinised by a panel of three expert judges from a Spanish university, selected against demanding criteria including at least ten years of experience in educational psychology or pedagogy, a decade of team leadership, and substantial experience teaching the master&#8217;s programme itself.</p>
<p>The expert judgement phase proved decisive. While the experts rated most items highly for clarity, coherence and relevance, statistical analysis using Aiken&#8217;s V content validity coefficient revealed unsatisfactory agreement regarding the adequacy of the three subscales as a whole. The qualitative feedback was rich and specific. One expert noted that the usefulness subscale focused too heavily on content and recommended adding items about participation, engagement and collaboration centred on students. Another highlighted the absence of variables such as perceived self-efficacy and the perceived control a future teacher has over the possibility of using these dynamics, a factor with direct influence on behaviour. Following this feedback, the researchers added new items, reworded others and reframed emotional experience items within the specific context of the group techniques trainees had encountered during their master&#8217;s coursework.</p>
<p>A second round of expert judgement confirmed the improvements. All items achieved Aiken&#8217;s V values above .71 in the lower bound of their confidence intervals, with sixteen items exceeding .80 and nine approaching the maximum. Overall agreement for the entire instrument reached an exceptional .93, and the adequacy of all three subscales was judged satisfactory. With content validity secured, the researchers turned to the field: 441 students, 161 men and 280 women, enrolled on the master&#8217;s programme at a Spanish university during the 2020/2021 academic year completed the questionnaire. This represented 61.33 percent of the total enrolment, with a mean age of 27.6 years and a strong representation of Arts and Humanities backgrounds at nearly half the sample.</p>
<p>The statistical analysis revealed both strengths and weaknesses in the draft instrument. Initial reliability coefficients were respectable, with Cronbach&#8217;s alpha at .853 and McDonald&#8217;s omega at .870, the latter being the preferred estimator for ordinal data. However, item-total correlation analysis flagged three items falling below the .300 threshold, indicating they contributed nothing to internal consistency. These items were removed, leaving a refined fifteen-item scale whose reliability improved to alpha of .871 and omega of .883, with all three subscales exceeding .700. The researchers then subjected the refined scale to confirmatory factor analysis, and the results were compelling: sampling adequacy was excellent with a Kaiser-Meyer-Olkin value of .915, and the three-factor model showed a good fit with a comparative fit index of .929, a Tucker-Lewis index of .914 and a root mean square error of approximation of .063, all within accepted benchmarks.</p>
<p>One of the study&#8217;s most technically sophisticated contributions is the multigroup analysis examining whether the instrument functions equivalently for men and women. Previous research had suggested that female trainees show greater willingness to adopt active methodologies, raising the question of whether a single questionnaire could fairly measure the construct across both groups. The researchers tested four increasingly constrained structural equation models, from an unconstrained baseline through measurement weights, structural covariances and measurement residuals. The measurement weights model, which assumes that the weight of each item on its factor is identical across groups, provided the best fit. All regression weights were statistically significant for both sexes, and factor correlations exceeded .797 for women and .829 for men, supporting factorial invariance and confirming that the scale measures the same construct in men and women.</p>
<p>The practical implications extend well beyond psychometrics. The authors argue the instrument could support selection and guidance processes for admission to the master&#8217;s programme, complementing interviews and other tests in line with recommendations from the OECD and the European Commission about attracting better-qualified teaching candidates. Within the programme itself, the scale can identify different dispositional profiles, allowing educators to tailor support for trainees who are sceptical, anxious or merely unfamiliar with group methods, and to stimulate emotional self-awareness and reflection on the advantages and disadvantages of group techniques in secondary education. Perhaps most powerfully, the instrument enables pre-post research designs: administering it before and after the master&#8217;s course would allow programmes to demonstrate empirically whether they succeed in fostering the attitudes that drive methodological change.</p>
<p>The researchers are candid about the limitations of their work. The sample was drawn from a single university using convenience sampling, which may limit generalisability, and self-selection bias could have under-represented less engaged students. Arts and Humanities students were over-represented while Health Sciences accounted for barely seven percent of participants. As a newly developed instrument, the results remain provisional, and the authors call for replication with representative nationwide samples, application to in-service teachers, and further construct validity evidence using techniques such as exploratory structural equation modelling. Nevertheless, the study delivers something teacher education has long lacked: a validated, theoretically grounded and sex-invariant measure of whether future secondary teachers are genuinely prepared to abandon lecture-based transmission in favour of the interactive, group-centred classrooms that educational policy has demanded for decades. In an era when student motivation and participation are central concerns, knowing not just what trainees can do but what they are willing to do may prove the decisive metric.</p>
<p><strong>Subject of Research:</strong> Design and validation of a questionnaire measuring pre-service secondary teachers&#x27; willingness to use group dynamics techniques</p>
<p><strong>Article Title:</strong> Assessing the willingness of Spanish pre-service secondary school teachers to use group dynamics techniques. Design and validation of an instrument</p>
<p><strong>Article References:</strong> Palomero-Fernández, P., &amp; Vicente-Sánchez, E. (2025). Assessing the willingness of Spanish pre-service secondary school teachers to use group dynamics techniques. Design and validation of an instrument. <em>Journal of New Approaches in Educational Research, 14</em>(1), Article 16. <a href="https://doi.org/10.1007/s44322-025-00037-w" rel="noopener noreferrer">https://doi.org/10.1007/s44322-025-00037-w</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44322-025-00037-w" rel="noopener noreferrer">10.1007/s44322-025-00037-w</a></p>
<p><strong>Keywords:</strong> teacher training, group dynamics, psychometrics, instrument validation, active learning, pre-service teachers, secondary education, tripartite model of attitudes, confirmatory factor analysis, factorial invariance, Spain, educational research</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">225058</post-id>	</item>
		<item>
		<title>New Questionnaire Aims to Measure How Staff Judge Mechanical Restraint in Psychiatric Care</title>
		<link>https://scienmag.com/new-questionnaire-aims-to-measure-how-staff-judge-mechanical-restraint-in-psychiatric-care/</link>
		
		<dc:creator><![CDATA[Glenn Wilkins]]></dc:creator>
		<pubDate>Thu, 01 Oct 2026 00:23:39 +0000</pubDate>
				<category><![CDATA[Psychology & Psychiatry]]></category>
		<category><![CDATA[acute mental health care]]></category>
		<category><![CDATA[clinical guidelines for restraint use]]></category>
		<category><![CDATA[development of Mechanical Restraint Questionnaire (MR-Q)]]></category>
		<category><![CDATA[ethical and legal considerations in psychiatric restraint]]></category>
		<category><![CDATA[exploratory factor analysis]]></category>
		<category><![CDATA[instrument validation]]></category>
		<category><![CDATA[internal consistency]]></category>
		<category><![CDATA[mechanical restraint]]></category>
		<category><![CDATA[Mechanical restraint assessment in psychiatric care]]></category>
		<category><![CDATA[mental health restraint intervention evaluation]]></category>
		<category><![CDATA[mental health staff training and assessment tools]]></category>
		<category><![CDATA[Norway]]></category>
		<category><![CDATA[nurses]]></category>
		<category><![CDATA[patient safety]]></category>
		<category><![CDATA[physical and psychological risks of mechanical restraint]]></category>
		<category><![CDATA[pilot study]]></category>
		<category><![CDATA[pilot study on restraint measurement in psychiatric wards]]></category>
		<category><![CDATA[psychiatry]]></category>
		<category><![CDATA[psychometric testing of psychiatric restraint measures]]></category>
		<category><![CDATA[psychometrics]]></category>
		<category><![CDATA[questionnaire development]]></category>
		<category><![CDATA[staff decision-making during restraint episodes]]></category>
		<category><![CDATA[staff perceptions of mechanical restraint]]></category>
		<category><![CDATA[validated instruments for psychiatric restraint assessment]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=220258</guid>

					<description><![CDATA[A Norwegian pilot study reports the development and first psychometric testing of the Mechanical Restraint Questionnaire, a 61-item tool designed to capture how nurses and ward staff perceive assessments before, during and after mechanical restraint episodes.]]></description>
										<content:encoded><![CDATA[<p>Mechanical restraint is one of the most drastic interventions available in acute mental health care. When a patient poses an immediate danger to themselves or others, staff on psychiatric wards may secure the person&#8217;s limbs or torso to a bed with belts, a measure that is legally regulated, ethically fraught and physically and psychologically risky. The practice can cause unexpected harm, ranging from physical injury to lasting mental trauma, which is why clinical guidelines demand continuous assessment and decision-making before, during and after every restraint episode. Yet until now, researchers have had remarkably few validated instruments to measure how the health care professionals responsible for those assessments actually perceive and carry out this demanding part of their work.</p>
<p>A pilot study published in BMC Psychiatry by researchers at Molde University College in Norway set out to close that gap. The team, led by Liv Bachmann together with Cecilie K. Utheim Grønvik, Atle Ødegård and Ingunn Mundal, developed the Mechanical Restraint Questionnaire, or MR-Q, and subjected it to its first round of psychometric testing. The instrument is a self-report questionnaire designed to assess nurses&#8217; and ward staff members&#8217; perceptions of the assessments related to the use of mechanical restraint, and its debut represents a small but potentially consequential step toward making restraint practice measurable, comparable and ultimately improvable.</p>
<p>The construction of the MR-Q followed a deliberately multi-source approach to item development. Rather than relying on a single theoretical framework, the authors drew on individual interviews with clinicians, an existing clinical guideline, and a review of the literature to generate candidate items. This triangulation matters because restraint assessment is not a single act but a chain of judgments: deciding whether restraint is necessary, monitoring the patient while it is in place, and evaluating the aftermath. The final version of the questionnaire contained 61 items, of which 26 addressed the practice environment, meaning the organizational and contextual conditions under which restraint decisions are made, and 35 addressed the caring process, the clinical and relational work that surrounds each restraint episode.</p>
<p>Data were collected in a cross-sectional design from 69 nurses and ward staff working on acute mental health wards in Norway. That sample size is modest, and the authors are explicit that this is a pilot study, but it was sufficient to begin probing the questionnaire&#8217;s underlying structure using standard techniques from psychometrics. The analysis pipeline combined parallel analysis, visual inspection of the scree plot, and exploratory factor analysis, the workhorse method for discovering how many latent dimensions a set of items actually taps. Parallel analysis, which compares observed eigenvalues against those generated by random data, suggested a possible six-factor solution, and the exploratory factor analysis ultimately extracted two dimensions with three factors in each.</p>
<p>The resulting structure has an intuitive logic that mirrors the questionnaire&#8217;s design. Three of the six factors covered the practice environment and three covered the caring process, exactly the two domains the item pool had been built around. In terms of explained variance, the three practice-environment factors together accounted for 69.3 percent of the total variance in that domain, while the three caring-process factors accounted for 52.0 percent of the variance in theirs. For a first-pass instrument measuring complex professional perceptions, those figures indicate that the items cluster into coherent dimensions rather than scattering randomly, which is precisely what construct validation is supposed to reveal at this stage.</p>
<p>Beyond the factor structure, the team assessed the questionnaire through face validity, content validity, construct validity and reliability testing of the identified subscales. Reliability, typically evaluated through internal consistency measures that gauge how well items within a subscale correlate with one another, came out in the acceptable to high range. In practical terms, this means that the subscales behave as unified scales rather than loose collections of loosely related questions, a prerequisite for anyone hoping to use the MR-Q to compare wards, track changes over time or evaluate interventions aimed at reducing restraint use.</p>
<p>Why does a measurement instrument for staff perceptions warrant this much attention? The answer lies in the peculiar epistemic position of restraint in psychiatry. Because the intervention happens behind closed ward doors, often in emergencies, much of what is known about how restraints are decided, applied and reviewed depends on the recollections and judgments of the professionals involved. Patient perspectives are increasingly collected, but the staff side of the encounter has been documented mostly through audits and incident reports, which capture events rather than the perceptions and assessments that shape them. A validated questionnaire turns that tacit clinical territory into data that can be quantified, benchmarked and studied statistically.</p>
<p>The potential applications follow from that. If the MR-Q performs well in larger samples, hospital managers could use it to map how assessment practices vary across wards, identifying units where staff feel less supported by the practice environment or less confident in the caring process. Researchers could use it as an outcome measure in studies testing whether specific interventions, such as debriefing routines, staffing changes or training programs, actually change how restraint-related assessments are perceived and performed. Regulators and quality-improvement programs, which in many countries are under pressure to reduce coercive measures, would gain a standardized way to monitor the conditions under which restraint is used rather than merely counting how often it happens.</p>
<p>The authors are careful about the limits of what this pilot can claim. Sixty-nine respondents is a small sample for factor-analytic work, and the participants came from acute mental health wards in a single country, raising questions about whether the six-factor structure would replicate in more heterogeneous professional groups and health systems. The team states that some improvements to the questionnaire are needed and that further investigation of the reliability and validity in a larger and more heterogeneous sample is required. In psychometric practice, the natural next step after exploratory factor analysis is confirmatory factor analysis on an independent sample, a test the present study did not include, and the abbreviation list of the paper itself signals that confirmatory work is on the research agenda.</p>
<p>Even with those caveats, the study fills a genuine void. Coercive measures in psychiatry are under intensifying scrutiny worldwide, with policy frameworks pushing toward prevention and, where restraint is unavoidable, toward rigorous documentation and review. Tools like the MR-Q give that agenda an empirical handle on the professional side of the equation, measuring not just whether restraints happen but how the people who carry them out perceive the assessments that are supposed to keep patients safe. As the questionnaire moves from pilot testing toward validation in larger and more diverse samples, it may become a standard instrument in a field that has long lacked one, turning a hidden dimension of psychiatric practice into something that can finally be seen, compared and improved.</p>
<p><strong>Subject of Research:</strong> Development and psychometric testing of a questionnaire measuring healthcare professionals&#x27; perceptions of mechanical restraint assessment in acute mental health care</p>
<p><strong>Article Title:</strong> Development and psychometric testing of the Mechanical Restraint Questionnaire (MR-Q): a pilot study</p>
<p><strong>Article References:</strong> Bachmann, L., Utheim Grønvik, C. K., Ødegård, A., &amp; Mundal, I. (2026). Development and psychometric testing of the Mechanical Restraint Questionnaire (MR-Q): a pilot study. <em>BMC Psychiatry</em>. <a href="https://doi.org/10.1186/s12888-026-08691-w" rel="noopener noreferrer">https://doi.org/10.1186/s12888-026-08691-w</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1186/s12888-026-08691-w" rel="noopener noreferrer">10.1186/s12888-026-08691-w</a></p>
<p><strong>Keywords:</strong> mechanical restraint, psychiatry, psychometrics, questionnaire development, exploratory factor analysis, nurses, acute mental health care, internal consistency, pilot study, instrument validation, patient safety, Norway</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">220258</post-id>	</item>
		<item>
		<title>New validated observation tool spots great digital teaching in real classrooms</title>
		<link>https://scienmag.com/new-validated-observation-tool-spots-great-digital-teaching-in-real-classrooms/</link>
		
		<dc:creator><![CDATA[Courtney Benton]]></dc:creator>
		<pubDate>Sat, 12 Sep 2026 22:02:50 +0000</pubDate>
				<category><![CDATA[Social Science]]></category>
		<category><![CDATA[Activity Theory]]></category>
		<category><![CDATA[activity theory-based educational research]]></category>
		<category><![CDATA[classroom digital technology assessment]]></category>
		<category><![CDATA[classroom observation]]></category>
		<category><![CDATA[classroom observation in digital learning]]></category>
		<category><![CDATA[content validity]]></category>
		<category><![CDATA[cross-level education digital practices]]></category>
		<category><![CDATA[Delphi method]]></category>
		<category><![CDATA[Design-Based Research]]></category>
		<category><![CDATA[Digital teaching observation tool]]></category>
		<category><![CDATA[digital technologies]]></category>
		<category><![CDATA[educational practices]]></category>
		<category><![CDATA[educational research]]></category>
		<category><![CDATA[educational technology quality indicators]]></category>
		<category><![CDATA[effective digital pedagogy recognition]]></category>
		<category><![CDATA[evidence-based teaching practices]]></category>
		<category><![CDATA[instrument validation]]></category>
		<category><![CDATA[inter-rater reliability]]></category>
		<category><![CDATA[international education technology standards]]></category>
		<category><![CDATA[teacher digital competence]]></category>
		<category><![CDATA[teacher professional development digital tools]]></category>
		<category><![CDATA[teaching and learning]]></category>
		<category><![CDATA[technology integration in classrooms]]></category>
		<category><![CDATA[validated educational practice guidelines]]></category>
		<guid isPermaLink="false">https://scienmag.com/?p=199052</guid>

					<description><![CDATA[Spanish researchers have designed and validated a classroom observation guideline, grounded in Activity Theory and refined through Delphi expert consensus and pilot testing, to identify good educational practices with digital technologies across all educational levels.]]></description>
										<content:encoded><![CDATA[<p>Researchers in Spain have developed and rigorously validated a direct classroom observation guideline designed to identify good educational practices that incorporate digital technologies, and the new instrument works across every level of education, from early childhood classrooms to university lecture halls. The study, published in the Journal of New Approaches in Educational Research, describes a systematic design process grounded in Activity Theory and refined through expert consensus and real-world pilot testing, offering schools, administrators and researchers a practical, evidence-based way to recognize teaching that genuinely improves learning with technology rather than merely using it.</p>
<p>The motivation for the work stems from a persistent gap in education systems worldwide. Governments and educational administrations increasingly call for the identification and dissemination of good practices with digital technologies in order to improve teaching quality, and international repositories such as the European School Education Platform and the UNESCO ICT in Education Repository collect and share such experiences. Yet defining what actually counts as a good practice has remained surprisingly elusive. Previous literature reviews have highlighted criteria such as effectiveness, transformative effect, sustainability, legitimacy and replicability, while regional agencies in Spain have emphasized features including curriculum integration, institutional support, improved outcomes, interdisciplinarity, inclusion and the involvement of the educational community. Without clear, observable criteria, judgments about quality risk being subjective and inconsistent.</p>
<p>To anchor the instrument in a coherent theoretical framework, the research team, based at the Department of Pedagogy of Universitat Rovira i Virgili, turned to Activity Theory, a cultural-historical framework originally developed by Vygotsky, Leontiev and Engestrom. Activity Theory conceives educational practice as an intentional, object-oriented action involving students and teachers working toward a common goal, mediated by physical and sociocultural tools. The theory models an activity system through its core components, including the subject, tools, community, rules, division of labor and object, and it emphasizes the dynamic, dialectical interactions among these elements. This holistic view aligns closely with the structure of a learning situation and allows observers to detect tensions, contradictions and disruptions within the classroom activity system, which the researchers see as essential to understanding how digital technologies are actually integrated into teaching and learning.</p>
<p>The study adopted a Design-Based Research approach, a methodology structured around iterative cycles of analysis, design, development and evaluation. The first cycle began with a systematic literature review following the PRISMA 2020 model, searching databases including Web of Science, Scopus, ERIC, SciELO and Dialnet for publications from 2013 to 2023 describing educational practices with digital technologies at any educational level. From an initial pool of 718 publications, 27 studies were selected for full content analysis using ATLAS.ti software. This analysis identified 26 variables linked to the seven components of Activity Theory, each operationally defined with its theoretical meaning, the type of evidence required for observation and its intended use within the guideline, forming the initial prototype of the instrument.</p>
<p>Validation proceeded through the Delphi method, a structured technique for aggregating expert judgment over successive rounds. The team formed a coordinating group and recruited a panel of experts across four professional profiles: teachers from early childhood, primary and secondary education; university professors; researchers; and professionals involved in educational management. Thirty-two experts were invited and 26 completed the competence questionnaire, which calculated an Expert Competence Index combining self-assessed knowledge and the quality of argumentation sources. Twenty-three experts scored at or above the 0.8 threshold denoting high competence and continued into the Delphi rounds, a panel size consistent with established guidance that Delphi studies typically require between 15 and 35 participants.</p>
<p>The three Delphi rounds progressively sharpened the instrument. In the first round, 23 experts ranked the variables derived from each Activity Theory component, and the researchers computed a Relative Importance Index to convert the rankings into interpretable percentages, selecting the two highest-priority variables per component to yield 14 variables and 14 indicators. In the second round, 20 experts rated the clarity and relevance of the proposed indicators, with most indicators exceeding 85 percent positive validation and five achieving unanimous acceptance; all indicators met the Content Validity Index threshold of at least 78 percent, the criterion the team used to classify indicators as excellent. In the third round, nine experts assessed their satisfaction with the revised indicators, and mean scores rose while standard deviations fell, with two indicators reaching a perfect average of 4.00 with zero variability, confirming that the iterative refinements had addressed the panel&#8217;s concerns.</p>
<p>With the validated prototype in hand, the team moved to pilot testing in authentic educational settings, an essential step for establishing external validity and feasibility. The pilot involved eight educational practices distributed across four stages, early childhood, primary, secondary and higher education, with two practices per stage, all hosted by different public institutions that had previously collaborated with the university. Sessions lasted one hour in the school stages and two hours at university, with class sizes ranging from 28 students in early childhood settings to as many as 80 university students. Researchers observed each practice simultaneously using individual copies of the guideline and then jointly interviewed the observed teachers with a complementary 13-question semi-structured interview guide, allowing triangulation of the observational data.</p>
<p>Reliability analyses during the pilot demonstrated that different observers could apply the instrument consistently. Weighted Cohen&#8217;s Kappa values ranged from 0.64 to 1.00 across the 14 indicators, with 10 of the 14 indicators exceeding 0.80, indicating substantial to almost perfect agreement according to commonly accepted interpretive benchmarks. Intraclass Correlation Coefficients, calculated under the two-way random-effects absolute-agreement model, ranged from 0.67 to 1.00 for single measures, while the global ICC reached 0.89 for single measures and 0.94 for average measures. These figures confirm that the guideline is robust, internally coherent and replicable across observers and contexts, a critical property for any instrument intended to support fair evaluation of teaching quality.</p>
<p>The pilot also generated concrete improvements to the instrument&#8217;s design and usability. The researchers added a new section capturing contextual information such as the observed teacher&#8217;s name, type of session, school type and dimensions, and technological devices used, reoriented the layout from horizontal to vertical for easier field use, reorganized indicators so that all indicators for each component appeared on the same page, expanded the open-observation field from three to seven lines, and moved illustrative examples directly into the guideline itself. One indicator was reworded to replace a conjunction with a disjunctive form, broadening its scope to capture situations in which community resources or external agents are identified but not necessarily used. Post-pilot analysis showed that 58 percent of the modifications involved new sections, 32 percent design adjustments and 10 percent content changes.</p>
<p>Beyond its immediate technical achievement, the instrument carries significant practical potential. The authors note that teacher training has traditionally emphasized technical mastery of digital tools rather than their pedagogical integration, and the observation guideline is well positioned to serve as a formative assessment tool, a self-reflection resource in mentoring programs, or an evidence-based component in departmental evaluations of technology-enhanced teaching. Its flexible design allows adaptation to different disciplines, educational levels and modalities, including hybrid and virtual environments. The authors caution that further research is needed, including criterial and convergent validation, sensitivity testing to detect changes in teaching practice over time, assessment of transferability across educational systems, and development of an English-language version, but the validated guideline already represents a substantial step toward making the evaluation of good digital teaching practices systematic, transparent and grounded in direct evidence from real classrooms.</p>
<p><strong>Subject of Research:</strong> Design and validation of an Activity Theory-based observation instrument for identifying good educational practices with digital technologies through direct classroom observation</p>
<p><strong>Article Title:</strong> Good educational practices with digital technologies: validation of an instrument for direct classroom observations</p>
<p><strong>Article References:</strong> Good educational practices with digital technologies: validation of an instrument for direct classroom observations. (n.d.). <a href="https://doi.org/10.1007/s44322-026-00065-0" rel="noopener noreferrer">https://doi.org/10.1007/s44322-026-00065-0</a></p>
<p><strong>Image Credits:</strong> AI Generated</p>
<p><strong>DOI:</strong> <a href="https://doi.org/10.1007/s44322-026-00065-0" rel="noopener noreferrer">10.1007/s44322-026-00065-0</a></p>
<p><strong>Keywords:</strong> digital technologies, educational practices, classroom observation, Activity Theory, Delphi method, instrument validation, teacher digital competence, Design-Based Research, educational research, inter-rater reliability, content validity, teaching and learning</p>
]]></content:encoded>
					
		
		
		<post-id xmlns="com-wordpress:feed-additions:1">199052</post-id>	</item>
	</channel>
</rss>
