Scientists Build First Validated Scale to Measure Irrational Behavior in Students’ Online Talk

Every day, millions of undergraduate students pour their thoughts into comment sections, group chats, and social media threads. Most of that discourse is ordinary and harmless, but psychologists have long suspected that a measurable slice of it is driven by something less rational: snap judgments, emotional flare-ups, impulsive posting, and outright deviation from reasoned argument. The problem has been that nobody had a reliable way to measure it. A team of Chinese researchers now reports in BMC Psychology that they have built and validated what they describe as a scientifically grounded scale for quantifying irrational behavior in undergraduate students’ online discourse, offering universities a standardized instrument for understanding how students behave when they argue, vent, and react behind a screen.

The study, led by Zhifang Liu of Shaanxi Normal University and Hunan First Normal University, together with Chengyu Zhang, Jing Li, and Bihua Yan, was motivated by a practical concern that has grown acute in the era of viral campus controversies: university administrators and researchers can observe online public opinion erupting in real time, but they lack validated tools to identify which students or which patterns of discourse signal genuine risk. Existing instruments in psychology measure related constructs such as cognitive distortions or impulsivity in general contexts, but the researchers argue that online discourse has its own dynamics—anonymity, immediacy, audience effects, and the amplifying architecture of platforms—that demand a purpose-built measurement tool. Their answer was to construct one from the ground up using a mixed-methods design that mirrors the gold-standard sequence of modern scale development.

The qualitative phase began with the raw material of student experience. The team conducted semi-structured interviews and collected 153 valid open-ended questionnaires, asking students to describe how they and their peers behave irrationally in online discussions. Rather than imposing a pre-existing theory, the researchers let the students’ own language drive the process. Five independent coders worked through the responses, systematically extracting recurring themes and translating them into candidate questionnaire items. This coding exercise generated an initial item pool organized around four theoretical dimensions, a structure that would later be tested statistically. The draft items were then scrutinized by a cross-disciplinary expert panel, whose job was to sharpen wording, eliminate ambiguity, and ensure that each item captured a distinct facet of the construct without overlapping its neighbors.

That four-dimensional architecture is the conceptual heart of the instrument. The first dimension, cognitive bias, covers the systematic errors in thinking that distort how students interpret online information—jumping to conclusions, accepting claims without evidence, or framing disagreement as hostility. The second, emotional sensitivity, captures how strongly online interactions trigger affective reactions, from taking criticism personally to being swept up in collective outrage. The third, self-regulation deficiency, measures failures of impulse control: the inability to pause before posting, to disengage from escalating arguments, or to moderate one’s own emotional state. The fourth, behavioral deviance, describes the observable endpoint—posting behavior that departs from reasoned norms, such as aggressive attacks, rumor spreading, or deliberate provocation. Together, the dimensions trace an arc from distorted thinking through emotional reactivity and failed self-control to problematic action.

With the item pool refined, the quantitative phase put the draft scale through the standard gauntlet of psychometric testing on two independent samples. The first, comprising 500 undergraduate students, was used for item analysis and exploratory factor analysis, the statistical procedure that reveals whether the items naturally cluster into the hypothesized dimensions or whether the underlying structure is different from what the qualitative work suggested. The second, a separate sample of 476 students, was reserved for confirmatory factor analysis, a stricter test in which the hypothesized four-factor structure is specified in advance and the data are asked whether they fit. Splitting the samples this way matters: a scale that only fits the data used to build it is a statistical artifact, not a measurement tool. The two-sample design gives the validation genuine out-of-sample credibility.

The numbers that emerged are, by conventional standards, solid. The confirmatory factor analysis returned a chi-square to degrees-of-freedom ratio of 2.094, comfortably within the accepted threshold of below 3. The comparative fit index reached 0.934 and the Tucker-Lewis index 0.928, both above the 0.90 benchmark that indicates good fit, while the standardized root mean residual of 0.057 and the root mean square error of approximation of 0.058 fell within the ranges generally interpreted as acceptable to good. Perhaps most striking is the internal consistency coefficient of 0.95, a value indicating that the 28 items of the final scale hang together with remarkable coherence. In practical terms, the statistics say that the four dimensions are real, that the items measure them reliably, and that the scale behaves consistently across the independent validation sample.

The final instrument therefore comprises 28 items spanning cognitive bias, emotional sensitivity, self-regulation deficiency, and behavioral deviance. But the researchers did not stop at validation; they applied the scale to see whether it detects meaningful differences between groups. It did. Scores varied significantly by gender, by the duration of internet use, and by family background, suggesting that irrational online behavior is not uniformly distributed across the student population. Intriguingly, no significant difference emerged across grade levels, implying that the tendency toward irrational discourse is not simply a matter of academic maturity or year of study—first-year students and near-graduates appear broadly comparable in this respect. The researchers suggest that identifying these at-risk subgroups could help universities target interventions more precisely than blanket awareness campaigns.

The implications reach beyond campus administration. The study was funded under Chinese research programs on risk prevention and guidance mechanisms for group polarization in university network public opinion, and on the interactive evolution of online and offline opinion dynamics—signaling that the scale is intended as a tool for understanding how irrational discourse can escalate into polarized collective phenomena. Group polarization, in which like-minded individuals push each other toward increasingly extreme positions, is one of the most studied dangers of online communication, and a validated individual-level measure of the psychological precursors could give researchers a way to study that escalation at its source. If cognitive bias and self-regulation deficits can be quantified before they harden into deviant posting behavior, interventions might be designed to strengthen exactly those weak points.

The work also carries the usual caveats that accompany any new psychological instrument. The scale was developed and validated with Chinese undergraduate samples, and cross-cultural generalization remains to be tested; constructs like emotional sensitivity and behavioral deviance in online discourse may manifest differently in other linguistic and platform environments. Self-report measures, however carefully constructed, depend on respondents’ willingness to admit to irrational tendencies, and the authors themselves describe the psychometric support as preliminary. Still, the methodological rigor of the two-sample validation, the expert refinement process, and the strong fit statistics give the instrument a credible foundation. The study was approved by the Ethics Committee of Hunan First Normal University and conducted with informed consent under the principles of the Declaration of Helsinki.

What makes the research resonate beyond the academy is how precisely it captures a phenomenon that anyone who has spent time online will recognize: the moment when a discussion stops being an exchange of reasons and becomes a contest of reactions. By decomposing that moment into measurable components—biased cognition, emotional triggering, failed restraint, and deviant expression—the scale turns a vague cultural anxiety about online irrationality into something that can be counted, compared, and tracked over time. Whether it ultimately helps universities defuse volatile opinion storms or simply gives psychologists a sharper lens on digital behavior, the instrument marks a step toward treating online discourse not as an unknowable torrent but as a measurable expression of human psychology, one 28-item questionnaire at a time.

Subject of Research: Development and psychometric validation of a scale measuring irrational behavior in undergraduate students' online discourse

Article Title: Development and validation of a scale for measuring irrational behavior in undergraduate students’ online discourse

Article References: Liu, Z., Zhang, C., Li, J., & Yan, B. (2026). Development and validation of a scale for measuring irrational behavior in undergraduate students’ online discourse. BMC Psychology. https://doi.org/10.1186/s40359-026-05678-w

Image Credits: AI Generated

DOI: 10.1186/s40359-026-05678-w

Keywords: online discourse, irrational behavior, undergraduate students, scale development, psychometrics, confirmatory factor analysis, exploratory factor analysis, cognitive bias, self-regulation, group polarization, BMC Psychology, social behavior