Disaster Training May Leave Responders Unprepared, Trainers Warn

When a major earthquake strikes or a flood overwhelms a region, the quality of the first response often depends on training that took place months or years earlier, in classrooms and simulation centers far from the disaster zone. Yet a new qualitative study suggests that the way disaster management training is monitored and evaluated may be leaving responders with a dangerously false sense of preparedness. The research, published in BMC Medical Education, offers a rare bottom-up look at how the people who actually organize and deliver disaster management courses judge their own assessment practices, and the picture that emerges is one of enthusiasm undermined by fragmentation, superficial measurement, and a lack of dedicated tools.

The study, led by George Teo Voicescu of CRIMEDIM, the Center for Research and Training in Disaster Medicine, Humanitarian Aid and Global Health at the Università del Piemonte Orientale in Novara, Italy, together with colleagues including Hamdi Lamine, Luca Ragazzoni and Marta Caviglia, set out to explore current monitoring and evaluation practices among disaster management training organizers. Monitoring and evaluation, often abbreviated as M&E, are two related but distinct activities. Monitoring refers to the ongoing, real-time tracking of a training course while it is running, allowing trainers to adjust content, pacing and methods as problems emerge. Evaluation, by contrast, typically happens after the fact and asks whether the training achieved its intended outcomes. In an ideal world, the two work together in a continuous feedback loop that steadily improves both the course and the competence of its graduates.

To understand how this loop functions in practice, the researchers conducted a prospective qualitative study using semi-structured online interviews. They deliberately sampled training organizers across diverse regions and operational contexts, contacting 53 individuals and ultimately enrolling 15 participants. The interviews were audio recorded, transcribed with pseudonymization, and subjected to thematic analysis, a qualitative method in which researchers systematically code transcripts to identify recurring patterns and themes. The team reported the work in accordance with the Consolidated Criteria for Reporting Qualitative Research, known as COREQ, a checklist designed to improve the transparency and rigor of interview-based studies. Ethical approval was obtained from the ethical review board of the Iuliu Hatieganu University of Medicine and Pharmacy in Cluj-Napoca, Romania, and all participants provided informed consent.

The first striking finding was heterogeneity. The training development process, the formats and the content of disaster management courses varied enormously across the interviewees. Some organizers followed established instructional design frameworks such as ADDIE, which stands for Analysis, Design, Development, Implementation and Evaluation, or evaluation models like CIPP, which examines context, input, process and product. Others relied on ad hoc approaches shaped by institutional habit or funder expectations. Courses ranged from short intensive workshops to extended programs covering topics as varied as major incident medical management, exemplified by frameworks like MIMMS, the Major Incident Medical Management and Support course, and broader disaster planning. This diversity, the authors note, is not inherently a weakness, but it makes standardized assessment extremely difficult, because a measurement tool designed for one type of course may be meaningless for another.

The second major finding concerned the imbalance between monitoring and evaluation. Both activities were valued by the trainers interviewed, but evaluation was frequently prioritized over monitoring. The dominant evaluation method was the pre-post survey, in which trainees rate their knowledge or confidence before and after a course. While such surveys are easy to administer, the study suggests they often capture little more than satisfaction or self-perceived learning, rather than genuine operational readiness. Monitoring, meanwhile, was often informal or absent altogether. The researchers warn that this lack of monitoring might drastically affect short courses in particular, because without real-time feedback trainers have no opportunity to improve the course while it is still ongoing, potentially leaving participants poorly trained by the time the course ends.

Underlying much of this superficiality, the interviewees suggested, was the influence of funders. Evaluation was often guided by funder requirements rather than by a genuine desire to improve training quality. When the primary audience for evaluation data is a donor or an administrative office demanding completion reports, the resulting instruments tend to measure accountability rather than learning. The study also points to a structural problem in the workforce itself: most disaster management trainers are operational professionals, not educational experts. They bring invaluable field experience to the classroom, but they may lack formal training in pedagogy, assessment design or measurement theory. Relying on such trainers to invent their own unstandardized monitoring and evaluation approaches, the authors argue, can be suboptimal, whereas dedicated M&E teams could professionalize the field in the same way that specialized quality assurance functions have improved other high-stakes industries.

The consequences of weak assessment extend beyond the classroom. After action reports from real disasters consistently identify disaster management training as fundamental to effective response, which means that flawed evaluation does not merely produce misleading course certificates; it obscures the true state of operational readiness across the emergency response system. A trainee who completes a course with high satisfaction scores but limited actual capability may be deployed into a crisis believing they are prepared. The study explicitly warns that current practices give limited insight into operational readiness and can create a false sense of preparedness among trainees. In a field where lives depend on performance under extreme pressure, that gap between perceived and actual competence is not a bureaucratic inconvenience but a safety issue.

What do the trainers themselves want? The interviewees recommended a flexible, user-friendly monitoring and evaluation platform dedicated specifically to disaster management, one that would support long-term tracking and follow-up of trainees over time and include networking capabilities connecting trainers, organizers and alumni. Long-term follow-up is particularly important because the true test of disaster training is not performance on a final quiz but the retention and application of skills months or years later, when a disaster actually occurs. A dedicated platform could also help standardize core metrics while remaining adaptable to the heterogeneity of courses, and the authors suggest that mixed methods approaches, combining quantitative indicators such as key performance indicators with qualitative insight into the learning process, could provide a more comprehensive understanding of how disaster responders actually learn.

The researchers are careful to acknowledge the challenges inherent in any such solution. Disaster management courses are short, their topics are diverse, and their training methodologies vary widely, so any monitoring and evaluation system must be adapted to these constraints rather than imported wholesale from other educational domains. There are also risks associated with centralized platforms, including the administrative burden they may impose on trainers and the possibility that standardized metrics could flatten the very diversity that makes disaster training responsive to local contexts. The authors argue that DM-adapted tools could more accurately assess trainees’ readiness to work and that a well-designed platform could fill existing gaps while mitigating the risks associated with it, a balance that will require careful design and consultation with the training community itself.

The study emerges from the International PhD in Global Health, Humanitarian Aid and Disaster Medicine organized by the Università del Piemonte Orientale, and the authors received no external funding for the work, declaring no competing interests. As climate change intensifies hazards and urbanization concentrates populations in vulnerable regions, the demand for disaster management training will only grow, and with it the stakes of getting assessment right. This study does not claim that disaster training is failing; rather, it reveals that the field currently cannot prove it is succeeding. By giving voice to the trainers who deliver these critical courses, the research lays the groundwork for a more professionalized, evidence-based approach to building the competence that disaster response ultimately depends on, one monitored session and one meaningful evaluation at a time.

Subject of Research: Monitoring and evaluation practices in disaster management training

Article Title: Monitoring and evaluating disaster management training: a qualitative study of trainers’ perspectives

Article References: Voicescu, G. T., Lamine, H., Ragazzoni, L., & Caviglia, M. (2026). Monitoring and evaluating disaster management training: a qualitative study of trainers’ perspectives. BMC Medical Education. https://doi.org/10.1186/s12909-026-10465-7

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10465-7

Keywords: disaster management, monitoring and evaluation, medical education, qualitative research, training assessment, emergency preparedness, disaster medicine, pre-post surveys, instructional design, capacity building, thematic analysis, humanitarian aid