Machine Learning Tracks Farm Injuries as Data Drifts Over Time

Agriculture, forestry, and fishing carry the highest fatality rate of any US occupational sector, yet the nonfatal injuries that scar this workforce remain poorly counted. A new study in BMC Medical Research Methodology tackles a stubborn piece of that surveillance problem: how to reliably flag agricultural injuries buried in the free text of pre-hospital care reports, and how to keep the automated system accurate as the data it reads slowly changes shape over the years.

Emergency medical responders write thousands of patient care records in everyday clinical language, not standardized codes. Machine learning classifiers can scan these narratives and pick out likely agricultural injury cases, but surveillance systems are not one-off experiments. They run for years, and the statistical character of the records they process shifts with time, a phenomenon the researchers call temporal drift. A model that performs brilliantly in its first deployment year can quietly lose accuracy in its third.

The team, led by Laura E. Jones and colleagues, built its benchmark on a manually coded dataset of 60,010 pre-hospital care records from Maine spanning 2008 to 2022. They compared four classifiers: Naive Bayes, penalized logistic regression using an elastic net, a support vector machine, and gradient-boosted trees known as XGBoost. Crucially, they used a temporal design rather than the random splits common in machine learning papers, training on records from 2008 to 2016 and testing on later years, which mirrors how a real surveillance system would actually operate.

On stable-period data, the three stronger classifiers were nearly indistinguishable, each reaching an area under the receiver operating curve of roughly 0.94. Naive Bayes matched the others on discrimination but required a higher false-positive rate to capture 90 percent of true cases, meaning more human review for every case found. The researchers summarized performance not only with AUC and 95 percent confidence intervals but also with a practical metric they call FPR*, the false-positive rate needed to reach that 90 percent capture target, a number that speaks directly to screening workload.

The drift story emerged when models trained on 2008 to 2016 were tested on each subsequent year. Performance held through 2019, then declined across 2020 to 2022 as feature distributions shifted and the labeled case fraction rose. Notably, the researchers attribute a substantial part of that rise not to more farm injuries but to how records were selected for labeling, a reminder that surveillance pipelines carry their own procedural fingerprints in the data.

The team quantified drift using the population stability index and then tested three remedies. Re-estimating the decision threshold on the most recent labeled year failed to hold the 90 percent capture target: in some years the carried-over cutoff over-captured, inflating review burden, while in others it missed cases outright as the score distribution slid relative to the fixed threshold. Retraining on all data to date and fine-tuning the existing model on recent data, a transfer learning approach, each delivered only modest recovery, which the authors interpret as evidence of genuine drift on sparse recent labeled data rather than simple miscalibration.

Year to year, the three leading classifiers remained statistically indistinguishable, so the choice of model mattered less than the maintenance strategy. The study’s practical conclusion is that pairing a fast-to-retrain classifier with continuous drift monitoring and threshold re-estimation on freshly coded data is more robust than freezing a model and cutoff at deployment. The authors propose a concrete protocol for detecting and managing drift in surveillance machine learning, aimed at agencies that cannot afford silent degradation in case capture.

The implications reach beyond agriculture. Any public health surveillance system that mines free-text clinical records over multiple years faces the same structural challenge: the world that generates the text changes, the labeling process changes, and the model’s decision boundary does not move on its own. This study offers a rare longitudinal, temporally honest benchmark of how quickly that erosion happens and how much of it standard fixes can recover.

Limitations deserve emphasis. The analysis rests on a single state’s records over one fifteen-year window, and the modest gains from retraining suggest that the amount of recently labeled data available in real surveillance settings constrains adaptation. The work is a methods benchmark, not a deployed system evaluation, and its findings describe classifier performance on coded records rather than the true incidence of agricultural injury itself.

For occupational safety researchers, the takeaway is sobering but actionable. Automated classification can indeed surface agricultural injuries hidden in emergency care narratives, and it can do so with strong discrimination, but sustaining that performance demands ongoing vigilance: monitoring feature stability, watching the labeled case fraction, and accepting that thresholds and models need periodic renewal on newly coded data rather than a one-time calibration.

Subject of Research: Benchmarking machine learning classifiers for detecting agricultural injuries in free-text pre-hospital care records under temporal drift.

Article Title: Improved classification of agricultural injuries from free-text analysis of pre-hospital care reports under temporal drift

Article References: Jones, L. E., Scott, E., Krupa, N., Hansen-Ruiz, C. S., Kern, M. E., & Jenkins, P. (2026). Improved classification of agricultural injuries from free-text analysis of pre-hospital care reports under temporal drift. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-026-03027-4

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03027-4

Keywords: agricultural injuries, machine learning, pre-hospital care reports, temporal drift, surveillance, XGBoost, support vector machine, population stability index, occupational health, classification, Improved, agricultural