The economic stakes are considerable. Productivity losses attributable to porcine reproductive and respiratory syndrome virus alone have been estimated to exceed USD 1 billion annually in the United States, with substantial losses also reported on Chinese pig farms. Yet diagnosis of respiratory disease in commercial herds still relies largely on farm inspections, slaughter checks, serology and post-mortem examination of dead animals. These approaches are labour-intensive, expensive, slow, and often applied only after disease has progressed or an animal has died. Contact-based monitoring devices such as ear tags and respiratory belts add cost and stress, limiting large-scale deployment. Coughing, by contrast, is one of the most prevalent and easily observable clinical signs of porcine respiratory disease, making it an attractive target for non-invasive, continuous monitoring.

Earlier cough-sound studies focused mainly on detecting that a cough occurred somewhere in a pig house, without identifying which pen or animal produced it. Sound source localisation techniques mapped coughs to pen-level risk areas but could not distinguish individuals within the same pen. A later audio-visual approach used motion history images to link cough sounds to transient body movements, but relied on handcrafted motion features and threshold-based rules whose robustness in busy group-housed settings remained uncertain. The new work builds on a deep learning multimodal method for coughing pig identification and extends it in three directions: a task-specific architecture for hierarchical audiovisual temporal modelling, a design that decouples the recurrent temporal module from the classification layer, and a dataset featuring low-light nighttime scenes for evaluating performance under both day and night conditions.

The experimental foundation was a 42-day challenge study conducted from August to October 2024 within a Biosafety Level 2 laboratory at the Jiangsu Academy of Agricultural Sciences, approved by the institutional animal care committee. Ten 60-day-old crossbred finishing pigs were split equally between an infected group and a control group, housed in separate but identical pens under controlled conditions of 23 to 28 degrees Celsius and 40 to 70 percent relative humidity, with antibiotic-free feed. Pigs in the experimental group received an intratracheal dose of virulent Mhp suspended in phosphate-buffered saline; controls received saline only. Infection was confirmed serologically: serum Mhp-IgG sample-to-negative ratios in the infected group fell below the 50 percent ELISA threshold on days 28 and 42, and nasal swab Mhp-sIgA sample-to-positive ratios rose above the 0.2 cutoff on day 28, while control animals stayed within negative ranges throughout. Post-mortem examination at day 42 revealed characteristic shrimp-like lung lesions in the infected group and none in controls, and testing ruled out other common swine pathogens.

A high-definition camera mounted roughly two metres above each pen recorded the pigs’ dorsal regions continuously at 25 frames per second. From these recordings, 300 clips showing coughing events were manually segmented, 200 from daytime and 100 from nighttime footage. The pipeline consists of three modules. First, a coughing event detection module based on a sound event detection convolutional recurrent neural network processes log-Mel spectrograms of two-channel audio to locate the onset and offset of coughs and extract acoustic features. Second, a pig object detection module applies YOLO26-OBB, chosen for its oriented bounding boxes that fit pigs’ varied postures and reduce overlap between adjacent animals, to the single frame with the highest cough probability, then crops each pig’s region across the surrounding frames of the event window. Third, a coughing pig identification module fuses the audio features with each candidate pig’s video sequence and decides which animal is actually coughing.

The identification module is where the architectural novelty lies. Cropped pig sequences are converted to grayscale and resized to 112 by 112 pixels before a visual encoder, comprising 3D convolutional layers, a ResNet18 backbone and a video temporal convolutional network, extracts frame-level visual embeddings. Audio and visual features are then combined through channel attention, which concatenates the modalities and generates time-varying channel weights via a one-dimensional convolution, adaptively emphasising informative dimensions while suppressing redundancy. A bidirectional gated recurrent unit (BiGRU) models short-term temporal dynamics in both directions, and an eight-head self-attention mechanism then reweights the temporal features globally to capture dependencies across the whole event. Because the self-attention module receives BiGRU outputs that already encode temporal order, no positional encoding was added. Training used a weighted cross-entropy loss that upweights positive coughing samples, with a positive weight of 2.8 chosen in preliminary experiments, reflecting the fact that in health monitoring a missed diseased pig is costlier than a false alarm that can be checked manually.

The results were strong across the board. In four-fold cross-validation, the coughing event detection module achieved an average frame-level F1 score of 0.88 with an error rate of 0.28, and event-level averages of 0.68 F1, 0.62 precision and 0.75 recall; the best fold, retained for building the audiovisual dataset, reached a frame-level F1 of 0.95. The object detection module attained a mean average precision (mAP) of 0.94 at IoU thresholds of 0.50 to 0.95, detecting all pigs reliably in both daytime and nighttime scenes. Across five independent training runs, the full identification system achieved an average mAP of 0.96, precision of 0.86, recall of 0.92, F1 of 0.89, error rate of 0.23 and video-level accuracy of 0.80, meaning that in most videos the coughing or non-coughing status of every pig was correctly determined.

Notably, performance held up in the dark. Under nighttime low-light conditions the model recorded an mAP of 0.96, precision of 0.92, F1 of 0.90, error rate of 0.19 and video-level accuracy of 0.82, with recall of 0.89. In other words, the model produced fewer false detections at night while slightly missing more coughing instances, with overall performance essentially unchanged. The authors note that the nighttime evaluation involved fewer videos, 24 versus 56 during daytime, which made the metrics somewhat more variable, though the fluctuations remained small. This matters practically because environmental noise and animal activity are typically lower at night and in the early morning, potentially offering a more stable window for health assessment.

Ablation experiments confirmed that each component contributes. Removing the self-attention mechanism caused the error rate to balloon from 0.23 to 0.71, with precision collapsing to 0.64 while recall stayed high, showing that self-attention mainly suppresses false positives arising from non-cough movements and slight audio-visual misalignment. Removing the BiGRU degraded mAP, recall, F1 and video-level accuracy while increasing metric variability, underscoring the value of bidirectional local temporal modelling for coughs of variable duration and ambiguous boundaries. Comparisons against existing methods were also favourable: the proposed model outperformed the earlier coughing pig detection model, LR-ASD and TalkNet-ASD on mAP, recall, F1, error rate and video-level accuracy, while a rule-based motion-history method performed poorly, with precision of 0.17 and F1 of 0.22. Paired t-tests across five runs showed statistically significant advantages in several metrics, including F1, error rate and video-level accuracy against the earlier coughing pig detection model, and recall, F1, error rate and video-level accuracy against LR-ASD, though LR-ASD retained significantly higher precision. Inference takes about 0.0121 seconds per pig sequence on an RTX A6000 GPU, comparable to competing audiovisual models, though the full three-module pipeline needed roughly 14.85 seconds for an 8.68-second video of five pigs.

The authors are candid about the limitations. All data came from two controlled pens in a single BSL-2 laboratory with only ten pigs, so generalisation to commercial farms, different stocking densities and pen configurations remains untested. The random event-level data split, stratified only by day and night, means segments from the same pig or recording day could appear in both training and validation sets, potentially inflating estimates of generalisation; future work will adopt animal-level or temporally blocked splits. Cough labels were annotated by a single researcher without inter-annotator reliability checks, and the identification experiments ran on manually confirmed cough-event segments rather than unfiltered continuous recordings, so the reported metrics do not represent end-to-end performance. Overlapping coughs from multiple pigs, fixed bounding boxes without tracking, and latency dominated by the event detection module all remain open challenges. Even so, the study demonstrates that learned audiovisual temporal modelling can attribute a cough to a specific animal within a crowded pen, day or night, laying groundwork for continuous, non-invasive respiratory disease surveillance in intensive pig production.

Subject of Research: Audiovisual multimodal deep learning for identifying coughing pigs in group-housed environments

Article Title: Coughing pig identification method based on multimodal fusion and hierarchical temporal modelling in group-housed environments

Article References: He, J., Zhuang, Y., Gan, Y., Qiu, F., yang, G., Liu, K., Wu, Y., Zhao, Y., Li, H., Li, J., Li, Q., & Yu, L. (2026). Coughing pig identification method based on multimodal fusion and hierarchical temporal modelling in group-housed environments. Smart Agricultural Technology, 15, Article 102658. https://doi.org/10.1016/j.atech.2026.102658

Image Credits: AI Generated

DOI: 10.1016/j.atech.2026.102658

Keywords: precision livestock farming, swine respiratory disease, Mycoplasma hyopneumoniae, audiovisual fusion, deep learning, cough detection, active speaker detection, animal welfare, nighttime monitoring, BiGRU, self-attention, object detection