Scientists Teach Computers to Watch Rock Avalanches Move, Frame by Frame

When a mountainside collapses, the transformation is terrifyingly fast. Millions of tons of rock detach from a slope, shatter, and race downhill at speeds that can exceed dozens of meters per second, burying villages and reshaping valleys in minutes. For decades, scientists studying these catastrophic rock avalanches have had to reconstruct their dynamics largely after the fact, relying on the geometry of deposits, eyewitness accounts, and the faint vibrations the events leave in the ground. Now, a research team in China has proposed a way to actually watch an avalanche move, using computer vision to extract frame-by-frame motion from video footage and seismic signals to keep the analysis honest.

The new study, published in the journal Natural Hazards by Yiwei Liu, Aiguo Xing, and colleagues at Shanghai Jiao Tong University, the China Institute of Geo-Environment Monitoring, and Hunan University, presents a joint analytical framework that fuses two very different data streams: ordinary video recordings and seismic observations. The work is a case-oriented, exploratory attempt to integrate visual and seismic information for rock avalanche analysis, and it demonstrates its approach on a representative large rock avalanche event in Southwest China. The goal is deceptively simple to state but technically demanding to achieve: to obtain velocity distributions and motion descriptions at any moment during an avalanche, something neither data source can deliver alone.

The core of the visual analysis is a technique called optical flow, a cornerstone of computer vision with a history stretching back to foundational work in the early 1980s. Optical flow algorithms estimate the apparent motion of pixels or features between successive frames of a video, producing a dense field of motion vectors that describes how every part of the image is shifting. In the context of a rock avalanche, those vectors translate directly into kinematic information: which parts of the flowing mass are moving fastest, where the front of the avalanche is advancing, and how the body of the flow stretches, compresses, and deforms as it descends. The researchers implemented their pipeline in Python, building on widely used open-source libraries including OpenCV, NumPy, and pandas, and they have released the code and data openly through GitHub and the Zenodo repository.

Video alone, however, has a fundamental weakness. A camera records only what is in its field of view, and the motion it captures is projected onto a two-dimensional image plane. Without independent confirmation, it can be difficult to know precisely when events in the video occurred in absolute time, or whether apparent motion reflects real ground movement or artifacts such as camera shake, changing illumination, or dust obscuring the scene. This is where the seismic half of the framework becomes essential. Large rock avalanches generate ground vibrations that are recorded by seismic instruments at considerable distances, and seismologists have developed increasingly sophisticated methods to invert those signals for landslide characteristics such as location, timing, momentum, and frictional behavior.

In the joint framework, seismic signal analysis serves as an independent reference for temporal alignment and validation. By matching prominent features in the seismic record, such as the abrupt onset of high-frequency energy when the mass impacts the valley floor, with corresponding moments in the video, the researchers can anchor the visual timeline to a precise, instrumentally measured clock. The seismic data then act as a cross-check: if the optical-flow analysis suggests a surge in motion at a particular moment, the seismic record should show a corresponding change in energy release. Agreement between the two independent streams increases confidence in the reconstructed dynamics; disagreement flags moments that deserve closer scrutiny.

The framework is organized into a four-stage architecture that integrates data acquisition, preprocessing, joint analysis, and result visualization. In the acquisition stage, video footage is gathered from monitoring systems and public sources, alongside seismic records covering the event window. Preprocessing addresses the practical realities of real-world imagery: stabilizing frames, correcting for lighting variation, and preparing the seismic traces through filtering and decomposition techniques. The joint analysis stage runs the optical-flow motion extraction and the seismic interpretation in parallel, aligning them in time. Finally, the visualization stage renders the results as velocity distributions and motion descriptions that can be examined at any moment during the avalanche, turning raw footage and seismograms into quantitative, interpretable pictures of the disaster in progress.

Applied to a large rock avalanche in Southwest China, the framework demonstrated its practicality. The results showed that the combined approach provides complementary insights to those obtained from seismic signal analysis alone. Seismic inversion can estimate bulk properties of a landslide, but it inherently averages the behavior of the entire moving mass. Video-based optical flow, by contrast, resolves spatial patterns: it can reveal that the front of the flow accelerates while the trailing mass decelerates, or that deformation concentrates along particular zones within the avalanche body. Merging the two perspectives yields a richer and more defensible reconstruction than either could offer independently.

The significance of this capability extends well beyond academic curiosity. Rock avalanches routinely cause severe casualties and economic losses, and improving the characterization of event-scale dynamic behavior and precursory deformation is considered essential for effective disaster mitigation and early warning. Landslides are among the deadliest geological hazards worldwide, and China, with its steep terrain, active tectonics, and dense population in mountainous regions, has experienced repeated catastrophic events, including the long-runout Pusa rock avalanche of 2017 in Guizhou Province and the massive Chamoli rock and ice avalanche in the Indian Himalaya in 2021, which killed hundreds. Understanding exactly how these masses accelerate, fragment, and spread is critical for designing warning systems, mapping hazard zones, and validating the numerical models engineers use to predict runout.

What makes the new framework particularly timely is the explosion of available video. Monitoring cameras now watch many hazardous slopes continuously, and when disasters strike, surveillance cameras, smartphones, and public platforms often capture the event from multiple angles. Until recently, this footage was mostly used qualitatively, as dramatic evidence of what happened. The authors argue that video-based analysis has become a valuable complementary data source for investigating landslide dynamics, and their framework provides a systematic, reproducible way to convert pixels into physics. Optical-flow methods have already proven useful in related domains, from measuring glacier surface velocities in satellite imagery to tracking slow slope creep in high-alpine settings, and this study extends that lineage to the most violent end of the landslide spectrum.

The researchers are careful to frame the work as a case-oriented and exploratory attempt rather than a finished operational system. Challenges remain, including the sensitivity of optical flow to dust, poor lighting, and low frame rates, and the difficulty of converting image-plane motion into true three-dimensional ground velocities without accurate camera calibration and terrain models. Nevertheless, the study highlights the potential of video-based computer vision techniques to enhance the interpretation of landslide dynamics and to support future monitoring and early-warning research. If the approach matures, the aftermath of the next catastrophic slope failure may be documented not just by trembling seismometers and stunned witnesses, but by algorithms that watched every frame, measured every surge, and helped explain one of nature’s most violent phenomena in unprecedented detail.

Subject of Research: Integration of computer vision and seismic signal analysis to characterize the dynamics of large rock avalanches

Article Title: Characterizing dynamic motion and deformation during large rock avalanches using a joint framework integrating computer vision and seismic signal analysis

Article References: Liu, Y., Xing, A., Wang, W., Wang, Q., Zhuang, Y., Bilal, M., & Zhu, K. (2026). Characterizing dynamic motion and deformation during large rock avalanches using a joint framework integrating computer vision and seismic signal analysis. Natural Hazards, 122(18), Article 628. https://doi.org/10.1007/s11069-026-08349-6

Image Credits: AI Generated

DOI: 10.1007/s11069-026-08349-6

Keywords: rock avalanche, optical flow, computer vision, seismic signal analysis, landslide dynamics, deformation characterization, early warning, disaster mitigation, video-based motion analysis, Natural Hazards, China, temporal alignment