Deep learning builds reports from laparoscopic cholecystectomy videos

A deep-learning system designed specifically for laparoscopic cholecystectomy achieved mean recognition performance above 95% for operative phases and instruments, according to the study. It also converted videos into structured procedural reports with phase timelines, instrument profiles, event flags, timestamped highlights and operative narration. The researchers presented the technology as a potential aid for postoperative video review, surgical education and competency assessment, but said those applications still require prospective evaluation.

The study focuses on a particular problem in minimally invasive surgery: operative procedures generate extensive video archives, but those recordings are rarely reviewed systematically and remain underused for education and assessment. Manual review, annotation and labeling are time-consuming, while feedback based on individual recall can be inconsistent or subjective. The research team sought to automate the conversion of laparoscopic cholecystectomy footage into a readable, timestamped account of the procedure.

At the center of the system is a two-stage deep-learning pipeline. The first stage uses an EfficientNet-V2-S convolutional neural network to examine individual video frames and make frame-level predictions about operative phases and instruments. The second stage uses a bidirectional long short-term memory network, or Bi-LSTM, to model temporal context and generate the final phase and instrument predictions. This architecture combines image-level recognition with sequence analysis so that the final output reflects both the content of individual frames and their ordering within the operation. The procedural information is combined with heuristic methods for flagging selected scene events.

The resulting data are processed by an on-device large language model, which turns the recognized phases, instruments and flagged events into a structured procedural report. In the system’s reported output, that report includes a timeline of operative phases, a profile of the instruments used, heuristic scene-event flags, timestamped highlights and a narrative account of the procedure. The authors describe this as a unified framework joining several artificial-intelligence technologies for automated analysis of laparoscopic cholecystectomy workflow.

Evaluation used a development cohort of 50 laparoscopic cholecystectomy videos. The frame-recognition model was trained once on frames drawn from all 50 videos before the temporal stage underwent five-fold cross-validation at the video level. The researchers then conducted qualitative external feasibility testing on three independent laparoscopic cholecystectomy videos that had not previously been seen by the system. In these tests, the pipeline generated complete structured procedural reports containing the expected phase timelines, instrument profiles, event flags, timestamped highlights and narration. The results establish feasibility for this procedure and workflow, rather than demonstrating that the reports improve surgeon training or performance assessment.

Several features of the evaluation limit the conclusions. The external test was qualitative and involved only three previously unseen videos, so it did not provide a quantitative assessment of report accuracy in an independent clinical sample. Moreover, the mean recognition performance above 95% describes the system’s phase and instrument recognition performance; it is not evidence of better educational outcomes, safer operations or more accurate competency decisions. Because the first-stage model was trained on frames from all 50 development videos before temporal cross-validation, the reported validation should not be interpreted as a fully independent train-and-test evaluation of every component.

The system is also narrowly scoped. All reported development and feasibility testing concerned laparoscopic cholecystectomy, and the authors state that applying the approach to other minimally invasive procedures would require procedure-specific training and external validation. The study therefore offers a technical demonstration of automated workflow analysis for one operation. Its proposed uses in postoperative review, surgical education and performance assessment remain potential applications that must be tested prospectively, not established clinical benefits.

Subject of Research: Automated analysis of laparoscopic cholecystectomy videos for surgical education and assessment

Article Title: Spatiotemporal deep learning for joint phase and instrument recognition in laparoscopic cholecystectomy: a tool for surgical training and assessment

Article References: Raptis, S. P., Theocharopoulos, C., Theocharopoulos, A., Levantis, G., Varvoglis, D. N., Papadakos, S. P., Ziogas, D. C., Machairas, N., Sotiropoulos, G. C., Kontis, E., & Vrahatis, A. G. (2026). Spatiotemporal deep learning for joint phase and instrument recognition in laparoscopic cholecystectomy: a tool for surgical training and assessment. Global Surgical Education – Journal of the Association for Surgical Education, 5(1), Article 201. https://doi.org/10.1007/s44186-026-00617-0

Image Credits: AI Generated

DOI: 10.1007/s44186-026-00617-0

Keywords: laparoscopic cholecystectomy, surgical video analysis, deep learning, operative phase recognition, instrument recognition, Bi-LSTM, EfficientNet-V2-S, large language models, surgical education, performance assessment, Spatiotemporal, deep