AI Watches Surgeons Train: Computer Vision Passes FLS Peg Transfer Test

An artificial intelligence system trained to watch surgeons perform one of the most familiar exercises in surgical education can grade their performance almost as reliably as expert human raters, according to a new study from researchers at UCLA David Geffen School of Medicine and Cedars-Sinai Medical Center. The work, published in Global Surgical Education, the Journal of the Association for Surgical Education, tackles a persistent bottleneck in how surgeons learn: the Fundamentals of Laparoscopic Surgery (FLS) program, a prerequisite for the American Board of Surgery Qualifying Exam, depends heavily on faculty members personally observing trainees as they practice. That dependence limits how often residents can receive feedback, and it introduces subjectivity into a process that ultimately helps decide who becomes a certified surgeon. The UCLA-led team set out to determine whether a computer vision pipeline could shoulder part of that burden, autonomously classifying performance on the FLS peg transfer task into beginner, intermediate, or expert skill levels.

The peg transfer task is deceptively simple to describe. Using a pair of laparoscopic graspers, a trainee must pick up small objects, transfer them between the left and right instruments, and place them on a pegboard, all while viewing the field only through a camera that removes natural depth perception. Speed and precision both matter, and expert raters have long used the task to distinguish novices from seasoned minimally invasive surgeons. Previous research has established that simulator performance on FLS tasks predicts intraoperative laparoscopic skill, which is precisely why the program carries such weight in certification. But the same research literature has also documented how resource-intensive human assessment can be, motivating a decades-long search for automated, objective measures of surgical motion, from early motion-analysis studies to modern deep learning approaches that interpret surgical video directly.

To build their dataset, the researchers recorded general surgery residents and medical students at an academic medical center performing the peg transfer task on camera. In total, 132 videos were captured. Two adjudicators then independently scored each recording as beginner, intermediate, or expert, basing their judgments on task duration and the perceived technical quality of the performance. This human labeling step is crucial: in supervised machine learning, the algorithm can only be as good as the ground truth it learns from, and the use of two independent adjudicators helps ensure the labels reflect genuine consensus about skill level rather than one rater’s idiosyncrasies. Of the 132 recordings, 100 were ultimately used for analysis, while 32 were excluded because of tracking dropout, a reminder that even mature computer vision systems can lose track of instruments when visibility degrades or movements become ambiguous.

Technically, the pipeline rests on several components that have become standard tools in modern computer vision. The team used Ultralytics YOLOv8 for object detection, identifying and localizing the laparoscopic instruments frame by frame, and paired it with ByteTrack, a multi-object tracking algorithm that associates detection boxes across consecutive frames to maintain consistent identities for each instrument through time. From these tracked trajectories, the system extracted three quantitative input features: task duration, instrument path length, and peg displacement. These features distill an entire performance into the quantities that surgical educators have long recognized as meaningful. Shorter completion times and shorter instrument travel distances have historically correlated with higher technical skill, while peg displacement captures the physical consequences of the surgeon’s actions on the training board itself. A gradient boosting classifier, LightGBM, then mapped these features to skill level labels.

The model was trained under five-fold cross-validation, a rigorous scheme in which the data are split into five subsets and the model is repeatedly trained on four while being tested on the fifth, so that every recording serves as an unseen test case. The results were strong. Overall classification accuracy reached 84 percent, and, notably, the model made almost no beginner-to-expert misclassifications, meaning it essentially never confused the least skilled performers with the most skilled ones. The macro-averaged area under the curve, a measure of the model’s ability to discriminate across classes, came in at 0.907 plus or minus 0.061. Broken down by class, the AUC was 0.971 for beginners, 0.850 for intermediates, and 0.900 for experts, indicating that the extremes of the skill spectrum were the easiest to identify while the middle category, as is often the case in ordinal classification problems, posed the greatest challenge.

A threshold-based variant of the classifier, which converts the model’s probability outputs into hard class assignments, achieved a macro F1-score of 0.83 and, strikingly, a precision of 0.96 when identifying experts. In practical terms, when the system declares a performance expert-level, it is right nearly every time. That property matters for real-world deployment in surgical education. A tool that occasionally under-recognizes an intermediate performer but almost never inflates a novice to expert status is far safer for high-stakes feedback than one with symmetric error rates. The authors suggest that their AI-enabled computer vision model may offer a scalable supplement to traditional expert-based evaluation, potentially increasing opportunities for trainee feedback without adding to faculty workload.

The significance of that scalability claim becomes clear when considering the structure of American surgical training. Because successful FLS completion is a prerequisite for board qualification, residents across the country must prepare for and pass the exam, and studies have shown that doing so improves operative performance and autonomy while boosting junior residents’ self-efficacy. Yet feedback during preparation typically requires a faculty surgeon to watch practice runs, an expensive use of attending physician time that caps the frequency of assessment. A vision-based system that can watch unlimited repetitions and deliver consistent, quantified scores could allow residents to practice more deliberately, tracking their trajectory from beginner toward expert using objective metrics rather than intermittent impressions. The authors argue that implementing such AI-enabled assessment systems may enhance the accessibility of feedback and promote technical skill development across training programs, not just at well-resourced academic centers.

The study also situates itself within a rapidly expanding body of work on artificial intelligence in surgery. A 2024 review in Nature Medicine charted the broad scope of AI applications across the surgical lifecycle, and recent projects have applied deep learning to simulated laparoscopic skill assessment, video-based formative and summative evaluation of surgical tasks, and competency gauging on novel laparoscopic training systems. Earlier efforts demonstrated that 3D convolutional neural networks could assess skill directly from raw video, while other teams built software-based motion tracking tools for the surgical skills assessment landscape. What distinguishes the new UCLA approach is its deliberate parsimony: rather than feeding entire video streams into a heavyweight neural network, the researchers reduced each performance to three interpretable features derived from instrument tracking. This design choice makes the model’s decisions easier to audit and explain to educators, an important consideration as AI tools move closer to credentialing processes.

The choice of features also connects the work to foundational research in surgical education. Landmark studies by Datta and colleagues at Imperial College showed decades ago that motion analysis metrics correlate strongly with expert technical assessments, and randomized trials demonstrated that FLS simulator training to proficiency translates into improved laparoscopic performance in the operating room. By automating the extraction of duration, path length, and displacement, the new system operationalizes those validated constructs at scale, converting what once required laboratory motion-tracking equipment into something achievable with a standard camera and open-source detection and tracking software. The exclusions for tracking dropout, however, highlight remaining engineering challenges: lighting, camera angle, occlusion, and instrument visibility all still influence whether the pipeline can reliably follow tools through a complete performance.

The research team, led by corresponding author Terrance Peng of UCLA and including collaborators from Cedars-Sinai Medical Center, reports no conflicts of interest related to the project. The authors caution that their model was developed and validated on recordings from a single academic medical center, and broader validation across institutions, camera setups, and trainee populations will be needed before such systems can assume a formal role in summative assessment. Privacy considerations also shape the field: the video recordings generated for the study are not publicly available, with additional data available only on reasonable request. Still, the trajectory of the results, 84 percent accuracy, near-zero confusion between skill extremes, and 96 percent expert precision, suggests that AI-assisted surgical assessment is moving from proof-of-concept toward practical tool. If future systems match this performance in everyday training environments, the hours faculty spend watching peg transfers could be redirected toward higher-value teaching, while residents gain the luxury of feedback after every single repetition, a shift that could quietly reshape how the next generation of surgeons learns to operate.

Subject of Research: Artificial intelligence-based computer vision assessment of laparoscopic surgical skill on the FLS peg transfer task

Article Title: Artificial intelligence-enabled evaluation of laparoscopic peg transfer performance

Article References: Peng, T., Alipour, A., Desai, K., Chen, D., Huang, G., Rosenthal, R. J., Barmparas, G., Chen, Y., & Benharash, P. (2026). Artificial intelligence-enabled evaluation of laparoscopic peg transfer performance. Global Surgical Education – Journal of the Association for Surgical Education, 5(1), Article 173. https://doi.org/10.1007/s44186-026-00580-w

Image Credits: AI Generated

DOI: 10.1007/s44186-026-00580-w

Keywords: artificial intelligence, computer vision, laparoscopic surgery, surgical education, FLS, peg transfer, skills assessment, machine learning, surgical training, object detection, YOLOv8, residency