Page 58 - Read Online

Page 461 Goyal et al. Art Int Surg. 2025;5:450-64 https://dx.doi.org/10.20517/ais.2024.108

[20]
TAPP videos, achieving an accuracy of 77.1%, with an F1 score of 75.4% and precision of 51.2% . While
these results underscore the trade-off between precision and scalability, they also highlight the adaptability
of transformer models in surgical phase recognition.

The study by Zygomalas et al. (2024) utilized the YOLOv8 deep learning model to analyze laparoscopic
TAPP repair procedures, reporting an accuracy of 87.3% and an F1 score of 82% . However, unlike the
[7]
TeCNO and ResNet-50 models, phase segmentation was not explicitly addressed, underscoring ongoing
variability in study endpoints. Notably, the study by Choksi et al. (2023) implemented the ResNet-50 model
[21]
in RIHR, reporting an accuracy of 68.7% and identifying seven procedural phases . This study also
provided a critical latency period of 250 milliseconds, a key consideration for real-time applications,
especially when paired with edge computing devices like the Nvidia Jetson Xavier™.

In contrast, Zang et al. (2023) analyzed RALIHR using a CV/DL (Computer Vision/Deep Learning) model,
achieving an accuracy of 85% with an F1 score of 78.51% . This study also utilized Nvidia A100 as its edge
[9]
computing device, further highlighting the role of hardware advancements in improving model
performance. Finally, the study by Ortenzi et al. (2023) stands out for its scale, analyzing 619 TEP repairs
[10]
across multiple centers and involving 85 surgeons . By employing the Surgical Intelligence Platform
incorporating Video Transformer Networks (VTN) and LSTM networks, this study achieved a mean
accuracy of 88.8%, with 89.59% for unilateral cases and 88.45% for bilateral cases. These findings reaffirm
the significance of large-scale datasets and multi-institutional collaborations in enhancing AI model
generalizability and minimizing bias.

Despite promising results, this systematic review has several limitations. First, the included studies exhibited
considerable heterogeneity in terms of AI model architectures (e.g., CNN-based models like ResNet-50,
transformer-based models such as DETR and VTN), dataset sizes (ranging from 25 to 619 videos), surgical
approaches (TAPP, TEP, RIHR), and annotation protocols (manual vs. semi-automated tools such as
CVAT and SuperAnnotate). This heterogeneity extended to evaluation metrics, with accuracy ranging from
74% to over 87%, and F1 scores from 65% to 82%, with some studies lacking complete reporting of precision
and recall. Such variability not only precluded meta-analysis but also limited the comparability and
generalizability of model performance across surgical contexts.

Second, internal validation was lacking or inconsistently reported in several studies. The absence of rigorous
methods such as k-fold cross-validation or independent test sets raises concerns regarding overfitting and
diminishes the robustness of the reported outcomes. Third, most studies were observational and
retrospective in design, introducing selection bias and limiting the applicability of findings to real-time,
prospective surgical workflows. Fourth, surgeons’ experience, both in performing the procedures and in
annotating surgical videos, was either underreported or not described in most studies. This omission may
influence the quality of annotations and affect model training and reliability, especially for complex or skill-
dependent phases. Finally, manual annotation is labor-intensive and time-consuming, and inconsistencies
in labeling standards, phase definitions, and inter-annotator agreement can negatively impact model
reproducibility. Lastly, institutional variability in operative techniques, equipment, and surgical team
experience further limits the external validity and clinical scalability of the current AI models reviewed.

Future research should prioritize the development of standardized, publicly available, multi-institutional
datasets with consistent surgical phase definitions and annotation protocols. This would enable robust
benchmarking of AI models across institutions and procedural techniques. Establishing open repositories
for annotated surgical videos, including TAPP, TEP, and RIHR procedures, could facilitate reproducibility