Page 25 - Read Online

Ding et al. Art Int Surg 2024;4:109-38 Artificial
DOI: 10.20517/ais.2024.16
Intelligence Surgery

Review Open Access

Digital twins as a unifying framework for surgical
data science: the enabling role of geometric scene

understanding

Hao Ding , Lalithkumar Seenivasan, Benjamin D. Killeen, Sue Min Cho, Mathias Unberath
Department of Computer Science, Johns Hopkins University, Baltimore, MD 21218, USA.

Correspondence to: Prof. Mathias Unberath, Department of Computer Science, Johns Hopkins University, Malone Hall, 3400 N
Charles St, Baltimore, MD 21218, USA. E-mail: unberath@jhu.edu

How to cite this article: Ding H, Seenivasan L, Killeen BD, Cho SM, Unberath M. Digital twins as a unifying framework for surgical
data science: the enabling role of geometric scene understanding. Art Int Surg 2024;4:109-38. https://dx.doi.org/10.20517/ais.
2024.16
Received: 29 Feb 2024 First Decision: 20 May 2024 Revised: 6 Jun 2024 Accepted: 26 Jun 2024 Published: 5 Jul 2024

Academic Editor: Thomas Schnelldorfer Copy Editor: Dong-Li Li Production Editor: Dong-Li Li

Abstract
Surgical data science is devoted to enhancing the quality, safety, and efficacy of interventional healthcare. While
the use of powerful machine learning algorithms is becoming the standard approach for surgical data science, the
underlying end-to-end task models directly infer high-level concepts (e.g., surgical phase or skill) from low-level
observations (e.g., endoscopic video). This end-to-end nature of contemporary approaches makes the models
vulnerable to non-causal relationships in the data and requires the re-development of all components if new
surgical data science tasks are to be solved. The digital twin (DT) paradigm, an approach to building and
maintaining computational representations of real-world scenarios, offers a framework for separating low-level
processing from high-level inference. In surgical data science, the DT paradigm would allow for the development of
generalist surgical data science approaches on top of the universal DT representation, deferring DT model building
to low-level computer vision algorithms. In this latter effort of DT model creation, geometric scene understanding
plays a central role in building and updating the digital model. In this work, we visit existing geometric
representations, geometric scene understanding tasks, and successful applications for building primitive DT
frameworks. Although the development of advanced methods is still hindered in surgical data science by the lack of
annotations, the complexity and limited observability of the scene, emerging works on synthetic data generation,
sim-to-real generalization, and foundation models offer new directions for overcoming these challenges and
advancing the DT paradigm.

Keywords: Surgical data science, digital twin, geometric scene understanding

© The Author(s) 2024. Open Access This article is licensed under a Creative Commons Attribution 4.0
International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, sharing,
adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as
long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and
indicate if changes were made.

www.oaepublish.com/ais