Page 75 - Read Online

Novotny et al. Art Int Surg 2024;4:376-86 https://dx.doi.org/10.20517/ais.2024.52 Page 378

The ability to import intraoperative imaging in the form of 3D images directly to the surgical site increases
[10]
anatomical accuracy, provided that the imaging is up-to-date . With regard to the intraoperative use of AI,
three categories can again be defined: intraoperative guidance, operative use of robotic devices, education
[7]
and training . Image-guided surgery (IGS) systems improve orientation in the surgical field and, thus,
patient outcomes. IGS can help the surgeon to identify anatomical structures quickly and reliably, thus
reducing the duration of the operation and the workload . Before IGS can be used, a precise plan must be
[11]
drawn up, starting with the creation of a 3D model based on available imaging (CT, MRI). Detailed 3D
planning of the prevailing conditions supports the surgical planning. VR-assisted planning uses VR to
enable precise preparation for surgery using a patient-specific model. VR-based surgical navigation often
combines augmented reality (AR) technology to assist during surgery by displaying additional information,
[12]
such as anatomical details of the patient of the planned procedure . The ability to import intraoperative
imaging in the form of 3D images directly to the surgical site increases anatomical accuracy, provided that
the imaging is upto-date . Using VR systems such as ImmersiveView VR, surgeons can analyze 3D
[10]
representations of the patient’s anatomy before surgery. An algorithm detects subcutaneous fat and muscle
and displays the anatomical structures . All the necessary landmarks, structures, surgical steps, and
[12]
positional control points are incorporated into this model or plan. These virtual landmarks are
imperceptible to the naked eye and do not require physical attachment to the patient. Instead, they are
[13]
digitally integrated into the 3D model .
The FDA-cleared AI tool “Cydar EV Maps (Cydar Medical, UK)” can use preoperative and intraoperative
fluoroscopic imaging to create an anatomical map and update it in real time during surgery. This
anatomical map helps to identify and address complex conditions in a more patient-specific way. Having an
up-to-date picture of the situation at all times can be a huge advantage in complex and lengthy operations,
as conditions can change, and preoperative images are no longer sufficient for assessment. To make this
map more usable for the surgeon, it is sent to head-mounted displays or glasses. The current market-leading
system is HoloLens (Microsoft Corporation, Redmond, WA, USA) . Furthermore, in complex resections
[14]
of facial tumors and reconstructions of the head and neck, wearing video goggles allows the surgeon to
superimpose three-dimensional X-rays over the patient so that the pathology can be clearly seen and
identified. This gives the surgeon more options and helps them make a better decision. Inevitably, the result
[15]
is a reduction in human error .

Mixed reality with HoloLens
VR submerges the user completely in a virtual world, blending out the real world. AR combines the virtual
and real worlds by inserting virtual content into the real image. This factor gives AR a major advantage over
VR and allows it to be used in medicine. Mixed reality (MR) is a specific form of AR. MR is an environment
where users can interact with both real and virtual objects in real time [14,16] . One of the leading providers of
such MR devices is Microsoft, with its HoloLens . HoloLens is a head-mounted display similar to typical
[17]
smart glasses. It allows the user to extend their real world and interact with the virtual world using
holograms . HoloLens enhances surgical outcomes by optimizing the surgeon’s visual-motor
[18]
coordination, addressing the misalignment that currently exists between the surgeon’s line of sight and
hand placement caused by monitor positioning. This misalignment can be eliminated by using HoloLens
glasses . Another success was achieved in the differentiation of tissue layers. Using HoloLens, the
[19]
individual tissue layers could be clearly labeled and identified by the surgeon . Conventionally experienced
[18]
surgeons differentiate the tissue macroscopically to confirm or exclude cancer cells; the incisions are sent
intraoperatively to the pathologist to ensure that the resection is inside the healthy tissue border.