Page 45 - Read Online

Page 6 of 14 Farinha et al. Mini-invasive Surg 2023;7:38 https://dx.doi.org/10.20517/2574-1225.2023.50

A questionnaire was completed immediately after using the 3D TMs [17-19,29] with a five- [18,29] or 100-point
[17]
Likert scale , and two studies did not report the type of assessment scale used [19,26] . All models were
reported as having “good realism” [18,26] concerning the form and structure of the kidney and as being
[29]
“high” or even superior to porcine or cadaveric models . One study reported detailed face validity data
[19]
for the model’s overall feel (mean 79.2), usefulness (mean 90.7), realism for needle driving (mean 78.3),
[17]
cutting (mean 78.0), and visual representation (mean 78.0) .
The VR TMs were evaluated with a questionnaire immediately after the model’s use [27,28] , and the questions
[27]
were scored using a five- or ten-point Likert scale . One VR TM study reported that the full-length AR
[28]
platform was very realistic (median 8/10, range 5-10) compared to the in vivo porcine model (median 9/10,
[27]
range 7-10, P = 0.07) , and another study reported a mean score for anatomical integrity of 3.4 (± 1.1)
using a five-point analog visual scale .
[28]
Content validity
Content validity measures whether skills training on a simulator is appropriate and correct, classifying the
model’s usefulness as a training tool . Our research identified content validity studies on animals [9,14-16] ,
[18]
four on 3D printed [17-19,29] and two on VR TMs [27,28] . All participants, including novices (without any surgical
experience), were evaluated in three animal [9,16,31] , four 3D [17-19,29] , and two VR [27,28] TM studies and were
[14]
exclusively assessed by experts in one animal study .
In animal TM studies, qualitative evaluations are derived from unspecified questionnaires. Either
[31]
participants found the model “helpful” , rated it as an “extremely useful” training tool for residents (9/10;
range 7.5-10) and Fellows (9/10; range 7-10), although less so for experienced robotic surgeons (5/10; range
3-10), or high ratings of usefulness (4/5) were attributed by participating residents . In one study, TM was
[9]
evaluated exclusively by experts who considered it to enhance their learning experience (96%) .
[14]

In 3D TM studies, unspecified questionnaires use qualitative evaluation and Likert scales to assess and
report results on content validity. One model is “recommended as a teaching tool” for residents and
fellows . Another was considered “useful as a training tool” by 93.7% of the participants , and another
[18]
[19]
study reported a total content score of 4.2 using a five-point Likert scale .
[29]
Using a non-validated questionnaire and a 0-100 Likert scale anchored to useless-useful, one model reached
90.7 for overall usefulness for training, being considered most useful “for trainees to obtain new technical
skills” (mean score 93.8) and less useful “for trainees to improve existing technical skills” (mean score
85.7) . The only study in this group of TMs, in which the assessment was exclusively performed by experts,
[17]
[26]
did not report data on content validity .
Using an unspecified questionnaire, experts rated the procedure-specific VR renorrhaphy exercise as highly
useful for training residents and fellows, although less useful for experienced robotic surgeons new to
RAPN. The model was highly rated for teaching surgical anatomy (median 9/10, range 4-10) and procedural
steps (8.5/10, range 4-10). Technical skills training was rated slightly lower, although still favorably (7.5/10,
range 1 to 10) . Using a visual analog scale (score range 1-5), the surgeons evaluated the utility of the
[27]
simulations, attributing a score of 4.2 (± 1.1) .
[28]
Construct validity
Construct validity denotes the ability of a simulator to differentiate between experts and novices on given
[18]
tasks , thereby providing clinically meaningful assessments . Our review identified six cohort studies on
[18]
construct validity [9,16,17,19,27,29] .