Page 18 - Read Online

Liu et al. Art Int Surg 2024;4:92-108 https://dx.doi.org/10.20517/ais.2024.19 Page 102

Figure 4. Graphical comparisons of distance traversal patterns among individual tracklets engaging in unique actions captured by 5-
second clips. Subjects engaging in walking movements (A and B, left column) exhibit substantially higher levels of cumulative and local
distance changes compared to individuals engaging in hand-tool interactions (C and D, middle column) and in observation of peer
activities (E and F, right column).

Lastly, we performed an experiment to analyze the effect of different frame sampling rates on model
performance. We found that sampling 10 frames for each second in the clip and constructing the mesh
sequence from the corresponding frames is optimal for performance. We observed performance drops
across all metrics at frame sampling settings that were lower and higher than this optimal setting [Table 4].

DISCUSSION
This section describes overarching interpretations of the results surrounding our HMR framework, surgical
behavior analysis, and action recognition model. We also discussed the limitations and practical
implications of our study.

Experimental interpretation
Our results provided evidence that an automated, human mesh-centered approach to OR video
understanding can produce meaningful insights into surgical behaviors and short-duration OR actions.
Notably, we showed that in addition to information on subject behavior that can be analyzed from meshes
in single frames, such as visual attention, subject positioning, and joint pose, we can make more nuanced
inferences on actions that persist across short durations. These capabilities are significant in the OR, as
subtle actions and behaviors can be important predictors of team performance, operation trajectory, and
patient outcomes.

Our experiments on surgical behavior analysis underscored the utility of granular mesh embeddings in
representing individual behavior. Comparisons of positional heatmaps provided an interpretable way to
understand how subjects are positionally distributed in the OR. Substantial differences in the positional and