Page 153 - Read Online

Ao et al. Intell Robot 2023;3(4):495-513 I http://dx.doi.org/10.20517/ir.2023.28 Page 3 of 19

results of an interpretability analysis of gesture recognition based on CNN [16] . Our approach visualizes the
crucial features of muscles and focuses on muscle conjugation analysis to reduce redundant calculations.

To address the issues mentioned above, this paper provides a Shapley-value-based muscle synergy (SVMS)
analysis method that quantitatively analyzes both the synergy among single-channel muscles and muscle
groups in different gestural movements [17] . First, the sEMG signal is preprocessed using a TD sliding win-
dow, and the resulting signal is converted into a color image to improve gesture recognition accuracy. Next,
theGrad-CAMinterpretableanalysismethodisutilizedtoidentifysignificantfeatureregionsforthesEMGim-
age. This approach not only directly reflects the correspondence between a gesture and electrode importance
but also reduces computational expense. Finally, the synergistic effects of forearm muscle groups, upper limb
extensors, and upper limb flexors related to gesture recognition are quantitatively analyzed using the Shapley
value method. The end-to-end functionality of the SVMS method resolves the issue of muscle correlation
analysis. The main contributions of this study are summarized as follows:
1. An end-to-end muscle synergy analysis method is designed to analyze the interaction between muscles
from a new perspective. This method facilitates exploration of the correlation between upper extremity
flexors, upper extremity extensors, and other muscles.
2. A data extraction method based on a sliding time domain window is designed to improve the prediction
accuracy of various gestures in the stage of data preprocessing. The Grad-CAM interpretation method is
applied to visualize the importance attribution of the inputs, which intuitively reflects the importance of
muscles corresponding to each electrode for different gestures.
3. sEMGelectrodesareemployedasgamememberstocomputethemarginalcontributionthroughinteractive
game theory. This approach enables the quantitative analysis of the interactions between different muscles.
The rest of this paper is organized as follows: Section 2 explains the related methods involved in SVMS, includ-
ingdatapreprocessingmethodsforsEMGimages, modelbuildingforaCNN,Grad-CAM,andmusclesynergy
analysis. Section 3 presents experimental and analysis results. Finally, concluding remarks are presented in
Section 4.

2. SVMS METHOD
In this section, we first provide the method of converting sEMG signals into sEMG images, which involves
convertingsEMGsignalsintogreyscaleimagesandthenmergingthegreyscaleimagesintoRGBthree-channel
color sEMG images. In the next step, we present the model of the CNN for the classification of the input color
sEMG images on hand gestures. We then provide the Grad-CAM method to visualize the important features
of the input color sEMG images and to gain an intuitive understanding of the important feature areas. Finally,
we implement a muscle synergy analysis based on SVMS for the muscle groups corresponding to the different
electrodes of the 12 gestures.

With the method designed in the above steps, we can realize the muscle synergy analysis of forearm muscle
groups and upper-limb flexor and upper-limb extensor, which are commonly used for gesture recognition.
Figure 1 shows the overall framework of the SVMS method.

2.1. sEMG image preprocessing
To convert the raw sEMG signal to a color sEMG image, it is necessary to preprocess the raw signal to obtain
the sEMG signal parameter matrix. The parameters of the parameter matrix are mapped to 0-255 to obtain a
greyscale image, and then the single-channel greyscale image is merged into a multi-channel color image.

We selected the sEMG signals acquired from sparse electrodes in the Ninapro dataset [18] . The twelve gesture
images corresponding to this data subset are shown in Figure 2 below. The raw signal is first band-pass filtered