Page 151 - Read Online

Ao et al. Intell Robot 2023;3(4):495-513 Intelligence & Robotics
DOI: 10.20517/ir.2023.28

Research Article Open Access

Muscle synergy analysis for gesture recognition based
on sEMG images and Shapley value

4
Xiaohu Ao 1,2,3 , Feng Wang 1,2,3 , Rennong Wang , Jinhua She 4
1 School of Automation, China University of Geosciences, Wuhan 430074, Hubei, China.
2 Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, Wuhan 430074, Hubei, China.
3 Engineering Research Center of Intelligent Technology for Geo-Exploration, Ministry of Education, Wuhan 430074, Hubei, China.
4 School of Engineering, Tokyo University of Technology, Hachioji, Tokyo 192-0982, Japan.

Correspondence to: Prof. Feng Wang, School of Automation, China University of Geosciences, No.388 Lumo Road, Hongshan
District, Wuhan 430074, Hubei, China. E-mail: wangfeng@cug.edu.cn

How to cite this article: Ao X, Wang F, Wang R, She J. Muscle synergy analysis for gesture recognition based on sEMG images and
Shapley value. Intell Robot 2023;3(4):495-513. http://dx.doi.org/10.20517/ir.2023.28
Received: 30 May 2023 First Decision: 21 Jun 2023 Revised: 12 Sep 2023 Accepted: 26 Sep 2023 Published: 24 Oct 2023

Academic Editors: Simon X. Yang, Zuojin Li Copy Editor: Yanbin Bai Production Editor: Yanbin Bai

Abstract
Muscle synergy analysis for gesture recognition is a fundamental research area in human-machine interaction, par-
ticularly in fields such as rehabilitation. However, previous methods for analyzing muscle synergy are typically not
end-to-end and lack interpretability. Specifically, these methods involve extracting specific features for gesture recog-
nition from surface electromyography (sEMG) signals and then conducting muscle synergy analysis based on those
features. Addressing these limitations, we devised an end-to-end framework, namely Shapley-value-based muscle
synergy (SVMS), for muscle synergy analysis. Our approach involves converting sEMG signals into grayscale sEMG
images using a sliding window. Subsequently, we convert adjacent grayscale images into color images for gesture
recognition. We then use the gradient-weighted class activation mapping (Grad-CAM) method to identify signifi-
cant feature areas for sEMG images during gesture recognition. Grad-CAM generates a heatmap representation of
the images, highlighting the regions that the model uses to make its prediction. Finally, we conduct a quantitative
analysis of muscle synergy in the specific area obtained by Grad-CAM based on the Shapley value. The experimental
results demonstrate the effectiveness of our SVMS method for muscle synergy analysis. Moreover, we are able to
achieve a recognition accuracy of 94.26% for twelve gestures while reducing the required electrode channel informa-
tion from ten to six dimensions and the analysis rounds from about 1000 to nine.

Keywords: Shapley-value-based muscle synergy analysis (SVMS), surface electromyography (sEMG) image, gesture
recognition, Grad-CAM

© The Author(s) 2023. Open Access This article is licensed under a Creative Commons Attribution 4.0
International License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, shar-
ing, adaptation, distribution and reproduction in any medium or format, for any purpose, even commercially, as long as you
give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate
if changes were made.

www.intellrobot.com