Page 152 - Read Online

Page 2 of 19 Ao et al. Intell Robot 2023;3(4):495-513 I http://dx.doi.org/10.20517/ir.2023.28

1. INTRODUCTION
Themusclesynergyofhandgesturesplaysacriticalroleinachievingaccuratecontrolofrobotichandsandintel-
ligentprosthetic limbs. Therearemanyrepresentative studies in related fields. Huetal. proposed anintelligent
stretchable capacitive electronic skin to endow soft body robots with high proprioceptive body awareness. The
proposed e-skin can accurately capture various complex three-dimensional (3D) deformations of the entire
soft body through multi-position capacitive measurements. The signals from the e-skin can be directly con-
verted into a high-density point cloud that depicts the complete geometry through a transformer-based depth
architecture. This high PGR proprioceptive system provides millimeter-scale local and global geometric re-
constructions that can assist in solving fundamental soft-body robotics problems such as accurate closed-loop
[1]
control and digital twin modeling . Park et al. described the design and control of a wearable robotic device
powered by a pneumatic artificial muscle actuator for ankle-foot rehabilitation. A key feature of the device
is its soft structure that provides active assistance without limiting the natural degrees of freedom of the an-
kle. Four actuated artificial muscles assist in dorsiflexion, plantarflexion, inversion, and valgus. The prototype
[2]
is also equipped with a variety of embedded sensors for gait pattern analysis . Gesture recognition is typi-
cally accomplished through the utilization of surface electromyography (sEMG) signals, which are a result of
[3]
the electrical activity of superficial muscles and nerves on the skin surface . In the field of human-machine
[4]
interaction, sEMG signals are highly practical due to their non-invasive nature and ease of manipulation .
Gesture recognition is a fundamental component of human-machine interaction and heavily relies on the fea-
tures of sEMG signals for classification and prediction. Conventional methods of gesture recognition involve
manual extraction of time-domain (TD) and frequency-domain features [5,6] . However, sEMG-image-based
gesture recognition is a promising new technique that has emerged. Geng et al. utilized a convolutional neural
network (CNN) with one frame of a high-density sEMG signal as input, achieving a gesture recognition rate of
[7]
89.30% . In addition, research has focused on improving preprocessing techniques to enhance the accuracy
[8]
of gesture recognition, such as the use of a sliding window to capture sEMG signals in the time domain .

In the neuromuscular control mechanism, the nerve does not control a muscle alone but recruits multiple mus-
cles on the spinal cord layer to form a muscle synergy, where muscles in the same muscle synergy are activated
simultaneously. Muscle synergy is considered to be the smallest unit of motor control in the central nervous
system. Various methods have been employed for muscle synergy analysis, including non-negative matrix
[9]
factorization , principal component analysis [10] , and independent component analysis [11] . Typically, these
methods extract the spatial and temporal components of muscle synergy from sEMG signals and then utilize
standard eigenvalues to analyze the correlation between data. However, these approaches necessitate prior
knowledge to perform correlation analysis between muscles, thus rendering them non-end-to-end. End-to-
end means that the input is the raw data, and the output is the result. The classical machine learning approach
is to preprocess raw data into features with human a priori knowledge and then use the features to classify the
data. The classification result depends on the goodness of the features, so it takes time to design the features.
The traditional method of muscle synergy analysis also requires experts to use various methods to extract the
spatio-temporal components of muscle synergy and then use standard feature values to analyze the correlation
between the data. Moreover, the acquisition of standard eigenvalues as objects is often a challenging task.

The theory of multiple game interactions is a powerful tool for analyzing the value of interactions between
different members [12] . This theory seeks to condense all interactions into a single metric, providing a new
method for explaining the underlying prototypical features encoded by neural networks. However, as the
number of interacting members increases, the computation required to calculate the marginal contribution of
each member increases exponentially. Grad-CAM (gradient-weighted class activation mapping) can be used
to address this issue, which is an interpretable method for feature visualization and input unit importance
attribution in neural networks [13–15] . While previous studies have not investigated the interpretability of ges-
ture recognition methods based on surface electromyographic signals using this technique, we present the first