Page 18 - Read Online

Page 16 of 26 Jin et al. Soft Sci 2023;3:8 https://dx.doi.org/10.20517/ss.2022.34

In addition to operating tasks using direct contact between the manipulator (tactile sensors) and the target
object, it is also necessary for robotics to grab an external tool for specific manipulation tasks. Since human
beings can extend operation capacity with various tools, robotics also greatly benefits from tool
manipulation in multiple application scenarios such as industrial production, medical surgery, and
domestic service. Compared with ordinary grasping or non-grasp-based operational tasks, tool
manipulation is more difficult since the tactile devices should measure the contact events or forces between
the tools and objects . Hoffmann et al. controlled a robotic gripper to manipulate a pencil using tactile
[17]
[163]
feedback [Figure 8D] . Other robotic tool manipulating works have also been reported, such as screw
twisting , knife cutting and so on. For the development of humanoid robots, the operated tools are
[40]
[164]
more desired to be apart from the manipulator or tactile devices, and researchers are more recommended to
extract features of contact events in the distance using advanced algorithms.
Human-machine interactions
Due to the inevitable uncertainty of working space caused by varied environmental parameters and other
subjects (human beings or other robots), robotics are more desired to perform sophisticated manipulations
and cope with changeable information, especially interactive missions with human beings. Human-machine
interaction (HMI) is an information exchange process between human beings and robotics or sensing
devices, which significantly contributes to the development of Tri-Co Robot (i.e., the Coexisting-
Cooperative-Cognitive Robot) . Based on the interaction level, HMI can be classified into human being
[165]
parameters measuring, input interface for robotic controlling, and close-looped duplex interaction with
sufficient feedback. This section discusses some recently reported works with different HMI applications.

Human being parameters monitoring
It is of significant priority to ensure that the robotics understand the characteristic parameters of the object,
which is fundamental to conducting interactive activities. Since human beings are the central constituent
part of most HMI tasks, numerous works monitoring human body parameters have been reported [4,18] .
Health monitoring has long been a research emphasis in clinical medicine, and skin electronics detecting
[166]
[167]
arterial pulse waves , body temperature , and wound damage can provide a practical reference for
[129]
patient-care and surgery robots. Wearable skin patches monitoring the composition of sweat or exhaled
[130]
[168]
air also contribute to disease diagnosis and prevention. Recently, biocompatible and implantable sensing
[169]
devices attached to internal organ surfaces have aroused wide interest . They can capture physiological
[170]
parameters such as electrocardiograph (ECG) or electroencephalogram (EEG) , which help robotics to
[51]
recognize the interactive target accurately.
In addition to the health state, human movement or activity information also serves as a critical reference
for HMI tasks. Stretchable devices attached to the fingers, wrists, necks, limb joints, or heel tendons can
easily tell the movement of key joints in the human body for posture identification [99,166,171-173] . Moreover,
strain or vibration-sensitive skin patches covering faces, throats or muscles spread all over the body also
help extract human emotion, voice information, or electromyography (EMG) signals [20,174,175] . Besides, some
wearable interactive interface with advanced data processing algorithms shows great advantages in human
motion or pose recognition . Sundaram et al. developed a knitted tactile glove with numerous sensing
[107]
units, and human hand gestures can be accurately identified using machine learning methods of CNN and
t-SNE, as seen in Figure 9A .
[34]
Input interface for robotic controlling
Once ensuring that the robots can receive and understand external information, it is determined by human
manipulators to input interactive goals and control instruction to interact with robotics to complete specific