Page 70 - Read Online

Page 56 Harib et al. Intell Robot 2022;2(1):37-71 https://dx.doi.org/10.20517/ir.2021.19

[139]
In 2017, He et al. proposed an Adaptive NN-based controller for a robotic manipulator with time-
varying output constraints. The adaptive NNs were utilized to adjust for the robotic manipulator system's
uncertain dynamics. The disturbance-observer (DO) is designed to compensate for the influence of an
unknown disturbance, and asymmetric barrier Lyapunov Functions (BLFs) are used in the control design
process to avoid violating time-varying output constraints. The effects of system uncertainties are
successfully corrected, and the system's resilience is increased using the adaptive NN-based controller. The
NN estimating errors are coupled with the unknown disturbance from people and the environment to form
a combined disturbance that is then approximated by a DO.

In a recent interesting paper, He et al. attempted to control the vibrations of a flexible robotic
[140]
manipulator in the presence of input dead-zone. The lumped technique is used to discretize the flexible link
system [141,142] . A weightless linear angular spring and a concentrated point mass are used to partition the
flexible link into a finite number of spring-mass parts. They design NN controllers with complete state
feedback and output feedback based on the constructed model. All state variables must be known to provide
state feedback. An observer is presented to approximate the unknown system state variables in the case of
control with output feedback. In summary, an overview of the evolution of NNs implementation in robotic
manipulation is shown in Table 4. Each of these papers has been categorized based on the nature of its
approach.

3.4. From machine learning to deep learning
ML has transformed various disciplines in the previous several decades, starting in the 1950s. NN is a
subfield of ML, a subset of AI, and it is this subfield that gave birth to Deep Learning (DL). There are three
types of DL approaches: supervised, semi-supervised, and unsupervised. There is also a category of learning
strategy known as RL or DRL, which is commonly considered in the context of semi-supervised or
unsupervised learning approaches. Figure 8 shows the classification of all the aforementioned categories.

The common-sense principle behind RL is that if an action is followed by a satisfying state of affairs, or an
improvement in the state of affairs, the inclination to produce that action is enhanced, or in other words
reinforced. Figure 9 presents a common diagram model of general RL. The origin of RL is well rooted in
computer science, though similar methods such as adaptive dynamic programming and neuro-dynamic
[143]
programming (NDP) were developed in parallel by researchers and many others from the field of
optimal control. NDP was nothing but reliance on both concepts of Dynamic-Programming and NN. For
the 1990’s AI community, NDP was called RL. This is what makes RL one of the major NN approaches to
[60]
learning control .
On the other hand, deep models may be thought of as deep-structured ANNs. ANNs were first proposed in
[144]
1947 by Pitts and McCulloch . Many major milestones in perceptrons, BP algorithm, Rectified Linear
Unit, Max-pooling, dropout, batch normalization, and other areas of study were achieved in the years that
followed. DL’s current success is due to all of these ongoing algorithmic advancements, as well as the
appearance of large-scale training data and the rapid development of high-performance parallel computing
[145]
platforms, such as Graphics Processing Units . Figure 10 shows the main types of DL architectures. In
2016, Liu et al. proposed a detailed survey about DL architectures. Four main deep learning architectures,
[146]
which are restricted Boltzmann machines (RBMs), deep belief networks (DBNs), autoencoder (AE), and
convolutional neural networks (CNNs), are reviewed.

4. RL/DRL FOR THE CONTROL OF ROBOT MANIPULATION
DRL combines ANN with an RL-based framework to assist software agents in learning how to achieve their