Page 7 - Read Online

Fan et al. Complex Eng Syst 2023;3:5 I http://dx.doi.org/10.20517/ces.2023.04 Page 3 of 15

As can be seen from the above, there are few studies that combine the DTC problem with the ZS game prob-
lem. It is necessary to take the related discounted cost function into account for the DTC system, which can
transform the DTC problem into an optimal control problem with disturbances. In practice, the existence of
disturbances will make an unpredictable impact on the plant. Hence, it is of vital importance to consider the
stability of the DTC system. In the experimental simulation, it is a challenge to achieve the goal of effective
online weight training, which is implemented under the tracking control law and the disturbance control law.
Consequently, in this paper, we put forward a novel method in view of ADP to resolve the DTC problem
with external disturbances for continuous-time (CT) nonlinear systems. More importantly, for the sake of
overcoming the difficulty of selecting initial admissible control policies, an additional term is added during
the weight updating process. Remarkably, in this paper, we introduce the discount factor for maximizing and
minimizing the corresponding cost function.

The contributions of this paper are as follows: First, considering the disturbance input in the DTC system, the
strategy feasibility and the system stability are discussed through theoretical proofs. It is worth noting that the
discount factor is introduced to the cost function. Moreover, in the process of online weight training, we can
make the DTC system reach a stable state without selecting the initial admissible control law. Additionally, we
present the experimental process of the spring-mass-damper system. Besides, we derive the desired tracking
error curves as well as control strategy curves, which demonstrates that they are uniformly ultimately bounded
(UUB).

The whole paper is divided into six sections. The first section is the introduction of relevant background
knowledges of the research content. The second section is the problem statement of basic problems about
the two person ZS game and the DTC strategy. In the third section, we design the decentralized tracking
controller by using the optimal control method through solving the HJI equations. Meanwhile, the relevant
lemma and theorem are given to validate the establishment of the DTC strategy. In the fourth section, the
design method in accordance with adaptive critic is elaborated. Most importantly, an improved critic learning
rule is implemented via critic networks. In the fifth section, the practicability of this method is validated by an
interconnected spring-mass-damper system. Finally, the sixth section displays conclusions and summarizes
overall research content of the whole paper.

2. PROBLEM STATEMENT
Consider a CT nonlinear interconnected system with disturbances, which is composed of interconnected
subsystems. Its dynamic description can be expressed as

¯
¤ ( ) = ( ( )) + ( ( )) ¯ ( ( )) + ( ( )) + ℎ ( ( )) ( ( )), (1)
where = 1, 2, . . . , , ( ) ∈ R is the state vector of the th subsystem and ( ) denotes the partial inter-

connected state related to other subsystems of the large-scale system. ¯ ( ( )) ∈ R is the control input

and ( ( )) ∈ R is the external disturbance input. As for the th subsystem, we denote ( ( )), ( ( )),

¯
ℎ ( ( )), and ( ( )) as the nonlinear internal dynamics, the input gain matrix, the disturbance gain matrix,
T T T T
and the interconnected item in sequence. Besides, [ , , . . . , ] ∈ R denotes the whole state of the

1
2
Í
large-scale system Equation (1), where = . Accordingly, 1 , 2 , . . . , are named local states and
=1
¯ 1 ( 1 ), ¯ 2 ( 2 ), . . . , ¯ ( ) are called local controllers. We let ∈ R × be the symmetric positive definite
1/2 ¯
( ( )). In addition, ( ( )) ∈ R is bounded as follows:
matrix and denote ( ( )) =

Õ Õ
k ( ( ))k ≤ ( ) ≤ ( ), (2)
=1 =1
where = 1, 2, . . . , , is the nonnegative constant, ( ) is the positive semidefinite function. Besides,
we define ( ) = max { 1 ( ), 2 ( ), . . . , ( )} and the element of { 1 ( ), 2 ( ), . . . , ( )}