Coordinated Energy-Efficient Walking Assistance for Para-plegic Patients by Using the Exoskeleton-Walker System

Overground walking can be achieved for patients with gait impairments by using the lower limb exoskeleton robots. Since it is challenge to keep balance for patients with insufficient upper body strength, a robotic walker is necessary to assist the walking balance. However, since the walking pattern is varying over time, how to control the robotic walker to coordinated follow the walking of the human-exoskeleton system is a critic issue. Inappropriate control strategy leads to the unnecessary energy cost of the human-exoskeleton-walker (HEW) system, and also results in the bad coordination between the human-exoskeleton system and the robotic walker. In this paper, we proposed a Coordinated Energy-Efficient Control (CEEC) approach for the HEW system, which is based on the extremum seeking control algorithm and the coordinated motion planning strategy. CEEC is used to find the optimal supporting force of the support joint and generate appropriate reference joint angles for wheels of the robotic walker, as a result, the energy efficiency of the HEW system can be maximized. The proposed approach has been tested on the HEW simulation model, and the experimental results indicate that the coordinated energy-efficient walking can be achieved with the proposed approach, which is increased by 60.16% compared to the conventional passive robotic walker. In addition to this, the proposed approach is adaptive to different subjects.


INTRODUCTION
Overground walking is necessary and important for patients with gait impairments, which can be implemented by using the lower limb exoskeleton robots.Usually, it is hard for patients with insufficient upper body strength to keep balance in the early rehabilitation stages, therefore a mobile robotic walker is necessary to assist the overground walking and gait training.As shown in Fig. 1, the human-exoskeleton-walker system is presented.Note that in the following sections, the human-exoskeleton system means a subject wearing the exoskeleton robot, and the human-exoskeleton-walker (HEW) system means the human-exoskeleton system with the robotic walker.As shown in Fig. 1, the human-exoskeleton system is connected to the mobile robotic walker with a solid cantilever, a support joint is attached to the cantilever for the vertical movement assistance and weight supporting during walking.The robotic walker ensures the stability of the human-exoskeleton system in the coronal plane, validly keeping balance and preventing falls.
However, without an appropriate supporting force offered by the support joint and coordinated movement of the wheels to follow the walking of the human-exoskeleton system, the human-exoskeleton system has to pull or push the robotic walker forward during walking, which leads to the bad walking posture and unnecessary energy-cost of the human-exoskeleton system.As a result, battery life has always been a severe challenge to the exoskeleton robots.The same problem occurs on some current existing exoskeleton robots, such as [1][2][3][4] .Indego [3] has only four hours of battery life.Even with the largest battery capacity, ReWalk [1] and SuitX [2] allow for continuous work not more than five hours.Therefore, it is crucial to improve the energy efficiency of the human-exoskeleton-walker system with an appropriate supporting force offered with the support joint as well as the coordinated movement of the wheels to follow the natural walking of the human-exoskeleton system.
In the last decades, several approaches have been studied with the body weight supporting (BWS) system for the energy-efficient walking assistance.Sun et al. proposed a BWS system for the three-dimensional walking in Cartesian space [5] , with the series elastic actuation structure to improve the human-robot interaction performance and reducing the energy cost of the human-exoskeleton system.Wei et al. proposed a surplus force control strategy named active loading compound control strategy for the BWS system, which is used for estimating and improving the loading accuracy [6] and reduce the surplus force as well as the energy cost.For the mobile robotic walker, Mun et al. [7] proposed a mobile robotic walker for the movement of the pelvis of human, and can be used to facilitate the over-ground walking without alteration of the normal gait dynamics.Similar structure has been developed in [8] .Daisuke et al. [9] developed a robotic walker to assist the standing motion and simple walking for the aged person in daily life, which estimates the load of the pelvis, knee and ankle joint of the human body, and generate appropriate joint angle for the support joint of the robotic walker.These mobile robotic walker is developed for the energy cost reduction of human, and not applicable for the HEW system.Some optimization based approach have been studied for the soft exosuit [10] and ankle exoskeleton robots [11] .
In this paper, we focus on the coordinated energy-efficient walking assistance of the HEW system, and the human-in-the-loop optimization for the energy consumption is the key research topic.The key point is to seek the optimal support force of the robotic walker during walking, and generate appropriate joint angles as the control reference of the wheels to produce a coordinated movement of the robotic walker and the human-exoskeleton system during walking.However, due to the unknown relationship between the energy efficiency and the support force, it is difficult to calculate the optimal support force during walking.In this paper, a Coordinated Energy-Efficient Control (CEEC) approach is proposed for the HEW system to provide the coordinated movement of the human-exoskeleton system and the robotic walker, as well as maximize the energy-efficiency of the HEW system.CEEC consists of a model-free Extremum Seeking Control (ESC) algorithm and a coordinated motion planning approach, which performs real-time modification of the support force and generation of the joint angles of the wheels.The main contributions are summarized as follows: • An optimal support force seeking strategy is proposed for the HEW system to improve the energy efficiency during walking, and adaptive to different subjects.• A coordinated motion planning approach is proposed for the HEW system, which performs the coordinated assistance of the robotic walker for human-exoskeleton system during walking.• The efficiency of the proposed approach has been tested on the HEW simulation models, the experimental results indicate that the energy-efficiency was improved by 60.16% compared to the conventional robotic walker.
The remainder of the manuscript is organized as follows: In Section 2, the detailed design of the proposed CEEC is presented.In Section 3, the simulation experiments of the proposed approach is presented, the experimental results and discussions are presented in Section 4. In Section 5, we concluded the paper and some future works are suggested.

METHODS
In this section, the design of the CEEC is presented, including the human-in-the-loop optimization of the support force and implementation of the coordinated motion planning approach of the wheels.As shown in Fig. 2, the framework of the proposed CEEC approach is presented, which consists of two parts: the support force optimization part and coordinated motion planning part.In the following two subsections, these two parts will be introduced in detail.

The optimization of the support force
In this subsection, the real-time optimization of the support force is presented, which aims at finding the optimal support force provided by the support joint of the robotic walker.The structure of the HEW system and the energy cost calculation during walking are presented in Section 2.1.1,and the implementation of the human-in-the-loop optimization for the support force is presented in Section 2.1.2.

Energy calculation of the HEW system with the support force
In this subsection, the energy cost of the human-exoskeleton system is presented, where the energy is determined by the power of the exoskeleton's active joints.The structure of the HEW system is shown in Fig. 3, where the exoskeleton robot includes hip, knee and ankle joints to drive the human-exoskeleton system walking forward.The support force   can be supplied by the support joint of the robotic walker, which is used to move the support joint vertically and support the weight of the human-exoskeleton system.The support joint of the robotic walker is actuated with a spring-based mechanism and a stepping motor, which provides a variable support force for the human-exoskeleton system with different motor positions.In addition,  represents the horizontal resistance of the robotic walker,   ,   represent the horizontal and vertical component of the supporting force offered by the support leg, respectively.  and   represent the length of the thigh and shank, respectively.  and   represent the hip and knee joint angles of the exoskeleton's support leg, respectively.Note that in this paper, the hip and knee joint angles are sampled from the healthy subjects as the control reference, and the hip and knee joints of the exoskeleton robot is driven by the DC motors with PID position controllers.The ankle joint of the exoskeleton robot is a passive joint with spring mechanism, which can be used to avoid the foot drop of the patients.
In this paper, we focus on the movement of the HEW system in the Sagittal plane, and the Center of Mass (COM) of the human-exoskeleton system has a good correspondence in the sagittal plane with the hip joints, therefore the COM of the human-exoskeleton system is set the the center of two hip joints.Assuming that the support leg's ankle joint is set as the origin of the Cartesian frame, the COM's position of the human-exoskeleton system can be described as follows: where the  com and  com represent the horizontal and vertical positions of the COM, respectively.Then the horizontal and vertical supporting force from the support leg   and   are expressed as follows: where   ,   and   represent the mass of the human subject, the mass of the robotic walker and the mass of the exoskeleton robot, respectively. com and  com are the second derivatives of the COM's position  com and  com . is the constant gravitational acceleration.The torques of the hip and knee joints of the support leg can be calculated as follows: where   and   are the torques of the hip and knee joints.As a result, the power of the exoskeleton robot is determined by the joint torques   and   , and the angular velocity of the joint   and   .Lets take the hip joint as an example, the power of the hip joint's motor   is calculated as: where   and   represent the torque and angular velocity of the motor in hip joint, respectively. represent the current of the motor, and  is the reduction ratio.Besides,   and   represent the resistance and torque constant of the motor, respectively.The first item of   in equation ( 4) represents the thermal power, while the second item represents the mechanical power.With the power of the motors in the joints of the support leg, the energy consumption of the support leg during the stance phase can be calculated as follows: where    and    represent motor power of the hip and knee joints, respectively. represents the duration of the stance phase in one gait cycle.
Based on the equations ( 2), (3), and (4), we can find that the torques of the hip and knee joints are decreased with the increasing of the support force   , i.e., the energy consumption of the exoskeleton robot is decreased with the increasing of the support force   .However, if the support force is too large, the human-exoskeleton system will be lifted off the ground, and the friction between the ground and the exoskeleton's foot will be reduced, resulting in a abnormal walking posture of the human-exoskeleton system and even with slipping over the ground.Therefore, how to find the appropriate support force to minimize the energy consumption and without any slip is a critic issue.Now lets construct an objective function to denote the energy efficiency: where  represents the energy consumption of both hip and knee joints of the support leg during the stance phase in one gait cycle,  represent the stepping length for one step.Lets call the objective function as Total Cost of Transport (TCoT).Now, lets find a way to solve the objective function and find the optimal support force.

Real-time optimization of the support force
In this subsection, the real-time optimization of the support force is presented, which employs the discrete-time Extremum Seeking Control (ESC) approach in the optimization approach.ESC is a model-free adaptive control method that finds an optimum set-point in order to minimize/maximize an objective function, whose analytical expression might be unknown [12] , [13][14][15] .Saurav Kumar [16] proposed a modified structure of the discrete-time ESC by introducing a stepper motor with an integrator [17] .In this modified structure, the ESC integration is performed by the motor dynamics itself.Moreover, the stepper motor also has the same characteristics as the zero-order holder, it does not have any closed-loop encoder feedback for position control.Instead, it accepts a variation in the motor location as an input command instead of the final motor location.In this paper, we used the variation in the motor location to tune the support force of the support joint.

E,S J(• )
High-pass Filter

Walking Energy
Efficiency The demodulation step The modulation step The block diagram of the modified ESC used in this paper is shown in Fig. 4, where the work flow of the ESC is as follows.Firstly, a periodic disturbance signal of small amplitude  1 () = − sin(  ) called the dither signal is added to the commanded change in the motor location Δ () in the modulation step.Assuming that the stepper motor dynamics is modeled as a cascade connection of a zero-order holder and a continuous-time integrator.The zero-order holder holds the sample Δ () +  1 () constant for one sampling interval Δ.Denoting that   is the sampling time, the expression for  (  ) ≈ Δ ()/Δ −  sin(  ), then the integrator dynamics of the stepper motor outputs  (  ) +  cos(  ), where  (  ) is the stepper motor's location at   .The output of the stepper motor is used to tune the support force by changing the position of the stepper motor in the support joint.Then the torques of the hip and knee joints is sampled in one gait cycle and calculate the power of the hip and knee joints according to the equation (4).Next, the power is multiplied by the sampling interval Δ, and we summed them up to get the total energy consumption of the hip and knee joints as .Therefore, the objective function can be rewrote as and the Taylor series approximation of  (•) is expressed as follows: where  ′ ,  ′′ are the first and the second derivatives of  (•) with respect to .Then the objective function  (•) is passed through a high-pass filter to remove the low frequency and DC components, the output of the high-pass filter  () is as follows: In the demodulation step,  () is multiplied by another dither signal  cos() and scaled by a gain −, then where Δ () indicates the amount that the stepper motor should be moved to minimize the TCoT, which realizes the support force adaptive tuning.From standard manipulations [18] , the equations of ESC can be updated as follows: where  represents the gain, and ℎ denotes the cut-off frequency of the high-pass filter.
Overall, with the proposed real-time optimization approach, the optimal support force can be found in real time by walking several steps overground with the human-exoskeleton-walker system.As a result, the energy cost of the HEW system will be reduced to the minimum value without any slipping overground.

The coordinated motion planning of the robotic walker
The coordinated motion planning is to make the human-exoskeleton system and the robotic walker moving coordinately, and avoid the pull or push between them.As shown in the Fig. 2, the coordinated motion planning is based on the hip and knee joint angles of the exoskeleton robot.As shown in Fig. 5 and Fig. 6, there are four wheels of the robotic walker, and all wheels rotate around the  axis.As we mentioned in Fig. 1, the exoskeleton robot is connected to the robotic walker with a solid cantilever, therefore the horizontal movement of the robotic walker is same with the COM of the human-exoskeleton system.Note that if the wheels are passive without any power, there is no active movement of the robotic walker, and the human-exoskeleton system has to pull or push the robotic walker while walking overground.If the wheels are actuated with the DC motors, they can drive the robotic walker to follow the movement of the human-exoskeleton system, and avoid the movement conflict between the human-exoskeleton system and the robotic walker.Therefore, the question is how to control these wheels to drive the robotic walker to coordinately follow the movement of the human-exoskeleton system?Based on the equation ( 1), the movement of the COM during walking can be calculated with the joint angles of the exoskeleton robot.And the horizontal movement of COM can be discretized with a constant unit time Δ, taking  as an index for discretization, then where Δ can be set to some small positive value such as Δ = 0.005 s.The COM horizontal displacement from  −1 to   can be calculated as follows: where   represent the horizontal position of the COM at   .Based on the COM horizontal displacement Δ  , the increment of the joint angles for the wheels from  −1 to   is calculated as follows: where   is the radius of the wheels on the robotic walker.Now, with the horizontal movement of the humanexoskeleton system, the reference joint angles of the wheels can be obtained, and the human-exoskeleton system and the robotic walker can be moved coordinately.

EXPERIMENTS
In this section, simulation experiments are conducted with the constructed human-exoskeleton-walker simulation models in the robot simulation platform CoppeliaSim1.The simulation model is shown in Fig. 5, the model retains the same configuration of degrees of freedom illustrated in Fig. 1, where the wheels are controlled with the PID position controllers and the support joint is controlled in the torque mode to support the weight of the human-exoskeleton system.The torques and the trajectories of the COM's movement can be obtained, and the power as well as the energy cost of the HEW system can be calculated, with the proposed CEEC approach, the optimal support force can be found after several steps of walking.In addition to this, the tracking performance of the COM's movement can be used to evaluate the coordination between the human-exoskeleton system and the robotic walker.

Experimental setup
To evaluate the performance of the proposed CEEC approach, several experiments were designed in the simulation experiments.Firstly, the hip and knee joint angles used in the experiment are sampled from the healthy subject, as shown in Fig. 7, where the joint angles in swing phase and stance phase are for the swing leg the support leg, respectively.The mass distribution of the human-exoskeleton system (torso, thigh, shank, and foot) follows average human anthropometry [19] , as shown in Table 1, and the length of the thigh and shank is set to 0.45 m, which is similar to the subject with the body height of 1.75 m.
Overall, the total mass of the human subject, the exoskeleton and the robotic walker as well as other parameters for the simulation experiments are shown in Table 2.Note that the parameters for the motors in hip and knee joints are refer to the manual of the DC motors used in our exoskeleton robots shown in Fig. 1.To evaluate the proposed CEEC approach and compare with the other approaches, four experiments were designed, four names were given to them.
• The first one is the baseline.there was no active assistance of the robotic walker, i.e., the human-exoskeleton system had to pull the robotic walker forward during walking.• The second one is the CMP, there was only the active assistance from the wheels with the generated coordinated motion planning, i.e., the wheels were controlled with the reference joint angles generated in the Section 2.2.In addition, there was no support force from the support joint.• The third one is the ESC, there was only the support force from the support joint under the ESC strategy, and there was no active assistance from the wheels.• The last one is the CEEC, there were both the active assistance from the support joint and wheels, the support force was optimized with the ESC strategy, the wheels were controlled with the reference joint angles in Section 2.2.
To evaluate the adaption of the proposed CEEC approach for different subjects, three subject simulation models with different masses are employed in the experiment, as shown in Table 3.Note that the ESC strategy is an online iterative algorithm, and the initial value of the support force should be set at the beginning of the experiment.Therefore two different initial support force were given, as shown in Table 3.For each trial of the experiment, seventeen steps (the first step and eight gait cycles) were conducted to test the efficiency of the proposed approach.Note that the first step of the experiment is a special step from the standing upright posture to walk, therefore the control strategy only works in the last sixteen steps.The sampling rate is 20 Hz, and the gait cycle is two seconds with two steps.The TCoT was computed with the sampled torque of the support leg's hip and knee joints after each gait cycle.The parameters of ESC were selected as follows: a=1.6, b=0.8, =0.8Hz, h=0.4 Hz, =-6.

Simulation experiments recording
The snapshots of the four experiments (baseline, CMP, ESC, CEEC) are shown in Fig. 8, where the four rows of figures are corresponding to the four different experiments, i.e., baseline, CMP, ESC and CEEC, respectively.It is significant that with different control strategies, the COM's tracking performances are different.The snapshots of the experiments with three different subjects and the CEEC control approach are shown in Fig. 9. Since the CEEC is adaptive to different scenarios, the walking performance are similar for subjects with different masses.For more detailed presentation of the whole walking experiment, please refer to the video attachment associated with the manuscript.

Experimental results for the comparison with the COM's trajectory tracking and the energy cost
First of all, the comparison of COM's trajectories tracking performance with baseline, CMP, ESC and CEEC during the whole walking experiments are shown in Fig. 10.Note that the HEW system is moving in the Sagittal plane, therefore only the  position and  position of the COM are presented.The black solid curve is the reference trajectory of the COM calculated from the reference joint angles based on the equation (1), the dashed purple curve is the COM's trajectory with the baseline (with no support force and no assistance from wheels), the solid red curve is the COM's trajectory with the CMP (with only active assistance from wheels and no support force), the dashed green curve is the COM's trajectory with the ESC (with only support force and no assistance from wheels), and the solid blue curve is the COM's trajectory with the CEEC (with both support force and assistance from wheels).
From the trajectory tracking comparison we can find that in the baseline case, since there is no any active assistance from the robotic walker, the human-exoskeleton system has to pull the robotic walker forward during 2) baseline: The human-exoskeleton walks without any assistance of the robotic walker.3) CMP: the HEW system walks with only the active wheel movements.4) ESC: the HEW system walks with only the support force under the ESC strategy.5) CEEC: The HEW system walks with the proposed coordinated energy-efficient control approach.
walking, the COM's trajectory tracking is bad, and the final position of the COM is far way from the desired position.
In the CMP case, since there is active assistance of the wheels with the coordinated motion generated with the reference COM trajectory, the tracking performance of the COM is well.For the ESC and CEEC cases, since there is a support force, the COM's trajectory tracking is better than the baseline, and especially for the CEEC case, the COM's trajectory tracking is better than the ESC case.Note that in the first several steps, the support force is not optimal, and the ESC algorithm is tuning to find the optimal support force, which results in a bad performance.But after several steps of optimization, the optimal support force is found, and the COM's trajectory tracking is better.
Above all, for the COM's trajectory tracking, CEEC is better than the ESC and baseline, but worse than the CMP.Now, lets see the energy cost during walking with these different strategies, as shown in Fig. 11.From the bars presented in Fig. 11, we can see that in baseline case, the energy cost is the very high than any other methods.In CMP case, the energy cost is always similar during the walking.In ESC and CEEC cases, the energy cost is very high at the beginning of the walking and decreased after several steps, this is because at the first several steps, the ESC algorithm needs to iteratively update the support force and find the optimal one, which leads to a bad walking performance and high energy cost.After several steps, the energy cost is reduced, and the CEEC is better than the ESC, this is because the CEEC not only optimizing the support force, but also provide horizontal coordinated walking assistance by wheels.Overall, considering on the COM's trajectory tracking performance and the energy cost, the CEEC is the best approach for the HEW system to finish a coordinated energy-efficient overground walking.

Experimental results for the comparison of subjects with different masses
To validate the adaptive capacity of the CEEC algorithm in different scenarios, subjects with different masses are employed as well as different initial support forces for CEEC are given, as shown in Table 3.As mentioned before, there are eight gait cycles of each experiment, the hip and knee joint torques of the support leg was recorded with the sampling frequency of 20 Hz for the CEEC algorithm.
The variation of the support force for different subjects and different initial support force settings are shown in Fig. 12, and all support forces will be iteratively updated and converged to the optimal one after several steps of walking.Note that for different subjects, the final optimal support forces are different, and mainly depends on the masses of the subject, this is because a bigger support force is needed for the heavier subject.So for the heaviest subject (A3 and B3) with the mass of 80 kg, the required support force is much bigger than the lightest subject (A1 and A3) with the mass of 40 kg.
The variation of the TCoT for different subjects and different initial support force settings are shown in Fig. 13.The TCoT can be calculated with the equations (4), ( 5) and (6), where the joint torques for during the walking can be found in Fig. 14.Then the support leg's energy consumption in one gait cycle can be obtained by integrating the power of hip and knee joints over the gait cycle, the stepping length is also obtained.Similar to the variation of the support force, the TCoT for different subjects could be converged to a almost constant value after three gait cycles.Note that for a same subject, the TCoT converged to a closed value after several steps, this is because the CEEC is a iterative updated algorithm and will tune the support force online, even with different initial support force settings, the final optimal support force only depends on the subject's masses in these experiments.The final optimal support force and the improvement of the energy-efficiency is shown in Table 4, where the TCoT in baseline case were chose to be compared with the CEEC case.The optimal support force in the table is the average final converged value in Fig. 12, and the converged TCoT in the table is the average final converged value in Fig. 13.Compared with the baseline, the CEEC reduced the TCoT by 58%, 60.9% and 61.6%, respectively.In other words, the average improvement of the energy-efficiency can be calculated with these three values, i.e., 60.16%.
The variations of the hip and knee joint torques in the experiments are shown as Fig. 14.With the iterative update of CEEC, the peak joint torques of both hip and knee joints in all scenarios are reduced significantly after several steps of walking with the increasing of the support force, that means the CEEC is an online real-time optimization approach for the HEW system.Note that for different subjects and different initial settings of the support force, the joint torques can be online optimized, and for subjects with different masses, the required joint torques are different.For example, in Fig. 14, the red solid curve with lighter subject mass are smaller than the black dashed curve with heavier subject mass.Over all, the experimental results indicate the CEEC approach proposed in this paper can realize the coordinated energy-efficient walking assistance for the human-exoskeleton-walker system, and achieved a significant improvement.

CONCLUSIONS AND FUTURE WORK
In this paper, a coordinated energy-efficient control approach is proposed for the human-exoskeleton-walker system, which is based on the discrete-time extremum seeking control strategy and the coordinated motion planning strategy.The proposed approach was able to real-time automatically tune the support force and adaptive to different subjects, and drive the robotic walker to follow the movement of the COM.Optimal support force and coordinated joint angles can be generated with the proposed approach for the robotic walker to assist the human-exoskeleton systems to implement a high energy-efficient walking.In the future, the efficiency of the method should be tested in more different scenarios, such as with different gait patterns, and applied to the real HEW systems.

Figure 1 .
Figure 1.The schematic of the human-exoskeleton-walker system.(a) A real exoskeleton robot with a robotic walker; (b) The support joint and the wheels of the robotic walker shall be controlled to follow the walking of the human-exoskeleton system.

Figure 2 .
Figure 2. The framework of the proposed approach CEEC.

Figure 3 .
Figure 3.The schematic diagram of HEW system.

Figure 4 .
Figure 4.The block diagram of ESC.

Figure 5 .Figure 6 .
Figure 5.The simulation model of the HEW system.

Figure 7 .
Figure 7.The joint angles of the exoskeleton's hip and knee joints.

Table 2 . 2 𝑟
Other parameters for simulation and energy cost computation.reduction ratio of exoskeleton reducers 60   torque constant of exoskeleton motors 0.162 Nm/A   motor resistance of exoskeleton motors 0.23 Ω   the radius of the wheels 0.038 m

Figure 8 .
Figure 8.The comparison of walking experiments with different control strategies.

Figure 9 .Figure 10 .
Figure 9.The comparison of walking experiments for different subjects.

Figure 11 .
Figure 11.The comparison of energy cost with different control strategies.

Figure 12 .Figure 13 .
Figure 12.The variation of the support force during walking.
The hip torques for initial suppor force of 300N.The hip torques for initial suppor force of 400N.The knee torques for initial suppor force of 300N.
The knee torques for initial suppor force of 400N.

Figure 14 .
Figure 14.The variation of the hip and knee joint torques during walking for different subjects.