Application of DQN Learning for Delayed Output Feedback Control of a Gait-Assist Hip Exoskeleton

摘要

Robotic hip exoskeletons hold significant potential for gait assistance and rehabilitation. Assistive controllers play an important role in the successful performance of these devices. Recently, some researchers have shown the effectiveness of a delayed output feedback controller as an assistive algorithm. This controller applies some assistive torques proportional to delayed feedback from the angle difference between two legs. This paper suggests that, if the walking speed of the wearer changes, the time delay should be altered to deliver the assistance at the right moment and maintain the performance of the controller. A deep reinforcement learning algorithm, namely Deep Q Network (DQN), is used to find the optimal value for the time delay in the delayed output feedback controller, according to the walking speed. Some simulations are conducted that include walking cycles with different walking speeds. The DQN algorithm is used to find the optimal value for the time delay, such that the energy consumption is minimized for each speed. The results verify that the time-delayed values must not be fixed, but rather should be changed based on the real-time walking speed of the wearer. It is shown that by proper adjustment of the time delay, the delayed output feedback controller maintains its assistive performance while a fixed value for the time delay may even result in resistance at some speeds.