Data-Efficient Reinforcement Learning for Energy Optimization Under Human Fatigue Constraints of Power-Assisted Wheelchairs
Résumé
The objective of this paper is to develop a method for assisting users to push Power-Assisted Wheelchairs (PAW) in such a way that the electrical energy consumption over a predefined distance-to-go is optimal, while at the same time bringing users to a desired fatigue level. This assistive task is formulated as an optimal control problem and solved in [17] by the model-free approach Gradient Partially Observable Markov Decision Processes. To increase the data efficiency of the model-free framework, we propose here to use Policy learning by Weighting Exploration with the Returns (PoWER) with 25 controller parameters. Moreover, we provide a new nearoptimality analysis of the finite-horizon fuzzy Q-iteration, which derives a model-based baseline solution to verify numerically the near-optimality of the presented model-free approaches. Simulation results show that the PoWER algorithm with the new parameterization converges to a near-optimal solution within 200 trials and possesses the adaptability to cope with changes of the human fatigue dynamics. Finally, 24 experimental trials are carried out on the PAW system, with fatigue feedback provided by the user via a joystick. The performance tends to increase gradually after learning. The results obtained demonstrate the effectiveness and the feasibility of PoWER in our application