A deep reinforcement learning algorithm to control a two-wheeled scooter with a humanoid robot

Jacky Baltes, Guilherme Christmann, Saeed Saeedvand*


研究成果: 雜誌貢獻期刊論文同行評審

1 引文 斯高帕斯(Scopus)


Balancing a two-wheeled scooter is considered a challenging task for robots, as it is a non-linear control problem in a highly dynamic environment. The rapid pace of development of deep reinforcement learning has enabled robots to perform complex control tasks. In this paper, a deep reinforcement learning algorithm is proposed to learn the steering control of the scooter for balancing and patch tracking using an unmodified humanoid robot. Two control strategies are developed, analyzed, and compared: a classical Proportional–Integral–Derivative (PID) controller and a Deep Reinforcement Learning (DRL) controller based on Proximal Policy Optimization (PPO) algorithm. The ability of the robot to balance the scooter using both approaches is extensively evaluated. Challenging control scenarios are tested at low scooter speeds, including 2.5, 5, and 10 km/h. Steering velocities are also varied, including 10, 20, and 40 rad/s. The evaluations include upright balance without disturbances, upright balance under disturbances, tracking sinusoidal path, and path tracking. A 3D model of the humanoid robot and scooter system is developed, which is simulated in a state-of-the-art GPU-based simulation environment as a training and test bed (NVidia's Isaac Gym). Despite the fact that the PID controller successfully balances the robot, better final results are achieved with the proposed DRL. The results indicate a 52% improvement on average in different speeds with better performance in path tracking control. Controller command evaluation on the real robot and scooter indicates the robot's complete capability to realize steering control velocities.

期刊Engineering Applications of Artificial Intelligence
出版狀態已發佈 - 2023 11月

ASJC Scopus subject areas

  • 控制與系統工程
  • 人工智慧
  • 電氣與電子工程


深入研究「A deep reinforcement learning algorithm to control a two-wheeled scooter with a humanoid robot」主題。共同形成了獨特的指紋。