Design and Implementation of Reinforce Learning Based Fuzzy Gait Controller for Humanoid Robot

• Main Authors: Shao-wei Lai, 賴劭韋
• Other Authors: Tzuu-hseng Li
• Format: Others
• Language: en_US
• Published: 2009
• Online Access: http://ndltd.ncl.edu.tw/handle/73722576689767430478

碩士 === 國立成功大學 === 電機工程學系碩博士班 === 97 === This thesis mainly proposes the implementation of gait learning control and the fuzzy based gait synthesis system for a small-sized humanoid robot. We accomplish the whole system on a biped robot named aiRobot-3. The machine learning approach we applied is policy gradient reinforcement learning (PGRL) which can execute the real-time performance and directly adjust the policy without calculating action value function. Given a parameterized walking motion designed for our robot, PGRL algorithm automatically searches the set of possible parameters and finds the faster possible walking motion. The reward function we mainly considered is the velocity of our robot which can be estimated from the vision system on itself. However, our experiment illustrates that there are some stability problems in the learning process. In order to solve these problems, we also attempt to employ the desired Zero Moment Position (ZMP) trajectory as another reward for the reward function. The results show that the robot learned its gait from 30.6 mm/s to 130.6 mm/s in about 1.3 hours. It is faster than manual tuning parameters that we used before. Besides, for some advanced performance of our robot, we also apply fuzzy logic controller (FLC) in our strategy system. We use the information of its vision system as the input of the FLC and integrate the robot’s gait to perform such the tracking tasks. To acquire the motion that mapping to the output value, we employ Lagrange polynomial interpolation to transform the existing motions to the motion we want. Finally, we implement these fuzzy based gait synthesis strategies to the tasks such as chasing a ball and tracing a line for FIRA and Robocup competitions.