Visual-based Parameterized Proximal Policy Optimization

• Main Authors: Huang, Ming-Xu, 黃明旭
• Other Authors: Wu, I-Chen
• Format: Others
• Language: en_US
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/c2d58n

碩士 === 國立交通大學 === 資訊科學與工程研究所 === 107 === We proposes a visual-based proximal policy optimization in parameterized (structured) action spaces based on the actor critic network. The optimization, named parameterized proximal policy optimization (P3O), is applied to RoboCup soccer simulation, robotic arm grasping and pushing. Experiments show that our method, P3O, converges fast and well. In RoboCup soccer task, P3O only needs 2/3 iterations of previous works to reach the same goal rate. Moreover, P3O can learn more stable policies in the end of training and have better policies, which means policies learned by P3O can reach goal with less steps. In agerage, policies from P3O can kick balls to the goal about 80 steps, and over 115 steps of previous works. For robotic arm grasping and pushing, we demonstrate that P3Os have capabilities to learn policies from high-dimension image observations, and successfully complete tasks, namely with success rates of 100% for pushing and 99% for grasping in about 30,000 updates. To our knowledge, our approach is the first visual-based deep reinforcement learning approach in parameterized action spaces.