Cooperative Strategy Based on Adaptive Q-Learning for Robot Soccer Systems

• Main Authors: Chien-Cheng Chen, 陳建成
• Other Authors: Kao-Shing Hwang
• Format: Others
• Language: zh-TW
• Published: 2003
• Online Access: http://ndltd.ncl.edu.tw/handle/28197903281272369009

碩士 === 國立中正大學 === 電機工程研究所 === 91 === The object of this thesis is to develop a self-learning cooperative strategy for robot soccer systems. The strategy enables robots to cooperate and coordinate with each other to achieve the objectives of offense and defense. Through the mechanism of learning, the robots can learn from experiences in either successes or failures, and utilize these experiences to improve the performance gradually. The cooperative strategy is built a hierarchical architecture. The first layer of the structure is responsible for assigning each role, that is, how many defenders and sidekicks should be played according to the positional states. The second layer is for the role assignment related to the decision from the previous layer. We develop two algorithms for assignment of the roles, the attacker, the defenders, and the sidekicks. The last layer is the behavior layer which robots executing their behaviors commands and tasks by their roles. The attacker is responsible for chasing the ball and attacking. The sidekicks are responsible for finding good positions, and the defenders are responsible for defending competitor scoring. The robots’ roles are not fixed. They can dynamically exchange their roles with each other. In the aspect of learning, we develop an Adaptive Q-Learning method which is modified form the traditional Q-Learning. By some experiments, it is more effective than the traditional Q-Learning observed in a simple ant experiment, and it is also applied to the learning of the cooperative strategy successfully.