Market-Driven Multi-Agent Collaboration in Robot Soccer Domain

Abstract

In this work, the target is the coordination problem among the members of a robot soccer team. In order to solve this problem several methods which are extensions of a marketdriven approach are implemented. In this work these approaches are studied and compared in detail. The first developed method was the method with static role assignment. Since it has many drawbacks, a novel market-driven approach was implemented to increase the team success by using the full benefits of collaboration. In this first version, roles are fixed, and the agents are assigned suitable roles according to the available cost functions to increase success, in the current situation. This strategy was quite successful and takes good results in during the matches done by other teams, but there are different teams with different game strategies like in the real life case, so there is a need to change the game strategy (e.g. playing offensive or defensive) according to the opponent team strategy. So the original MarketTeam is extended by the addition of reinforcement-based learning method, which allows the team to learn new strategies, as it plays matches with other teams, and use a dynamic strategy to choose the roles for the players. Later this strategy which uses marketbased cost values and other domain specific values in its state vector is further extended to eliminate the drawbacks, and increase success. The results show that reinforcement learning is a good solution for role assignment problem in the robot soccer domain. However, encoding of the problem into the learner is an important issue. When the configuration space is quite large, the policy may not cover all possible states. As a result, the agent is forced to select random actions and the system performance decreases. The communication problem is not addressed in this work. It is assumed that each agent can broadcast limited amount of data. The controller simply collects available data from any other agent. The data may be noisy. Since, at each frame the communication data is refreshed, the error is not cumulative. The solution can also be used in other highly dynamic environments where it is possible to introduce some reinforcement measures for the team. In the robot soccer domain, the reinforcement measures are the goals scored by either our team or the opponent team.