Adapting to Subsequent Changes of Environment by Learning Policy Preconditions

摘要

This paper describes a method which senses a changing environment by collecting failed instances, uses concept learning for acquiring a precondition for a control policy, and partially modifies the policy for adapting to subsequent changes of the environment by reinforcement learning. A precondition for a policy represents a condition to reach a goal using the policy. Our method learns a precondition for a policy from instances of the policy's success or failure by concept learning. Using concept learning, our method has the ability to improve its behavior in states not experienced by the robot. We experimented using our method on a profit-sharing reinforcement learning system, and a decision tree learning system, C4.5. It adapted to a changing environment faster than the relearning methods. In addition, we have confirmed that concept learning provides a method that adapts effectively to a changing environment.