Learning in Non-stationary Conditions: A Control Theoretic Approach

Abstract

Robots operating in dynamic environments must adapt their control strategy to each new situation presented; success is determined by the robot's ability in identifying transitions between distinct situations or contexts. This paper proposes a novel approach for partitioning the agent's experience into a set of discrete states. The state representation proposed is based on empirically-derived models describing the system dynamics induced by a set of control primitives. These generative models allow the agent to consider all possible outcomes and select the control primitive with highest utility in each context. The representation proposed is applied to the multi-fingered grasping problem, in which the agent must grasp objects with unknown geometries, independent of the initial relative orientation between hand and object. The results obtained show that indeed the representation proposed allows the robot to adapt its control strategy to each object (or context), even...