Autonomous learning of multiple curricula with non-stationary interdependencies

Abstract

Autonomous open-ended learning is a relevant approach in machine learning and robotics, allowing artificial agents to acquire a wide repertoire of goals and motor skills without the necessity of specific assignments. Leveraging intrinsic motivations, different works have developed systems that can autonomously allocate training time amongst different goals to maximise their overall competence. However, only few works in the field of intrinsically motivated open-ended learning focus on scenarios where goals have interdependent relations, and even fewer tackle scenarios involving non-stationary interdependencies. Building on previous works, we propose a new hierarchical architecture (H-GRAIL) that selects its own goals on the basis of intrinsic motivations and treats curriculum learning of interdependent tasks as a Markov Decision Process. Moreover, we provide H-GRAIL with a novel mechanism that allows the system to self-regulate its exploratory behaviour and cope with the non-stationarity of the dependencies between goals. The system is tested in a simulated and real robotic environment with different experimental scenarios involving interdependent tasks.