A developmental perspective on humanoid skill learning using a hierarchical SOM-based encoding

Abstract

Hand-coding is an impractical approach to developing motion repertoires for humanoid robots, requiring both task and programming expertise. Physical demonstration of skills, on the other hand, is an approach with which humans are both competent and familiar. When following a programming-by-demonstration approach, the adaptiveness of a robot can be further increased by giving it the ability to compose novel skills from skills already acquired from demonstration. We have previously presented [1] an extension to the Piaget-inspired Constructivist Learning Architecture [2], featuring a hierarchical SOM-based algorithm that encodes skills as a hierarchy of fixed-length subsequences. At the core of the extended algorithm lies a novel principle for comparing long-term memory and short-term memory, represented as connection weights and decaying node activation values, respectively. In this article, we present an in-depth analysis of how this comparison, can provide a robot control algorithm that is both state-sensitive and goal oriented. We present results from experiments using an abstract chain walk problem that includes hidden states, to demonstrate how the algorithm disambiguates states and selects actions yielding higher rewards. Furthermore, we present results from an experiment where we use programming-by-demonstration to encode and reproduce a figure-8 gesture with a Nao humanoid robot. The results show that our algorithm is capable of identifying hidden states in both real and abstract problem domains.