Mutually constrained multimodal mapping for simultaneous development: Modeling vocal imitation and lexicon acquisition

Abstract

This paper presents a method of simultaneous development of vocal imitation and lexicon acquisition with a mutually constrained multimodal mapping. A caregiver is basically assumed to give matched pairs for mappings, for example by imitating the learner's voice or labelling an object that it is looking at. However, the tendency cannot be always expected to be reliable. Subjective consistency is introduced to judge whether to believe the observed experiences (external input) as reliable signal for learning. It estimates the value of one layer by combining the values from other layers and external input. Based on the proposed method, a simulated infant robot learns mappings among the representations of its caregiver's phonemes, those of its own phonemes, and those of objects. The proposed mechanism enables correct mappings even when caregivers do not always give correct examples, as real caregivers do not for their infants.