Incremental learning of integrated semiotics based on linguistic and behavioral symbols

Abstract

This paper describes an novel approach towards linguistic processing for robots through integration of a motion language module and a natural language module. The motion language module represents association between symbolized motion patterns and words. The natural language module models sentences. The motion language module and the natural language module are graphically integrated. The integration allows robots not only to interpret observed motion as a sentence but also to generate motion with a sentence. This paper proposes incremental learning algorithm of association between symbolized motion patterns and words. The incremental learning is required for robot to autonomously develop the linguistic skill. The algorithm can be derived from optimization of the motion language module under stochastic constraints such that the associative probability of a new training pair composed of symbolized motion pattern and sentence becomes larger. Test of interpreting observed motion as sentences demonstrates the validity of the proposed incremental learning algorithm.