Playing with the masters: A model for improvisatory musical interaction between robots and humans

Abstract

We present our approach for facilitating musical interaction between robotic musicians and humans (musicians as well as non-musicians) in an improvisatory and exploratory manner. By using the iPhone as a musical instrument, we offer a quick and effective way for anyone to start creating music and communicating directly with our marimba playing robot, Shimon. Through automated style analysis of jazz masters, our system creates a novel improvisatory and interactive musical language. In a call and response exchange, the robot expands on short tunes created by human performers, in a musically rich and meaningful manner. As part of the interaction, users can also controlling the musical influences that combine to make up Shimon's improvisatory language, creating novel and inspiring responses.