Humanoid recognizing piano scores techniques

Abstract

In this paper, we present a novel technique of optical music recognition (OMR) which is distinct from the traditional methods that recognize the contents of images produced from a scanner. We apply this technique to make an entertainment robot can read the printed piano scores that humans actually employ the recognition results and play them in real time. In our robot vision system, the PTZ camera commonly used in video surveillance is adopted, whose functionalities are precisely manipulated to simulate the human behavior of reading printed piano scores in a “recognizing while looking” manner. It is realized by a module of capturing an enlarged partial image followed by executing a recognition process repeatedly until each part of a printed piano score has been read. How many times of activating this module and executing it ingeniously during the process of the reading depends on the number of measures of a printed piano score. In the experiments, we set the image resolution of 640 × 480 pixels to evaluate both the detection and recognition results from the printed piano scores of 10 songs. This module only costs 200~300 milliseconds. Moreover, our system can recognize and play the printed piano scores in real time and the recognition rate is over 87% on an average.