Long Short-Term Memory Projection Recurrent Neural Network Architectures for Piano’s Continuous Note Recognition

Abstract

Long Short-Term Memory (LSTM) is a kind of Recurrent Neural Networks (RNN) relating to time series, which has achieved good performance in speech recogniton and image recognition. Long Short-Term Memory Projection (LSTMP) is a variant of LSTM to further optimize speed and performance of LSTM by adding a projection layer. As LSTM and LSTMP have performed well in pattern recognition, in this paper, we combine them with Connectionist Temporal Classification (CTC) to study piano’s continuous note recognition for robotics. Based on the Beijing Forestry University music library, we conduct experiments to show recognition rates and numbers of iterations of LSTM with a single layer, LSTMP with a single layer, and Deep LSTM (DLSTM, LSTM with multilayers). As a result, the single layer LSTMP proves performing much better than the single layer LSTM in both time and the recognition rate; that is, LSTMP has fewer parameters and therefore reduces the training time, and, moreover, benefiting from the projection layer, LSTMP has better performance, too. The best recognition rate of LSTMP is <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M1"><mml:mn mathvariant="normal">99.8</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math>. As for DLSTM, the recognition rate can reach <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML" id="M2"><mml:mn mathvariant="normal">100</mml:mn><mml:mi mathvariant="normal">%</mml:mi></mml:math> because of the effectiveness of the deep structure, but compared with the single layer LSTMP, DLSTM needs more training time.