Computational Analysis of Audio Recordings of Piano Performance for Automatic Evaluation

We developed a computational evaluation method for piano performance with the goal of building a practice support system for beginners. We recorded students’ performances as audio data and applied several recent methods for audio-to-MIDI transcription based on deep neural networks to extract the pitch, onset time, and offset time of musical notes. To determine the correctness of the performance, we aligned the extracted MIDI data with the musical score using a hidden Markov model (HMM). We compared the audio-to-MIDI transcription methods and optimized the weight on different types of performance errors to conform to teacher’s assessment. Our experiments showed a strong correlation between the rate of performance errors obtained from the alignment and the evaluation by a teacher who listened to the performance. The results that indicate performance errors and tempo stability can be used in a practice support system that provides feedback to learners.