How to practice playing the piano; The deep learning way

Last year, I was interested in deep learning, so thought that I could apply methods and metrics from deep learning to playing the piano.

I applied deep learning to practicing the piece, Chopin Nocturne Op 9-2, which was one of my favorites. Always wanted to play it but could not 'because I thought it was too hard to play' for more than twenty years.

So I devised my ways to play the piece by using deep learning approaches and learning from a few youtube videos teaching you how to practice a piece effectively.

Still making many mistakes, but here is the result of my practice:

The one played at a fast tempo (with notes followed):

Here are some tips and tricks to practice the piano effectively and efficiently using deep learning approaches.

Practice only one hand very slowly.

Build a mixture of experts.

You need to build at least three big experts in your brain. The first is the left-hand expert, the second is the right-hand expert, and the last is the pedal expert. When you practice with two hands at the same time, your brain takes longer to train each of these two experts. Train each of these two experts separately. Also, you need to practice your pedal play separately. If you try to train any of them simultaneously in the underfitting zone, which requires more practice, it will take more time for you to train the three experts.

Backpropagation takes time.

The process of adjusting weight values between your neurons, backpropagation takes time. It just takes time. So you need to give enough time for each note you play. Listening to what you play, focusing on what you see in your score all helps to adjust weight values in your neurons. But the weight values are adjusted effectively when you play slowly. Your brain needs time to store the information you perceive.

Repeat only one measure until your hand can automatically play it.

Leverage a mixture of experts.

You need to build each expert for each measure. If there are 50 measures, you are training 50 experts in your brain. When you play the piece, you will use each of these experts by gating them.

Gate each expert.

When you play the piece from the first measure to the last measure, you are gating each expert for each measure.

Do curriculum learning.

Master one measure and then master another measure. In case you have 50 measures in your piece, you have 50 curriculums to learn. A measure is a curriculum. You can also make two or four consecutive measures as a group, which is another type of curriculum. Try to make a hierarchy of curriculums for your piece by grouping measures like you learn algebra, arithmetic, and geometry to master mathematics.

Do transfer learning

In your piece, you may find measures with an exactly same sequence of notes. Also, there are measures just a little bit different from one in your piece. Figure out the difference and memorize the difference only. You can reduce the capacity required to play the piece by minimizing the number of experts(=the number of kinds of measures to play) you need to train.

Practice each measure from the last measure to the first measure.

Do not depend on recurrent neural network

Your brain has a recurrent neural network, which tries to fetch next note based on the current note and to fetch next measure based on the current measure. But this great function becomes a hurdle when you practice the piece. Even though you do not focus on each note/measure when you play, your hand plays each of them automatically.

This is great, but playing each measure from the last to the first helps each of your experts for each measure to not depend on the recurrent neural network you have. It means, your experts will have more expertise for each measure.

Provide an even opportunity for each measure to be learned

When you play a piece you usually play from the start to the end. Usually, at the beginning, you focus on the piece but as you reach the end, you start to lose focus. To give the same amount of opportunity for each measure you practice, you also need to practice from the last measure to the first measure. That way, you start with focused at the last measure and then you slowly lose your focus when you reach the first measure.

It is important to give even 'focus' opportunities for all experts to be trained and practicing from the last measure to the first measure helps a lot.

Do not continue to play when you made a mistake.

Avoid training with examples(=scores) that have wrong labels(=wrong notes, mistakes)

Basically, we are doing supervised learning. If you continue to play even though you made a mistake, your brain learns the mistake. You need to stop and play the measure from the start over and over until you do not mistake.

Continuing to play the piece even though you made mistake is like playing with a score that has wrong notes. You need to stop when you made a mistake, and correct your mistake.

Stop recurrent neural network to build sequences of notes with mistakes mixed.

Your brain starts to build sequences of notes based on your mistake if you don't stop when you made a mistake. Before mistakes are built into the sequence of notes, you need to stop and start practicing from the start of a measure.

Listen to different pianists playing the piece you are practicing.

Leverage generative adversarial network w/ reinforcement learning.

Try to listen to great pianists. Try to memorize how they play. When you practice if you make a similar sound that you heard, give positive feedbacks to you. Such as saying 'nice!', or some delicious cookie.

Reduce generalization error by listening to plays from different pianists.

By listening to many players, you are not overfitting to a specific player, but learning common playing technique from many players.

Listen/practice many times ( several hundred times ;-) )

Practice as many epochs as possible.

Deep learning requires training with many epochs. An epoch consists of all training examples to learn. One epoch in our case is playing the piece once. A measure is an example. Repeat as many epochs as possible.

Listen to great pianists as many times as possible.

You need a great judge for your generative adversarial network. To train the judge, you need to listen to plays of great pianists a few hundred times. Focus on rubato. Different players have different styles when they play with rubato. Also, focus on articulations. Learn from them.

When you make mistakes, just practice more. Do not give negative feedbacks to you. Making mistakes while practicing is natural.

Just think your learning phase is in the underfitting zone.

In the underfitting zone, it is natural to make mistakes. The mistakes should not become negative feedbacks. If so, playing the piano will become stressful. Just take mistakes as natural.

Memorize the piece with (1) name of each note, (2) your hand position, (3) pitch of each note, (4) chords in each measure.

Use hierarchical mixture of experts

Within each expert of each measure, you can build small and many experts in it. I call these micro-experts. These small experts can output the note based on different inputs such as name/pitch of the note, chords of multiple notes, and your hand position.

Repeat the measures you make mistakes over and over.

Train a specific expert that is not trained enough.

If you make mistakes in a specific measure, it means either the expert is in the underfitting zone requiring more training, or the expert learned mistakes. You need to practice more to unlearn mistakes. You need to practice more to get out of the underfitting zone.

Practice measures in random order.

Shuffle examples(=measures) in your epoch(=piece).

This makes your experts stronger. In deep learning, when you train an epoch, it is crucial to shuffle examples in it so that the model does not have any bias on the order of examples in the epoch. Likewise, playing measures in random order helps. Just start playing any measure comes up from your mind. You can also memorize the measures you made mistake while you play, and play them in random order.

Do not press keys on your keyboard but just touch them without making any sound.

Train the hand position micro expert

This trains your specific model about the movement of your hands. By not listening to the sound at all, you can focus on the position and movement of your hands. See if your hands are prepared for each note. Your hands should be already on top of the keys you need to press, several hundred milliseconds before you press them. That way you can also control the power of touching keys to make a better sound.

Train your model once more before playing in front of your friends.

When you play in front of friends, you have a different environment. Different piano, many people around you, etc. Your brain needs time to adapt to the new environment. Unlike machines, your brain is unable to find the starting point to the trained model you when you feel nervous because of the new environment. Ask your friends to wait a while, and practice on the piano without making any sound. You will now have better confidence, as you trained each of your experts in your model just now.

Believe in yourself. 'Divide and conquer' approach will finally make you do what you want.

We have divided the activity of practicing complicated piece into small and easy tasks. We divided the piece by left hand and right hand, We divided the piece by each measure. For each measure, we divided each note by name, pitch, hand position, and chord.

No matter how hard a piece is, you can master it by dividing the piece into small and easy tasks and merging them all together.