I'm looking to add some machine learning to my Max/MSP (www.cycling74.com) project. I'd like to have it written in Tensorflow and communicate back and forth with Max via TCP (not over the Internet, just on the local machine). Therefore I will need a TCP sender and receiver as part of the script.
The neural net I would like written will probably be a relatively straightforward LSTM model built to predict the likelihood of the next moment in an incoming stream of audio. In other words, given the audio at any time-step, how well does it compare with what the model expected to see then, based on its training? This will probably require a custom loss function measuring how closely the model's output matches the true training data as it progresses.
I was contemplating "warm starting" (https://determined.ai/blog/warm-stating/) each training session because the way my project works I want to train and periodically retrain in real-time, so training speed will be of the essence.
I'll need a script which has the following functions:
- Receive a continuous stream of training data to be compiled and later used to train the neural net. This training data will most likely be bark coefficients of audio, e.g. lists of 24 floating-point numbers arriving roughly every 10-20 ms.
- When the intermittent command arrives, train the net on all of the training data that's arrived by that point. The training should take place on the GPU. The function should return a message signaling when the training is complete.
- Run the model on a continuous stream of new data (again, bark coefficients), continuously outputting a number (0-1) denoting how predictable it was. So a value of 1 would mean the net knew exactly what it was about to see and a value of 0 that it was completely caught off guard by something unpredictable.
- When designated, save the model to disk for later use.
- When designated, load a previous model to begin working with it again.
I've dabbled with machine learning before, so I know a decent amount about what I'm asking here, but I'm certainly no expert. We can absolutely discuss if there are better/easier ways to go about achieving my goals than what I've outlined. If you see something, don't hesitate to bring it up!