The Singing Stream (MIDI generator)

I would like to divide this job up into multiple milestones. After each milestone is complete there should be a functioning product and the pay for that milestone will be given out.

Milestone 1

The core of this job will be creating a system for turning streams of data into MIDI notes. The system will be generic enough that it could potentially take any sort of datastream as input, but for Milestone 1 this data will simply be the X and Y coordinates of the mouse. In addition to getting this mouse data streaming in, there will essentially be 2 aspects to this task.

1. Store high and low values for the streams in a dictionary. This will be done to calibrate the stream and provide a target zone that will trigger MIDI notes.
2. Compare new incoming data to this calibrated target range. When appropriate, create MIDI notes and send them from a custom named MIDI device that appears like any other on the system.

1. Calibration numbers will need to be stored in some format, probably a dictionary.

• The hierarchy of the dictionary should go [device (e.g. Controller 1)][pitch (0-11, denoting C-B in terms of lettered pitches)][control stream (e.g. Joystick 1 X-Axis)][low calibration value, high calibration value]
• The dictionary should be empty at first. All of the calibration values and keys should enter the dictionary only while a hotkey combination is held down.
  • The hotkey combinations will be CTRL + <1 of 12 letter keys>, each denoting one of the 12 pitches in an octave.
  • The following image shows the layout of these keys and what pitch they will correspond to. So for example, CTRL + A will calibrate a pitch of C (denoted by 0 in our system) and CTRL + T will calibrate a pitch of F# (or 6 in our system). The layout of these keys are chosen because they resemble the white and black keys on a piano.
  • While a hotkey combination is held down all controls streaming from all sources should compare each of their values received to the low and high calibration values stored under that note in the dictionary, replacing any that are higher or lower. This will keep track of the lowest and highest values received during that calibration.
  • So for example, if the only controls streaming data while CTRL + H are held down are the X and Y axis of Joystick 1 on Controller 1 the entries in the dictionary after calibration might look like "Controller 1":9:"Joystick 1 X-Axis":[-312, 262] and "Controller 1":9:"Joystick 1 Y-Axis":[-281, 141].
• Again, for Milestone 1 the only device will be mouse and control streams will be horizontal position and vertical position.
• This calibration data will need to be automatically saved in some sort of database or other form on the computer and automatically reloaded each time the program is run.

2. While not calibrating, all incoming values in the stream will need to be compared to the calibrated target range for that control. MIDI note-on messages will be created when the values enter the range and note-offs created when they leave.

• For each control stream, values will need to be continually stored temporarily in a couple of variables, one for the current value and one for the prior value.
• For each pitch key 0-11 that is present under the device in question and each control key present under that pitch key, the low and high calibration values stored there will need to be compared to that control's current value. If the current is between the calibration values, a MIDI note at that pitch should be "on".
  • If the all the prior values for the controls under the pitch key in question were not within the calibration range, but the current ones are, a MIDI note-on message should be created.
    • The pitch of this created MIDI note will be based on the key (0-11) that was being checked. For Milestone 1 there will be only one octave for the sake of simplicity. So we can say that the number 0-11 can simply always be added to 60 to get the actual MIDI note number. So if the key being checked was 5, the MIDI note pitch produced would be 65.
    • The velocity of the created MIDI note should be based on how quickly the control stream(s) moved into the target range from outside of it, the faster the stream moved into the target range, the louder the note will be. To calculate this, you can simply take abs(current - prior) for each stream.
      • These numbers will need to be normalized to the standard 0-127 MIDI range to get the end velocity. To do this, you can compare the difference calculated in the prior step to the full size of the target range for each control stream. So for example if abs(current - prior) equals 400 and the calibrated target range is [-312, 262] (a full size of 574), the ratio between the two would be 400/574 = 0.69686411149. If there are multiple control streams, this ratio should be calculated for each and then they should all be averaged. This averaged ratio can then simply be multiplied by 127, i.e. 0.69686411149*127 = ~89 as a final MIDI velocity.
      • Obviously a MIDI velocity can never be greater than 127, so if the ratio is greater than 1 the velocity should simply be 127.
    • For now, all notes can be on channel 1.
  • If the all the prior values for the controls under the pitch key in question were within the calibration range, but not all of the current are, a MIDI note-off message should be created for that pitch.
• The MIDI notes should come from a named MIDI device that appears like any other on the computer and this name should be the device name used for the topmost key in the dictionary hierarchy, e.g. mouse.

3. Set up the X and Y coordinates of the mouse as the initial default streams for the MIDI generator, to illustrate that it's working.

---

All code will need to be clean, organized and very well-commented. Please contact me if you have any questions or would like clarification about anything! If you see a better way to do something than what I'm suggesting, please bring it up!