How does MPO Predict the Future?
A basic tenet of my approach to musical accompaniment systems is that
one must anticipate the future evolution of the music to achieve good
ensemble. This is, of course, one of the reasons musicians rehearse
together -- so they share a collective notion of the way the music will
unfold. The computer accompanist also benefits from rehearsal, in
my case, by training a probabilistical model that allows for accurate
prediction of future note onsets times. I have given many technical
talks on this subject which have essentially explained the ideas
about probabilistic modeling of music presented in my papers on
Bayesian belief networks. The interested reader is welcome to
explore these, though I take a completely different approach here.
I have prepared a movie that demonstrates the essential ideas of
how my system coordinates through prediction. In this movie one
can see a visual depiction of the audio in the form of a spectrogram,
with frequency content evolving over time. As the audio is played,
my system will occasionally detect notes in the solo part. In the
movie these are shown in green. Note that, while these note onsets
are usually detected reasonably accurately, the detections are made
with a small amount of latency. This is because the computer (and
human) must hear some of the new note before it can be recognized.
Thus the green lines appear slighly after the notes sound.
Below the top row of green lines are a collection of red lines that
describe the systems predictions of when orchestra events should be
played. My system cares only about scheduling the pending orchestra
note and will reschedule this note every time new information becomes
avaialble. Thus, if you watch the rightmost orchestra note, you will
see that its position wiggles around as more and more solo notes are detected.
Eventually, the scheduled time will finally occur, at which point the note
is "yesterday's news" and the system shifts it focus to the next orchestra
note. Thus the time that an orchestra note is actually played remains
in red on the image.
I chose the piece, the Strauss Oboe Concerto 1st movement, since it
does an especially good job of illustrating this idea. The movement
has a repeated motive consiting
of a string of sixteenth notes for the oboe ending in a point
of synchonization with the orchestra on a strong beat. If you watch
carefully, you will see the orchestra's prediction of the goal point
shifting subtley until it finally occurs. I hope this movie is illuminating.
Strauss Oboe Concerto Mvmt 1