Why Mimi?
Imagine this: you stand, center stage,
in an opera house filled with admiring fans.
Your voice soars over the the shimmering textures of the
strings and intertwines with solo lines in the winds, borne
up by the quiet strength of the brass.
The orchestra is led
by a sensitive conductor who follows, even anticipates, your most
subtle nuance and gesture.
The goal of my current research project is to package this fantasy and deliver
it to your doorstep in a box. Everything except the fans, that is.
The box requires Linux, of course.
In this musical example I have taken the beloved aria "Mi Chiamano Mimi"
from Puccini's La Boheme. I begin with an audio recording of the orchestral
accompaniment, courtesy of the folks at Music Minus One and the acclaimed
Festival Symphony Orchestra of Bulgaria. I then use
a score matching algorithm to find the alignment between the score and
the audio file. A technique known as "phase vocoding" allows me to play
the audio file back at variable rate without introducing pitch distortions.
The main challenge is controlling the time varying audio play rate throughout
the evolution of the performance. A mathematical model combines the
real-time analysis of the soloist's performance with examples of past
performances to produce a sequence of local
goals for the accompaniment, much like a trail of bread crumbs. The
phase vocoder adjusts the play rate to follow this trail.
Here is the performance. Hope you like it.
"Mi Chiamano Mimi" from Puccini's La Boheme
More musical examples of my accompaniment research
More involved discussions of the science behind
this work