The Electric Duet By Fenella Saunders

Christopher Raphael, an applied mathematician and professional oboist, found a novel solution to a problem many musicians face when they rehearse at home. If he wanted to practice an oboe solo that will be accompanied by a piano, for instance, he might play along with a recording of the piano part-but a recording is a sterile and unyielding partner. "That is the antithesis of what it's supposed to be," he says. So Raphael, who is at the University of Massachusetts at Amherst, melded his talents to create a musical computer program that can listen and adapt its accompaniment to him.

The program, called Music++, picks up the solo through the small microphone built into most laptops or desktop computers, and learns to recognize notes from a particular instrument using the same kind of processing techniques as in speech recognition software. It also has an adaptive element that allows it to learn a piece of music and the soloist's playing style. After several practice sessions, the program can make real-time decisions to change its interpretation of the accompanying part, for instance by adapting the speed and spacing, even if the soloist changes tempo on the fly or makes a mistake. "The model understands both that musical performances are highly variable, while there are certain tendencies that appear from performance to performance," Raphael says. There are other musical accompaniment programs out there, he says, but they do not learn and adapt in the same way as Music++.

Raphael's main goal for the program is to give musicians a 21st century tool to aid in their development. "Musicians spend the majority of their practice time alone," he says. "This makes it more involved and helps prepare the musician for group playing." Although he has no desire to take the human musician out of the mix, he does think that the program might also help to advance machine learning. "What's really hard for the computer is getting the expressive component of music," he says. "Here, it can learn this from actual examples."

- Posted 9/17/02


"Making the Computer 'listen' to Music." Christopher S. Raphael, Journal of the Acoustical Society of America, Vol. 111, No 5, Pt 2, May 2002. See www.acoustics.org.

"The Computer as Musical Accompanist" by Christopher Raphael can be found at www.acoustics.org/press/143rd/Raphael.html.

Hear examples of Raphael's program accompanying his oboe playing at http://fafner.math.umass.edu/music_plus_one/