June 4-10, 2003

The Piano in the Computer

Aleksandr Moskaliuk

The use of computer technology in music is not limited only to digitizing sound or creating new compositions. For example, the personal computer can be used as an accompanist in playing classical works.

Rafael, a mathematician from the State University of Massachusetts, was always concerned with the problem of the "live quality" of music produced on a computer. Of course, digitizing and reproducing any classical composition in the MP3 format is not a problem today, but if one listens to the same melody over and over, it will sound exactly the same every time. However, the audiences at classical concerts and amateurs of this art form know very well how much live music depends on the mood of the musicians, the degree of "team work" of the orchestra, the type of auditorium and a variety of other factors.

Precisely for this reason Rafael (classical music is his hobby) decided to elaborate a program for computer accompaniment. In practice it works like this: a soloist (in this case Christopher Rafael himself) plays the oboe accompanied by a program produced on a piano. The selected composition can be a very simple collection of notes, as well as something more complicated, like Rimsky-Korsakov’s "Flight of the Bumblebee".

How does all of this work? At the basis of Rafael’s program are three "regimes": Listen, Anticipate, and Play. Before the utility can accompany independently, it has to listen to the soloist. With the help of a microphone, the sound of the oboe is digitized, and then the user must introduce his part into the computer. After this begins a series of trials.

Rafael affirms that when musicians in symphonic orchestra rehearse, they do it not because of their lack of mastery, but in order to synchronize their parts and for the possibility of quickening the tempo in a dynamic fashion so that the audience doesn’t notice accidental slip ups during a concert. When a musician rehearses with a computer, Rafael’s pack "Music++" attempts to keep the same tempo as the musician. As a result, the melody is livelier and more rhythmical.

Here an algorithm of hidden Markov models is used, the specifics of which were published at the end of the 1960s. These models became widespread and popular in scientific circles just recently primarily due to excellent results in speech recognition.

The application of hidden Markov models in speech recognition

During speech recognition, hidden Markov models can assign the sounds in question a probability coefficient, thus "guessing" possible further variants. If in this situation, the "PO" for processing the entering signal got mixed up and cannot exactly recognize the phonemes, then the probability model will make its own conclusions based on previous data. In the case of music, the situation is somewhat simplified, since the proposed conclusions must basically maintain the pitch and duration of the sound, and here the human ear forgives small sins.

After several rehearsals, the system, as is stated, "feels" the musician and automatically adjusts itself to the tempo and pitch of the music being played. Now the soloist’s part, as previously, becomes accessible to the computer with the help of a built-in microphone and all calculations are carried out in real time. Rafael uses a Linux based server as his platform.

Who needs this? As Rafael affirms, he very often collides with the opinions of "Luddites", who consider that classical music is the prerogative of the human race and not of computers. They say that it is not worth giving the "life" that has remained in music over to computers and programming applications. The author of "Music++" has a skeptical attitude toward such opinions. Recalling his own musical experience, Rafael affirms that on the contrary, electronic accompaniment engenders interest in music among those who dream of playing in an orchestra but who have to practice alone.

Examples of Christopher Rafael playing the oboe the Linux server playing the piano can be found on his site.