The amplitude and frequency functions for "Danny Boy"
Over the past couple of decades there has been a good deal on the synthesis of expressive piano performance,
as in the RENCON competition. My related work instead looks at the problem of
synthesizing expressive audio for melody, which finds its richest form with "continuosly
controlled" instruments.
Here the musical performance is represented in terms
of two time-varying functions, one for pitch and one for intensity. This allows the
performance to capture many different aspects of expression, such as timing, within-
and between-note dynamics, vibrato, glissando, pitch foreshadowing, and some degree of articulation.
The basic emphaisis here is on capturing prosodic (stress and grouping) aspects of the
melody. To do this I have developed a set of labels, assigned to each melody note, that describe the
function the note plays in a larger context, including notions of forward motion, arrival,
and receeding motion. This melody markup is the way in the the prosodic interpretation
is represented.
One can then look at the expressive rendering problem
as one of estimation, where one seeks to estimate the labels for each melody note.
A "hardwired" algorithm constructs the pitch and intensity function from the prosodic
labeling.
Numerous examples of synthesized melodies, from hand-labeled prosody, computer estimated prosody, and random
prosody are shown here
