GenJam in Transition: from Genetic Jammer
to Generative
Jammer
John A. Biles
Information Technology Department, Rochester
Institute of Technology
e-mail: jab@it.rit.edu
Abstract
GenJam, short for Genetic Jammer, is an evolutionary computation (EC) based software agent that models a jazz improviser. Recently GenJam has evolved away from its roots as an interactive genetic algorithm toward its current state as an autonomous generative system. GenJam has retained its chromosome-based representations and mappings, its intelligent selection, crossover and mutation operators, and its real-time interactive performance capabilities. However, it no longer needs any explicit representation of fitness, which arguably makes it no longer an EC system.
This
paper considers GenJam as a generative art system. Generative art produces “unique and non-repeatable
events” that express a designer’s generating idea. The designer’s generating idea
defines a species of events, represented in a genetic code. In music, these events could be
individual notes, melodic phrases, even entire pieces. In GenJam the events are four-measure
phrases, or “licks” in the jazz vernacular.
The
format for the genetic code, then, defines a species space from which
unique individual events can be generated. Uniqueness is important in jazz because improvisation must
be spontaneous and “new.”
Hence, improvisation is tailor-made for the generative art paradigm, and
in fact, one could argue that improvisation is, by definition, the purest
example of generative art applied to music. In other words, generative music is improvisation,
and GenJam is the Generative Jammer.
Jazz improvisation is a highly spontaneous, highly interactive, highly creative activity. Jazz improvisers compose and perform simultaneously, usually accompanied by a rhythm section that typically consists of at least a bass instrument (upright or electric bass), a chording instrument (piano or guitar), and a drum set. In straight up jazz (also called mainstream, straight ahead, and several other labels) the group will usually begin a specific tune by playing a (sometimes) recognizable melody line, then improvise one or more choruses over the form of the tune, and finally play a recapitulation of the written melody to end the tune. The improvisational choruses are usually the focus of a jazz piece, and performers are judged by their ability to improvise compellingly.
Improvisers can perform in at least three modes: full-chorus solos, chase choruses, and collective improvisation. When taking a full-chorus solo, a performer improvises alone over the entire form of the tune, accompanied by the rhythm section. The emphasis is on presenting creative ideas and developing those ideas coherently to present an original melodic invention.
In a chase chorus two or more performers take turns improvising over successive portions of the tune, usually in units of four measures (trading fours) or eight measures (trading eights). The focus here is often competitive, topping what the other performer just played to show him or her up. This requires clearly hearing what the other performer plays and spontaneously finding a way to “play rings around it.”
Collective improvisation is when two or more performers improvise simultaneously. The emphasis here is often on generating spontaneous counterpoint or harmony, complementing what the other performer is doing as he or she is doing it. This also can be competitive, but the goal is usually to weave a collaborative tapestry over the form of the tune.
GenJam is an evolutionary computation (EC) based, real-time interactive computer music system that improvises in all three of these modes in collaboration with a human performer. GenJam uses a chromosome-based representation for measures and phrases of melodic content and evolves populations of these individuals from which it generates improvisations in real time. GenJam’s architecture, chromosome representations, mappings, musically meaningful genetic operators, and musical performance characteristics have been described elsewhere [1, 2]. This paper will focus on GenJam as a generative art system that applies the generative design paradigm to jazz improvisation.
The generative design paradigm focuses on the realization of “metaprojects…software that is able to generate an endless sequence of products” [3]. The designer’s generating idea defines a species of products, represented by a genetic code. Unique individual products are generated from this species space in collaboration with the designer. In most generative design applications the products are usually complete, stand-alone objects, like lamps, for example [3], and the process, then, would generate a sequence of unique lamps, rather than copies of a single mass-produced lamp model. GenJam’s “products” are phrases of melodic content, or licks in the jazz vernacular. An “endless sequence” of unique and original licks is certainly appropriate for improvisation, but there are differences between adapting the generative design paradigm to lamps and adapting it to jazz improvisation.
One major difference is that the licks played by GenJam are performed once and are then gone. The analogy to lamps would be to generate a new lamp every time someone turned on a light switch and then dispose of the lamp when the light switch is turned off. Improvisations are spontaneous products of the moment; once performed, they “dissipate into the ether” unless someone bothers to record them. This is an issue in jazz aesthetics, where major artists are often faithfully recorded at every performance because every performance is unique, will never be repeated, and (some would say) must be recorded for posterity. The problem is: who has time to listen to all the recordings? The philosophy with GenJam, on the other hand, is that improvisations are disposable; GenJam can always generate a new one if desired.
A second difference between generating lamps and generating licks is that the individual licks generated by the process are not stand-alone objects. A single lick must exist in two contexts to be meaningful: a harmonic context and a melodic context. The harmonic context is the chords being played by the rhythm section over which the lick will be played. GenJam handles this context by knowing what chords the rhythm section is playing at any given time and mapping the melodic content of measure chromosomes to specific pitches taken from scales suggested by those chords. GenJam understands 18 different chord types and also uses chromatic passing tones and other heuristics to generate melodic interest without sounding “wrong.” The harmonic context in which licks are generated, then, is fairly straightforward. The melodic context, however, is more involved and is different for the three improvisational modes.
The melodic context of a given lick is essentially the licks that precede and follow it, the lick’s place in an improvised chorus. When GenJam improvises a full-chorus solo, the surrounding licks are other individuals from the population. The constraints placed on a lick in this mode are minimal; GenJam literally selects licks to play at random. The path through the species space of licks, then, is fairly discontinuous, although since two-thirds of the licks are intelligent crossovers of the remaining one third, a melodic motive can be developed, albeit serendipitously.
When GenJam trades fours with a human in a chase chorus, its licks are surrounded by licks performed by the human. GenJam listens to the human perform a lick using a pitch-to-MIDI converter, maps what it hears to its chromosome structure, mutates the chromosomes to develop the lick, and plays the mutated lick as its response in the next four. In other words, GenJam uses its musically meaningful mutation operators to develop what the human has played into what it will play in response. The constraints placed on a GenJam lick in this mode are more pronounced. The “seed” chromosomes come from the human performer, and the modifications derive from the musically meaningful mutations, which include inversion, retrograde, transposition, and other intelligent operators. The path through the species space of licks for this mode is more continuous than for full-chorus solos because both the human and GenJam tend to play off of what the other just played.
When GenJam and the human performer improvise collectively, GenJam listens to the human while it is improvising. In essence, it plays an adaptation of what the human plays, delayed by a measure or a portion of a measure. Collective improvisation is probably the most challenging mode for humans to perform because the human must be thinking in three places at once: remembering what he played one measure ago, playing in the present, and planning what to play as a complement one measure from now. GenJam plays it safe in collective improvisation mode and only maps the pitches to the current chords without mutation. The melodic context in collective improvisation, then, is both the last measure and the next (future) measure. The path through the species space of licks in this mode tends to be even more continuous than when trading fours because GenJam is literally trying to parrot what the human just played, and the human is trying to complement that coherently. Clearly, this is a stylistic decision on the human’s part. One can use this mode to play very intense avant garde improvisations, but I tend to play more inside.
A third difference between generating lamps and generating licks is that lamps typically are generated before they are used, while licks are generated as they are used, at least when GenJam trades fours and improvises collectively. This is important because there is no chance to reject undesirable licks in these improvisational modes the way one could reject undesirable lamps. Consequently, GenJam uses no fitness when trading fours and relies on its intelligent crossover and mutation operators to always generate appealing variations. If the human plays well, GenJam will too. Even if the human plays poorly, GenJam will at least sound competent. This need for spontaneous interactivity leads naturally to the ways in which GenJam collaborates with its human partner, which in turn provides some insight into collaboration in general between generative art systems and designers.
The generative design paradigm is powerful because it can provide a stimulating and even inspiring environment in which a designer can explore a design space. The key factor in facilitating that exploration is the level of collaboration between the designer and the system. Designers need inspiration, and while that can come from within, it often comes from collaboration with others. Generative design systems can stimulate, provoke and challenge designers as they explore a design space through discovery, serendipity, and spontaneity. GenJam is a highly collaborative system that definitely provokes and challenges its human partner in performance. This section explores several forms of collaboration, using GenJam as an example.
One form of collaboration places the designer in the role of mentor [1]. When the original, interactive version of GenJam is used to train a soloist, GenJam improvises full-chorus solos while a human listens and provides simple real-time feedback on whether the improvisation sounds good or bad. The feedback is used to derive fitness values for individual measure and phrase individuals in their respective populations, which in turn provides the means to perform tournament selection and replacement while breeding new generations of the soloist. This is a traditional interactive genetic algorithm (IGA), where fitness is provided by a human because no algorithm can be devised to compute fitness values. In this type of collaboration, then, the system generates individuals, which the designer selects, eliminates or rates.
A related form of collaboration allows the designer to modify the system’s generated individuals, instead of just selecting or rating them. This is a deeper level of collaboration because the designer is directly determining or at least directly influencing the content of the individuals. The disadvantage of this form is that the designer may over-specify the modifications and lose spontaneity, or in EC terms, the designer may converge the search prematurely. With GenJam the human performer collaborates in this way when responding to a four GenJam has played. If the human response is too close to what GenJam just played, the chorus can “get stuck” on a lick that bounces between the performers until one or the other breaks the cycle with a fresh four.
In another form of collaboration the designer generates individuals and the system modifies them in interesting ways. When GenJam trades fours, it collaborates with the human performer in this way when it responds to what the human just played. The human plays a four, GenJam listens to it, mutates it and plays it back as a development of the human’s four. This is a still deeper level of collaboration because the system is creatively manipulating the human’s input.
The deepest level of collaboration is when the designer and system both manipulate each other’s individuals in a cycle. This is a true partnership and approaches human collaboration among equals. GenJam reaches this level when trading fours. I find it at least as stimulating as any human I have encountered at the over 700 jam sessions in which I’ve participated over the years, and it is a more than worthy sideman. Besides, it always shows up on time and sober, and it doesn’t rush the tempo!
The requirements that jazz improvisation be spontaneous, original and collaborative make it a natural application of the generative design paradigm. In fact, I would argue that improvisation is the purest application of generative art to the musical world. In the case of GenJam, the “designer” is actually a fellow performer, which elevates the generative system’s status to the same level as the designer because their collaboration is largely symmetric and their roles are essentially the same, at least during performance.
The conclusion, then,
is that GenJam is a successful generative art system that supports a deep level
of collaboration to perform jazz improvisation in real-time. GenJam’s original interpretation
as the Genetic Jammer is still valid, but it now also can be interpreted as the
Generative Jammer!
[1] Biles, John A., GenJam: Evolution of a Jazz Improviser. In P. J. Bentley and D. W. Corne (ed.), Creative Evolutionary Systems. San Francisco: Morgan Kaufmann, 2001.
[2] GenJam: Evolutionary Computation Gets a Gig. In Proceedings of the 2002
Conference for Information Technology Curriculum,
Rochester, New York, Society for Information Technology Education, 2002.
[3] Soddu, Celestino, Recognizability of
the Idea: The Evolutionary Process of Argenia. In P. J. Bentley and D.W. Corne (ed.), Creative
Evolutionary Systems. San Francisco: Morgan Kaufmann, 2001.