Improvising Sound Art

By Jesse Stewart

Over the past half-century, there has been a strong connection between musical improvisation and the visual arts in Canada. Numerous creative practitioners in this country including Michael Snow, Nobua Kubota, John Heward, and John Oswald (to name only a few), have had successful parallel careers as both visual artists and improvising musicians. Groups such as the Artist’s Jazz Band (which included artists Graham Coughtry, Nobua Kubota, Dennis Burton, and Gordon Rayner); CCMC (the current incarnation of which includes creative polymaths Michael Snow and John Oswald); and the Nihilist Spasm Band (which included painter Greg Curnoe until his untimely death in 1992 and continues to feature sculptor/instrument builder Murray Favro), played a significant role in the development of musical improvisation in Canada and, more specifically, in the transition from jazz-based improvisation to “free” improvisation.

In recent decades, improvisation has come to play an important role within Canadian “sound art,” an interdisciplinary genre that typically focuses on sound within institutional frameworks associated with the contemporary visual art world. At some level, the act of engaging with any work of art is an improvisatory one: the eye improvises as it looks over the surface of a painting, lingering in some areas of the canvas longer than others.  When we experience a piece of sound art, both the eye and ear improvise as we choose what to look at and listen to, as well as how we look and listen.

Take, for example, Janet Cardiff’s and George Bures Miller’s landmark 2001 work Forty Part Motet (on display at the National Gallery of Canada from December 17, 2011—March 25, 2012), which reworks “Spem in Alium,” Thomas Tallis’ 1573 motet, written for eight choirs of five voices each. In Cardiff and Bures’ work, the forty parts of Tallis’ piece are individually recorded and played back on a loop through forty speakers placed throughout the exhibition space. Unlike most live performances of Tallis’ work, Forty Part Motet affords listeners/viewers the opportunity to move throughout the space, changing their position in relation to the eight groups of speakers, focusing on different performance details, and determining for themselves the duration that they spend with the piece. In short, visitors to the gallery are able to improvise, making decisions in the moment that affect their experience of the work.

Dipna Horra’s 2011 installation Dhunia: Septet (on display at A Space in Toronto from March 8 to March 31, 2012) offers visitors a similar set of improvisatory opportunities.  The piece transforms seven old storm windows into speakers that resonate with recordings of various ambient sounds drawn from the urban and natural environments as well as a recording of her grandmother telling a parable in Punjabi. By turning window frames—objects that we normally look through—into objects that we listen to (and through) Horra allows us to improvise our own audio-visual mix, constructing our own narratives in relation to those presented in her work.

Several examples of Canadian sound art encourage an even greater degree of audience interaction and improvisation. Consider Ottawa artist Donna Legault’s recent audio-visual installation titled Cymatic Imprints, on display from January 20, 2012 to March 4, 2012 at the Ottawa City Hall Art Gallery. In this piece, suspended microphones capture the sounds of visitors to the exhibition. A computer program transposes the pitch of the sounds to sub-audible levels and plays them back instantaneously through a series of suspended speakers. Long, thin chains connect the speakers to the floor. The movement of the speakers causes the chains to dance, creating erratic drawings in spots of white sand on the floor. By creating a biofeedback loop between the sounds of gallery patrons and the speaker/chain assemblages, the piece encourages visitors to improvise within the space, turning sound into image.

Governor General’s award-winning artist David Rokeby’s classic work, Very Nervous System (1986-2004), creates a similar biofeedback loop between body and machine. First developed in the mid 1980s, when sound art, interactive art, and computer-based art were each in their infancy, Very Nervous System uses video surveillance technology, synthesizers, a sound system, computers, and image-processing software designed by Rokeby to translate movement into music and/or sound. The movements of gallery patrons are monitored by an inconspicuously placed wall or ceiling-mounted video camera. The video signal is routed to a hidden computer where the image processing software developed by Rokeby divides each frame into an 8×10 grid. A different audio file is linked to each of the grid’s eighty cells. The software compares each video frame to the previous one. When it detects even a slight difference between two adjacent frames, it triggers the playback of the sound(s) associated with the part(s) of the grid in which change has occurred. Due to the relative compactness of the grid, even small gestures can trigger multiple contiguous sounds. The sounds are played back into the space through the sound system virtually instantaneously, in roughly 1/20^th of a second.

Very Nervous System encourages an intuitive process of bodily improvisation, kinetic experimentation, and creative play, opening a critical space for those who interact with the work to reflect on the inherent tensions and contradictions between the logic of the computer binary code that underpins the piece and the intuitive, embodied improvisatory gestures that activate it. The piece inverts the traditional relationship between movement/dance and sound/music: instead of the body responding to music, the improvisatory movements of the body cause the sounds, orchestrating the music that one hears. The work thus functions as what Rokeby has called a “transforming mirror,” both reflecting and refracting our actions in space and time and our changing our awareness of body’s relationship(s) to sound.

Very Nervous System has undergone several transformations over its twenty-five year history. Advances in video technology and image processing capabilities have made the work much more sensitive. Where earlier versions of the work responded best to large, dramatic gestures, the current version responds to minute movements, creating a heightened sense of intimacy between the viewer and the work. The sonic dimension of the piece has changed too. In its early incarnations, the sound was more traditionally musical in nature, using different synthesizer patches to allude to a variety of musical traditions. In recent years, these explicitly musical sounds have been largely replaced by more intimate sounds that do not signify particular musical or cultural traditions to the same extent—the sound of paper crinkling or the unrolling of duct tape, for example. Many of these sounds have a decidedly tactile quality, further complicating the interplay of gestural discourses involved in the work and heightening the intimacy of the experience to an even greater extent.

Rokeby has adapted the Very Nervous System technology in a variety of other pieces including Measure (1992), 60 (1995), Watch (1995), Taken (2002), Seen (2002), and Dark Matter (2010). In a sense, the VNS software has become something of a transforming mirror within Rokeby’s own creative practice. And it has transformed, and continues to transform, the field of interactive media art more generally. A growing community of creative practitioners around the world is using the highly flexible VNS software (now known as softVNS) to create interactive systems of their own. The technology has had numerous practical applications in contexts outside of art and music as well.  For example, VNS was modified to help a paralyzed woman who could only move her eyelids communicate. By calculating the changing distance from her eyelash to her eyebrow, the VNS system enabled her to use eyelid movements to communicate in code, typing up to fifteen words per minute. VNS has also been used extensively in rehabilitation centres to allow people with physical or mental disabilities, including those with extremely limited motor control, to create music and sound using their bodies as musical instruments.

The experience of improvising with Very Nervous System is more complex than one might think. The relationship between bodily gesture and auditory response does not feel like a unidirectional form of mimesis in which gesture x causes sound y. Having spent extended periods of time interacting with different incarnations of the work, I have noticed a curious perceptual phenomenon on numerous occasions: I sometimes get the feeling that the system actually anticipates my bodily movements. It becomes difficult to discern who is leading, human or machine. Rokeby has commented on this phenomenon as well, describing his early attempts to “out draw” the system in his studio. Of course, the system can’t really anticipate our gestures. But the speed with which it responds seems to be faster than our ability to process what is taking place in an analytical, left-brained manner. This enables the improvising mind/body to become fully immersed in the moment, to “let go,” and enter into what psychologist Mihaly Csikszentmihalyi has termed a state of flow, “the state in which people are so involved in an activity that nothing else seems to matter” (Cskikszentmihalyi, 1990, p. 4).

Unlike many other improvisatory modes of music making that require years of training and the internalization of genre-specific rules, Very Nervous System allows us to enter into a flow state very quickly. For those who interact with the work, there is only one rule: move your body to hear a sound.  All of the other constraints governing the interaction are beyond our control, having been established in advance through lines of computer code. No one can be “better” at improvising with VNS than anyone else. Therein lies the revolutionary potential of Very Nervous System and of interactive sound art more generally: by placing greater creative agency in the ears, eyes, minds, and bodies of the viewer/listener, they make improvisers of us all.

References

Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York:

Harper & Row, 1990.

Rokeby, David. “Transforming Mirrors: Subjectivity and Control in Interactive

Media.” Critical Issues in Electronic Media. Simon Penny, ed. Albany: State

University of New York Press, 1995:133-157.

Jesse Stewart is a composer, percussionist, improviser, visual artist and instrument-builder. He is also a professor in the School for Studies in Art and Culture at Carleton University in Ottawa.