residUUm: user mapping and performance strategies for multilayered live audiovisual generation

Abstract

We propose residUUm, an audiovisual performance tool that uses sonification to orchestrate a particle system of shapes, as an attempt to build an audiovisual user interface in which all the actions of a performer on a laptop are in- tended to be explicitly interpreted by the audience. We pro- pose two approaches to performing with residUUm and dis- cuss the methods utilized to fulfill the promise of audience- visible interaction: mapping and performance strategies ap- plied to express audiovisual interactions with multilayered sound-image relationships. The system received positive feedback from 34 audience participants on aspects such as aesthetics and audiovisual integration, and we identified fur- ther design challenges around performance clarity and strat- egy. We discuss residUUm’s development objectives, modes of interaction and the impact of an audience-visible inter- face on the performer and observer.