Nü Soundworks: Using spectators smartphones as a distributed network of speakers and sensors during live performances

Abstract

This paper presents the Nu ̈ framework. The objective of the framework is to provide composers with a tool to con- trol web-based audio processes on spectators smartphones during live performances. Connecting their devices to a web- page broadcasted by the performer’s laptop, spectators become part of the composition: from simple sound sources to active musicians. From a Max based interface, the performer can then control the behaviours of conceptual units, referred to as Nu ̈ modules, designed for live composition (distributed room reverb, granular synthesis, etc.). Each module is composed of a pair of JavaScript classes – one for the client, another for the server – relying on the Web Audio API for audio processing, and OSC messages for parameters control. Nu ̈ is an open source project based on the Soundworks framework.