The Effect of Audio Cues and Sound Source Stimuli on Looming Perception.

Abstract

Objects that move in depth (looming) are ubiquitous in the real and virtual worlds. How humans interact and respond to these approaching objects may affect their continued survival in both the real and virtual words, and is dependent on the individual's capacity to accurately interpret depth and movement cues. In computer-generated environments, including hyper and virtual reality,film, and gaming, these cues are often complex sounds with multiple audio cues that are creatively designed for maximum effect. To accurately generate a dynamic and rich perception of looming objects, the design of such complex stimuli should be based on a firm scientific foundation that encompasses what we know about how people visually and aurally perceive events and interactions. Conversely, many psychological studies investigating auditory looming depict the object's movement using simple audio cues, such as an increase in the amplitude, which are applied to tones that are not regularly encountered in the natural world, such as sine, triangle, or square waves. Whilst the results from these studies have provided important information on human perception and responses, technological advances now allow us to present complex audiovisual stimuli and to collect measurements on human perception and responses to real and hyper-real stimuli. The research in this thesis begins to address the gap that exists between the research corpus and industry usage. This is initially accomplished by conducting a feature analysis of the audio cues and complex sounds constructed by sound designers for film scenes presenting objects moving in depth. This is followed by a perceptual study measuring human responses, both physical and emotional, to the complex audio cues designed for the film scenes. Using physical models, we then select a number of audio cues for closer inspection and introduce the parameter of `room reflections' as an audio cue. We investigate whether or not human responses to various audio cues differ when they are presented individually or in combination, or when they are applied to an artificial (square wave) sound source or a real world sound source. Finally, we test the capacity of these audio cues to bias multimodal auditory-visual perception of an approaching object.