Evaluating usability evaluation methods for location-aware interactive systems in contextually rich environments.
In this research we investigate the evaluation of usability evaluations methods (UEMs). In particular we are concerned with evaluating their suitability for the evaluation of location-‐ aware systems. Not all approaches for the evaluation of UEMs have been extensively validated for such types of dynamic interaction, while their application is not clearly documented. We overview the strengths of the current approach and suggest how to improve them. We examine navigation systems as examples for issues with location-‐aware systems in a contextually rich environment. The setting is very different to a traditional desktop-‐based application. Take the use of the navigation device for example. It is a secondary task; the primary task is to safely drive the car. The interface is continuously changing to adapt to the current location of the user. The user navigates in a complex dynamic environment encompassing various stimuli and unpredictable external factors. We present in the thesis a methodological and systematic way to approach the evaluation of UEMs. A comparative study of analytical and empirical techniques was carried out, to assess them in identifying usability problems within both static and dynamic contexts of use. Four analytical methods (CW, UAN, EMU, and Design Guidelines) and one empirical were compared. In this thesis, we validate the existing classification scheme of Blandford et al. (2008) and highlight relevant issues. We present an alternative systematic approach building on this scheme (CoHUM), and its shortcomings with dynamic systems. We show how a rigorous and systematic error analysis identifies phenotypes as the outcome of empirical techniques, whilst genotypes are the outcome of analytical techniques. Finally, we present new dimensions that previous literature had not identified for the evaluation of UEMs. This research will help future researchers by providing them with a stronger methodological approach for comparing UEMs and, in particular, categories of UEMs.
- Theses