Field experiments in HCI
promises and challenges
pp. 87-116
in: Hannakaisa Isomki, Pertti Saariluoma (eds), Future interaction design II, Berlin, Springer, 2009Abstract
Experimental methods have been under criticism since the advent of mobile and ubiquitous technologies, due to clear limitations in their suitability for studies in the field. However, the laboratory paradigm cannot be directly transferred to field conditions because of its strict notions of experimentation. This chapter examines the theory of quasi-experimentation as an alternative conceptualization of causality, control, and validity. Several threats to experimental validity in field experiments in HCI are discussed. These concerns must be addressed at all levels of experimentation, from the design and execution of a field experiment to analysis of data. Noteworthy also are new technical solutions that have enabled high-fidelity data collection and that generally support endeavors in ensuring validity. If field experimentation is to become the de facto standard of research in human–computer interaction, the methodological core and technical tools must be developed in concert.