Strategies for evaluating information visualization tools
- 23 May 2006
- proceedings article
- Published by Association for Computing Machinery (ACM)
Abstract
After an historical review of evaluation methods, we describe an emerging research method called Multi-dimensional In-depth Long-term Case studies (MILCs) which seems well adapted to study the creative activities that users of information visualization systems engage in. We propose that the efficacy of tools can be assessed by documenting 1) usage (observations, interviews, surveys, logging etc.) and 2) expert users' success in achieving their professional goals. We summarize lessons from related ethnography methods used in HCI and provide guidelines for conducting MILCs for information visualization. We suggest ways to refine the methods for MILCs in modest sized projects and then envision ambitious projects with 3-10 researchers working over 1-3 years to understand individual and organizational use of information visualization by domain experts working at the frontiers of knowledge in their fields.Keywords
This publication has 16 references indexed in Scilit:
- The studio as laboratory: Combining creative practice and digital technology researchInternational Journal of Human-Computer Studies, 2005
- Informing the design of computer-based environments to support creativityInternational Journal of Human-Computer Studies, 2005
- Turning pictures into numbers: extracting and generating information from complex visualizationsInternational Journal of Human-Computer Studies, 2000
- Scenarios in user-centred design—setting the stage for reflection and actionInteracting with Computers, 2000
- Rapid ethnographyPublished by Association for Computing Machinery (ACM) ,2000
- Creating creativityACM Transactions on Computer-Human Interaction, 2000
- Supporting the creative user: a criteria-based approach to interaction designDesign Studies, 1997
- The design and long-term use of a personal electronic notebookPublished by Association for Computing Machinery (ACM) ,1996
- Evaluating hypermedia and learningACM Transactions on Information Systems, 1994
- Ethnographically-informed systems design for air traffic controlPublished by Association for Computing Machinery (ACM) ,1992