How natural is a natural interface? An evaluation procedure based on action breakdowns

This paper describes an issue-based method to evaluate the naturalness of an interface. The method consists of the execution of a series of tasks on that interface, which is subsequently systematically analyzed to identify breakdowns in the users’ actions. The systematic analysis of breakdowns is allowed by the support of video-coding software (The Observer by Noldus). This method is described on its theoretical bases and then applied to the evaluation of a natural interface, a walk-in-place locomotion system for virtual spaces called Superfeet. The procedure is comparative, since Superfeet is compared to two locomotion devices, Superfeet enhanced with headtracker and a more traditional Joypad. The test involves 36 participants (mean age = 23.68, SD = 3.14). The outcomes of the breakdown analysis are illustrated at a progressively finer level of granularity from the amount and length of breakdowns, to the circumstances of the breakdowns, to the type of actions involved in the breakdowns. The potential of this procedure for usability studies is finally synthesized.

[1]  Ronald Azuma,et al.  A demonstrated optical tracker with scalable work area for head-mounted display systems , 1992, I3D '92.

[2]  Mariano Alcañiz Raya,et al.  Super-Feet: A Wireless Hand-Free Navigation System for Virtual Environments , 2007, HCI.

[3]  Kristopher J. Blom,et al.  Intuitive, Hands-free Travel Interfaces for Virtual Environments , 2005 .

[4]  Lucy Suchman Plans and situated actions: the problem of human-machine communication , 1987 .

[5]  Masayuki Nakajima,et al.  "Hands-free navigation methods for moving through a virtual landscape walking interface virtual reality input devices" , 2004, Proceedings Computer Graphics International, 2004..

[6]  I. Blinnikova,et al.  Task performance under influence of interruptions , 2003 .

[7]  Makoto Sato,et al.  Realizing a New Step-in-place Locomotion interface for Virtual Environment with Large Display System , 2002, EGVE.

[8]  Maxwell J. Wells,et al.  The Virtual Motion Controller: A Sufficient-Motion Walking Simulator , 1996 .

[9]  J. P. Morgan,et al.  Design and Analysis: A Researcher's Handbook , 2005, Technometrics.

[10]  Jock D. Mackinlay,et al.  Rapid controlled movement through a virtual 3D workspace , 1990, SIGGRAPH.

[11]  Jakob Nielsen,et al.  Gestural interfaces: a step backward in usability , 2010, INTR.

[12]  A.W.K. Gaillard,et al.  Operator Functional State: The Assessment and Prediction of Human Performance Degradation in Complex Tasks , 2003 .

[13]  Harry Budi Santoso,et al.  Measuring the user experience , 2008 .

[14]  Stephen A. R. Scrivener,et al.  The Use of Breakdown Analysis in Synchronous CSCW System Design , 1993, ECSCW.

[15]  Lucy A. Suchman,et al.  Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, , 1987 .

[16]  Rob Procter,et al.  Design guidelines for dealing with breakdowns and repairs in collaborative work settings , 2000, Int. J. Hum. Comput. Stud..

[17]  Rudy Darken,et al.  The omni-directional treadmill: a locomotion device for virtual worlds , 1997, UIST '97.

[18]  Jurriaan D. Mulder,et al.  Spatial input device structure and bimanual object manipulation in virtual environments , 2006, VRST '06.

[19]  Volker Paelke,et al.  Foot-based mobile interaction with games , 2004, ACE '04.

[20]  Roope Raisamo,et al.  Appropriateness of foot interaction for non-accurate spatial tasks , 2004, CHI EA '04.

[21]  Hiroo Iwata,et al.  Walking about virtual environments on an infinite floor , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[22]  Terry Winograd,et al.  Understanding computers and cognition , 1986 .

[23]  James K. Hahn,et al.  Animation of human walking in virtual environments , 1999, Proceedings Computer Animation 1999.

[24]  Brian P. Bailey,et al.  Measuring the effects of interruptions on task performance in the user interface , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[25]  Greg Welch,et al.  The HiBall Tracker: high-performance wide-area tracking for virtual and augmented environments , 1999, VRST '99.

[26]  Doug A. Bowman,et al.  A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods , 2002, Presence: Teleoperators & Virtual Environments.

[27]  Mary C. Whitton,et al.  Walking > walking-in-place > flying, in virtual environments , 1999, SIGGRAPH.

[28]  M. McDaniel,et al.  (www.interscience.wiley.com) DOI: 10.1002/acp.1002 Delaying Execution of Intentions: Overcoming the Costs of Interruptions , 2022 .

[29]  Tim May,et al.  Analysing Interaction: Video, Ethnography and Situated Conduct , 2002 .

[30]  Susanne Bødker,et al.  Through the Interface: A Human Activity Approach To User Interface Design , 1990 .

[31]  Donald A. Norman,et al.  Natural user interfaces are not natural , 2010, INTR.

[32]  Eric Horvitz,et al.  Disruption and recovery of computing tasks: field study, analysis, and directions , 2007, CHI.