Hwayoung Cho1, Dakota Powell2, Adrienne Pichon2, Lisa M Kuhns3, Robert Garofalo3, Rebecca Schnall2. 1. College of Nursing, University of Florida, Gainesville, FL, United States. Electronic address: hcho@ufl.edu. 2. School of Nursing, Columbia University, New York, NY, United States. 3. Division of Adolescent Medicine, Ann & Robert H. Lurie Children's Hospital of Chicago, Chicago, IL, United States; Department of Pediatrics, Feinberg School of Medicine, Northwestern University, Chicago, IL, United States.
Abstract
OBJECTIVE: To report on the use of an eye-tracking retrospective think-aloud for usability evaluation and to describe its application in assessing the usability of a mobile health app. MATERIALS AND METHODS: We used an eye-tracking retrospective think-aloud to evaluate the usability of an HIV prevention mobile app among 20 young men (15-18 years) in New York City, NY; Birmingham, AL; and Chicago, IL. Task performance metrics, critical errors, a task completion rate per participant, and a task completion rate per task, were measured. Eye-tracking metrics including fixation, saccades, time to first fixation, time spent, and revisits were measured and compared among participants with/without a critical error. RESULTS: Using task performance analysis, we identified 19 critical errors on four activities, and of those, two activities had a task completion rate of less than 78%. To better understand these usability issues, we thoroughly analyzed participants' corresponding eye movements and verbal comments using an in-depth problem analysis. In areas of interest created for the activity with critical usability problems, there were significant differences in time spent (p = 0.008), revisits (p = 0.004), and total numbers of fixations (p = 0.007) by participants with/without a critical error. The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33), reflecting strong usability of the app. DISCUSSION AND CONCLUSION: An eye-tracking retrospective think-aloud enabled us to identify critical usability problems as well as gain an in-depth understanding of the usability issues related to interactions between end-users and the app. Findings from this study highlight the utility of an eye-tracking retrospective think-aloud in consumer health usability evaluation research.
OBJECTIVE: To report on the use of an eye-tracking retrospective think-aloud for usability evaluation and to describe its application in assessing the usability of a mobile health app. MATERIALS AND METHODS: We used an eye-tracking retrospective think-aloud to evaluate the usability of an HIV prevention mobile app among 20 young men (15-18 years) in New York City, NY; Birmingham, AL; and Chicago, IL. Task performance metrics, critical errors, a task completion rate per participant, and a task completion rate per task, were measured. Eye-tracking metrics including fixation, saccades, time to first fixation, time spent, and revisits were measured and compared among participants with/without a critical error. RESULTS: Using task performance analysis, we identified 19 critical errors on four activities, and of those, two activities had a task completion rate of less than 78%. To better understand these usability issues, we thoroughly analyzed participants' corresponding eye movements and verbal comments using an in-depth problem analysis. In areas of interest created for the activity with critical usability problems, there were significant differences in time spent (p = 0.008), revisits (p = 0.004), and total numbers of fixations (p = 0.007) by participants with/without a critical error. The overall mean score of perceived usability rated by the Health IT Usability Evaluation Scale was 4.64 (SD = 0.33), reflecting strong usability of the app. DISCUSSION AND CONCLUSION: An eye-tracking retrospective think-aloud enabled us to identify critical usability problems as well as gain an in-depth understanding of the usability issues related to interactions between end-users and the app. Findings from this study highlight the utility of an eye-tracking retrospective think-aloud in consumer health usability evaluation research.
Keywords:
Eye movement measurements; Eye movements; Eye-tracking; Health IT; Information technology; Mobile applications; Mobile health; Usability evaluation
Authors: Rebecca Schnall; Lisa M Kuhns; Marco A Hidalgo; Dakota Powell; Jennie Thai; Sabina Hirshfield; Cynthia Pearson; Matt Ignacio; Josh Bruce; D Scott Batey; Asa Radix; Uri Belkind; Robert Garofalo Journal: AIDS Educ Prev Date: 2018-12
Authors: Rebecca Schnall; Lisa Kuhns; Marco Hidalgo; Sabina Hirshfield; Cynthia Pearson; Asa Radix; Uri Belkind; Joshua Bruce; D Scott Batey; Robert Garofalo Journal: Stud Health Technol Inform Date: 2018