William Brown1, Po-Yin Yen, Marlene Rojas, Rebecca Schnall. 1. HIV Center for Clinical and Behavioral Studies, New York State Psychiatric Institute and Columbia University, New York, NY, United States; Department of Biomedical Informatics, Columbia University, New York, NY, United States.
Abstract
BACKGROUND: Over two decades of research has been conducted using mobile devices for health related behaviors yet many of these studies lack rigor. There are few evaluation frameworks for assessing the usability of mHealth, which is critical as the use of this technology proliferates. As the development of interventions using mobile technology increase, future work in this domain necessitates the use of a rigorous usability evaluation framework. METHODS: We used two exemplars to assess the appropriateness of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating the usability of mHealth technology. In the first exemplar, we conducted 6 focus group sessions to explore adolescents' use of mobile technology for meeting their health Information needs. In the second exemplar, we conducted 4 focus group sessions following an Ecological Momentary Assessment study in which 60 adolescents were given a smartphone with pre-installed health-related applications (apps). DATA ANALYSIS: We coded the focus group data using the 9 concepts of the Health-ITUEM: Error prevention, Completeness, Memorability, Information needs, Flexibility/Customizability, Learnability, Performance speed, Competency, Other outcomes. To develop a finer granularity of analysis, the nine concepts were broken into positive, negative, and neutral codes. A total of 27 codes were created. Two raters (R1 and R2) initially coded all text and a third rater (R3) reconciled coding discordance between raters R1 and R2. RESULTS: A total of 133 codes were applied to Exemplar 1. In Exemplar 2 there were a total of 286 codes applied to 195 excerpts. Performance speed, Other outcomes, and Information needs were among the most frequently occurring codes. CONCLUSION: Our two exemplars demonstrated the appropriateness and usefulness of the Health-ITUEM in evaluating mobile health technology. Further assessment of this framework with other study populations should consider whether Memorability and Error prevention are necessary to include when evaluating mHealth technology.
BACKGROUND: Over two decades of research has been conducted using mobile devices for health related behaviors yet many of these studies lack rigor. There are few evaluation frameworks for assessing the usability of mHealth, which is critical as the use of this technology proliferates. As the development of interventions using mobile technology increase, future work in this domain necessitates the use of a rigorous usability evaluation framework. METHODS: We used two exemplars to assess the appropriateness of the Health IT Usability Evaluation Model (Health-ITUEM) for evaluating the usability of mHealth technology. In the first exemplar, we conducted 6 focus group sessions to explore adolescents' use of mobile technology for meeting their health Information needs. In the second exemplar, we conducted 4 focus group sessions following an Ecological Momentary Assessment study in which 60 adolescents were given a smartphone with pre-installed health-related applications (apps). DATA ANALYSIS: We coded the focus group data using the 9 concepts of the Health-ITUEM: Error prevention, Completeness, Memorability, Information needs, Flexibility/Customizability, Learnability, Performance speed, Competency, Other outcomes. To develop a finer granularity of analysis, the nine concepts were broken into positive, negative, and neutral codes. A total of 27 codes were created. Two raters (R1 and R2) initially coded all text and a third rater (R3) reconciled coding discordance between raters R1 and R2. RESULTS: A total of 133 codes were applied to Exemplar 1. In Exemplar 2 there were a total of 286 codes applied to 195 excerpts. Performance speed, Other outcomes, and Information needs were among the most frequently occurring codes. CONCLUSION: Our two exemplars demonstrated the appropriateness and usefulness of the Health-ITUEM in evaluating mobile health technology. Further assessment of this framework with other study populations should consider whether Memorability and Error prevention are necessary to include when evaluating mHealth technology.
Authors: Elina Mattila; Juha Pärkkä; Marion Hermersdorf; Jussi Kaasinen; Janne Vainio; Kai Samposalo; Juho Merilahti; Juha Kolari; Minna Kulju; Raimo Lappalainen; Ilkka Korhonen Journal: IEEE Trans Inf Technol Biomed Date: 2008-07
Authors: Dean J Wantland; Carmen J Portillo; William L Holzemer; Rob Slaughter; Eva M McGhee Journal: J Med Internet Res Date: 2004-11-10 Impact factor: 5.428
Authors: Lee M Ritterband; Gerhard Andersson; Helen M Christensen; Per Carlbring; Pim Cuijpers Journal: J Med Internet Res Date: 2006-09-29 Impact factor: 5.428
Authors: Rebecca Schnall; Anastasia Okoniewski; Victoria Tiase; Alexander Low; Martha Rodriguez; Steven Kaplan Journal: J Med Internet Res Date: 2013-03-06 Impact factor: 5.428
Authors: Jeffrey I Campbell; Isaac Aturinda; Evans Mwesigwa; Bridget Burns; Data Santorino; Jessica E Haberer; David R Bangsberg; Richard J Holden; Norma C Ware; Mark J Siedner Journal: AIDS Behav Date: 2017-11
Authors: Belén Cruz Zapata; Antonio Hernández Niñirola; Ali Idri; José Luis Fernández-Alemán; Ambrosio Toval Journal: J Med Syst Date: 2014-06-24 Impact factor: 4.460
Authors: Hwayoung Cho; Dakota Powell; Adrienne Pichon; Lisa M Kuhns; Robert Garofalo; Rebecca Schnall Journal: Int J Med Inform Date: 2019-07-12 Impact factor: 4.046