Nina B Baltierra1, Kathryn E Muessig2, Emily C Pike1, Sara LeGrand3, Sheana S Bull4, Lisa B Hightow-Weidman1. 1. Department of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. 2. Department of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; Department of Health Behavior, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. Electronic address: kmuessig@email.unc.edu. 3. Center for Health Policy and Inequalities Research, Duke University, Durham, NC, USA. 4. Colorado School of Public Health, University of Colorado Denver, Aurora, CO, USA.
Abstract
BACKGROUND: There has been a rise in internet-based health interventions without a concomitant focus on new methods to measure user engagement and its effect on outcomes. We describe current user tracking methods for internet-based health interventions and offer suggestions for improvement based on the design and pilot testing of healthMpowerment.org (HMP). METHODS: HMP is a multi-component online intervention for young Black men and transgender women who have sex with men (YBMSM/TW) to reduce risky sexual behaviors, promote healthy living and build social support. The intervention is non-directive, incorporates interactive features, and utilizes a point-based reward system. Fifteen YBMSM/TW (age 20-30) participated in a one-month pilot study to test the usability and efficacy of HMP. Engagement with the intervention was tracked using a customized data capture system and validated with Google Analytics. Usage was measured in time spent (total and across sections) and points earned. RESULTS: Average total time spent on HMP was five hours per person (range 0-13). Total time spent was correlated with total points earned and overall site satisfaction. CONCLUSION: Measuring engagement in internet-based interventions is crucial to determining efficacy. Multiple methods of tracking helped derive more comprehensive user profiles. Results highlighted the limitations of measures to capture user activity and the elusiveness of the concept of engagement.
BACKGROUND: There has been a rise in internet-based health interventions without a concomitant focus on new methods to measure user engagement and its effect on outcomes. We describe current user tracking methods for internet-based health interventions and offer suggestions for improvement based on the design and pilot testing of healthMpowerment.org (HMP). METHODS: HMP is a multi-component online intervention for young Black men and transgender women who have sex with men (YBMSM/TW) to reduce risky sexual behaviors, promote healthy living and build social support. The intervention is non-directive, incorporates interactive features, and utilizes a point-based reward system. Fifteen YBMSM/TW (age 20-30) participated in a one-month pilot study to test the usability and efficacy of HMP. Engagement with the intervention was tracked using a customized data capture system and validated with Google Analytics. Usage was measured in time spent (total and across sections) and points earned. RESULTS: Average total time spent on HMP was five hours per person (range 0-13). Total time spent was correlated with total points earned and overall site satisfaction. CONCLUSION: Measuring engagement in internet-based interventions is crucial to determining efficacy. Multiple methods of tracking helped derive more comprehensive user profiles. Results highlighted the limitations of measures to capture user activity and the elusiveness of the concept of engagement.
Authors: Lisa B Hightow-Weidman; Emily Pike; Beth Fowler; Derrick M Matthews; Jessica Kibe; Regina McCoy; Adaora A Adimora Journal: AIDS Care Date: 2012-01-24
Authors: Darrel H Higa; Nicole Crepaz; Khiya J Marshall; Linda Kay; H Waverly Vosburgh; Pilgrim Spikes; Cynthia M Lyles; David W Purcell Journal: AIDS Behav Date: 2013-05
Authors: Ricardo F Muñoz; Adrian Aguilera; Stephen M Schueller; Yan Leykin; Eliseo J Pérez-Stable Journal: J Med Internet Res Date: 2012-06-27 Impact factor: 5.428
Authors: Mick P Couper; Gwen L Alexander; Nanhua Zhang; Roderick J A Little; Noel Maddy; Michael A Nowak; Jennifer B McClure; Josephine J Calvi; Sharon J Rolnick; Melanie A Stopponi; Christine Cole Johnson Journal: J Med Internet Res Date: 2010-11-18 Impact factor: 5.428
Authors: Wendy Brouwer; Willemieke Kroeze; Rik Crutzen; Jascha de Nooijer; Nanne K de Vries; Johannes Brug; Anke Oenema Journal: J Med Internet Res Date: 2011-01-06 Impact factor: 5.428
Authors: Liesje Donkin; Ian B Hickie; Helen Christensen; Sharon L Naismith; Bruce Neal; Nicole L Cockayne; Nick Glozier Journal: J Med Internet Res Date: 2013-10-17 Impact factor: 5.428
Authors: José A Bauermeister; Jesse M Golinkoff; Kathryn E Muessig; Keith J Horvath; Lisa B Hightow-Weidman Journal: Curr Opin HIV AIDS Date: 2017-09 Impact factor: 4.283
Authors: Lisa B Hightow-Weidman; Kathryn E Muessig; José A Bauermeister; Sara LeGrand; Lynn E Fiellin Journal: Curr Opin HIV AIDS Date: 2017-09 Impact factor: 4.283
Authors: Shannon Phillips; Julie Kanter; Kenneth J Ruggiero; Martina Mueller; Mary A Johnson; Teresa J Kelechi Journal: Res Nurs Health Date: 2019-08-08 Impact factor: 2.228