Caroline A Figueroa1, Adrian Aguilera1,2, Bibhas Chakraborty3,4,5, Arghavan Modiri6, Jai Aggarwal6, Nina Deliu6,7, Urmimala Sarkar2, Joseph Jay Williams6, Courtney R Lyles2. 1. School of Social Welfare, University of California Berkeley, Berkeley, California, USA. 2. UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA. 3. Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore. 4. Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore. 5. Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA. 6. Department of Computer Science, University of Toronto, Toronto, Canada. 7. Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy.
Abstract
OBJECTIVE: Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making. MATERIALS AND METHODS: Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains. RESULTS: Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings. CONCLUSION: The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility. TRIAL REGISTRATION: clinicaltrials.gov, NCT03490253.
OBJECTIVE: Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making. MATERIALS AND METHODS: Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study "DIAMANTE" for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains. RESULTS: Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness/implementation in real-world settings. CONCLUSION: The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility. TRIAL REGISTRATION: clinicaltrials.gov, NCT03490253.
Authors: Courtney R Lyles; Margaret A Handley; Sara L Ackerman; Dean Schillinger; Pamela Williams; Marisa Westbrook; Gato Gourley; Urmimala Sarkar Journal: Am J Med Qual Date: 2018-09-10 Impact factor: 1.852
Authors: Sebastian Vollmer; Bilal A Mateen; Gergo Bohner; Franz J Király; Rayid Ghani; Pall Jonsson; Sarah Cumbers; Adrian Jonas; Katherine S L McAllister; Puja Myles; David Granger; Mark Birse; Richard Branson; Karel G M Moons; Gary S Collins; John P A Ioannidis; Chris Holmes; Harry Hemingway Journal: BMJ Date: 2020-03-20
Authors: Joseph Firth; John Torous; Jennifer Nicholas; Rebekah Carney; Abhishek Pratap; Simon Rosenbaum; Jerome Sarris Journal: World Psychiatry Date: 2017-10 Impact factor: 49.548
Authors: David C Mohr; Aaron R Lyon; Emily G Lattie; Madhu Reddy; Stephen M Schueller Journal: J Med Internet Res Date: 2017-05-10 Impact factor: 5.428
Authors: Inbal Nahum-Shani; Shawna N Smith; Bonnie J Spring; Linda M Collins; Katie Witkiewitz; Ambuj Tewari; Susan A Murphy Journal: Ann Behav Med Date: 2018-05-18