Nathaniel J Williams1, Molly Candon2,3, Rebecca E Stewart2,3, Y Vivian Byeon2,4, Meenakshi Bewtra5,6,7, Alison M Buttenheim3,8,9,10, Kelly Zentgraf2, Carrie Comeau11, Sonsunmolu Shoyinka11, Rinad S Beidas12,13,14,15,16,17. 1. School of Social Work, Boise State University, Boise, ID, USA. 2. Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA. 3. Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA. 4. Department of Psychology, University of California, Los Angeles, Los Angeles, CA, USA. 5. Center for Clinical Epidemiology and Biostatistics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. 6. Division of Gastroenterology, University of Pennsylvania, Philadelphia, PA, USA. 7. Department of Biostatistics and Epidemiology, University of Pennsylvania, Philadelphia, PA, USA. 8. Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. 9. Center for Health Incentives and Behavioral Economics, University of Pennsylvania, Philadelphia, PA, USA. 10. Department of Family and Community Health, School of Nursing, University of Pennsylvania, Philadelphia, PA, USA. 11. Department of Behavioral Health and Intellectual disAbility Services (DBHIDS), Philadelphia, PA, USA. 12. Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA. rbeidas@upenn.edu. 13. Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu. 14. Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu. 15. Center for Health Incentives and Behavioral Economics, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu. 16. Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA. rbeidas@upenn.edu. 17. Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, 3535 Market Street, 3015, Philadelphia, PA, 19104, USA. rbeidas@upenn.edu.
Abstract
BACKGROUND: Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling. METHODS: A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles. RESULTS: On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics. CONCLUSIONS: The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.
BACKGROUND: Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling. METHODS: A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles. RESULTS: On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics. CONCLUSIONS: The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.
Authors: Byron J Powell; J Curtis McMillen; Enola K Proctor; Christopher R Carpenter; Richard T Griffey; Alicia C Bunger; Joseph E Glass; Jennifer L York Journal: Med Care Res Rev Date: 2011-12-26 Impact factor: 3.929
Authors: Charles E Cunningham; Melanie Barwick; Kathy Short; Yvonne Chen; Heather Rimas; Jenna Ratcliffe; Stephanie Mielko Journal: School Ment Health Date: 2014
Authors: Rinad S Beidas; Nathaniel J Williams; Emily M Becker-Haimes; Gregory A Aarons; Frances K Barg; Arthur C Evans; Kamilah Jackson; David Jones; Trevor Hadley; Kimberly Hoagwood; Steven C Marcus; Geoffrey Neimark; Ronnie M Rubin; Sonja K Schoenwald; Danielle R Adams; Lucia M Walsh; Kelly Zentgraf; David S Mandell Journal: Implement Sci Date: 2019-06-21 Impact factor: 7.327
Authors: Rinad S Beidas; Emily M Becker-Haimes; Danielle R Adams; Laura Skriner; Rebecca E Stewart; Courtney Benjamin Wolk; Alison M Buttenheim; Nathaniel J Williams; Patricia Inacker; Elizabeth Richey; Steven C Marcus Journal: Implement Sci Date: 2017-12-15 Impact factor: 7.327