Bryan R Garner1, Sarah B Hunter2, Rodney R Funk3, Beth Ann Griffin4, Susan H Godley5. 1. RTI International, P. O. Box 12194, Research Triangle Park, NC, 27709-2194, USA. Electronic address: bgarner@rti.org. 2. RAND, P.O. Box 2138, Santa Monica, CA, 90407-2138, USA. Electronic address: shunter@rand.org. 3. Chestnut Health Systems, 448 Wylie Dr., Normal, IL, 61761, USA. Electronic address: rfunk@chestnut.org. 4. RAND, P.O. Box 2138, Santa Monica, CA, 90407-2138, USA. Electronic address: bethg@rand.org. 5. Chestnut Health Systems, 448 Wylie Dr., Normal, IL, 61761, USA. Electronic address: sgodley@chestnut.org.
Abstract
BACKGROUND: Developing consistent, valid, and efficient implementation outcome measures is necessary to advance implementation science. However, development of such measures has been limited to date, especially for validating the extent to which such measures are associated with important improvements in client outcomes. This study seeks to address this gap by developing one or more evidence-based measures of implementation (EBMIs; i.e., implementation outcome measure that is predictive of improvements in key client outcomes) for the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based practice (EBP) for adolescent substance use. METHODS: Data for the current study were collected as part of a large-scale federally funded EBP dissemination and implementation initiative. The multilevel dataset included 65 substance use treatment organizations, 308 clinicians, and 5873 adolescent clients. Adjusted multilevel regression analyses were used to examine the extent to which client-level outcome measures assessed at 6-month follow-up (i.e., substance use, emotional problems) could be predicted by four implementation outcomes: two measures of fidelity (i.e., session exposure, procedure exposure) and two measures of penetration (i.e., absolute client penetration, absolute staff penetration). RESULTS: Adjusting for client substance use at intake, as well as several client characteristics (e.g., age, race, criminal justice involvement), client substance use at follow-up was significantly lower for treatment organizations that had higher procedure exposure (B=-1.227, standard error [SE]=0.583, 95% confidence interval=-2.370, 0.252; p<.05). None of the other three implementation outcome measures were found to predict improvements in client outcomes. CONCLUSIONS: The current study provides support for procedure exposure as an organizational-level EBMI for A-CRA. Thus, future efforts focused on implementing A-CRA could be improved by measuring and monitoring the extent to which A-CRA procedures are being delivered to clients. Additionally, given the dearth of studies that have examined the relationship between organizational-level measures of implementation and client outcomes, this article provides a prototype for future research to identify EBMIs for other behavioral treatments.
BACKGROUND: Developing consistent, valid, and efficient implementation outcome measures is necessary to advance implementation science. However, development of such measures has been limited to date, especially for validating the extent to which such measures are associated with important improvements in client outcomes. This study seeks to address this gap by developing one or more evidence-based measures of implementation (EBMIs; i.e., implementation outcome measure that is predictive of improvements in key client outcomes) for the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based practice (EBP) for adolescent substance use. METHODS: Data for the current study were collected as part of a large-scale federally funded EBP dissemination and implementation initiative. The multilevel dataset included 65 substance use treatment organizations, 308 clinicians, and 5873 adolescent clients. Adjusted multilevel regression analyses were used to examine the extent to which client-level outcome measures assessed at 6-month follow-up (i.e., substance use, emotional problems) could be predicted by four implementation outcomes: two measures of fidelity (i.e., session exposure, procedure exposure) and two measures of penetration (i.e., absolute client penetration, absolute staff penetration). RESULTS: Adjusting for client substance use at intake, as well as several client characteristics (e.g., age, race, criminal justice involvement), client substance use at follow-up was significantly lower for treatment organizations that had higher procedure exposure (B=-1.227, standard error [SE]=0.583, 95% confidence interval=-2.370, 0.252; p<.05). None of the other three implementation outcome measures were found to predict improvements in client outcomes. CONCLUSIONS: The current study provides support for procedure exposure as an organizational-level EBMI for A-CRA. Thus, future efforts focused on implementing A-CRA could be improved by measuring and monitoring the extent to which A-CRA procedures are being delivered to clients. Additionally, given the dearth of studies that have examined the relationship between organizational-level measures of implementation and client outcomes, this article provides a prototype for future research to identify EBMIs for other behavioral treatments.
Authors: Diane E Sholomskas; Gia Syracuse-Siewert; Bruce J Rounsaville; Samuel A Ball; Kathryn F Nuro; Kathleen M Carroll Journal: J Consult Clin Psychol Date: 2005-02
Authors: Michael Dennis; Susan H Godley; Guy Diamond; Frank M Tims; Thomas Babor; Jean Donaldson; Howard Liddle; Janet C Titus; Yifrah Kaminer; Charles Webb; Nancy Hamilton; Rod Funk Journal: J Subst Abuse Treat Date: 2004-10
Authors: Alex R Dopp; Sarah B Hunter; Mark D Godley; Chau Pham; Bing Han; Rosanna Smart; Jonathan Cantor; Beau Kilmer; Grace Hindmarch; Isabelle González; Lora L Passetti; Kelli L Wright; Gregory A Aarons; Jonathan Purtle Journal: Implement Sci Commun Date: 2022-05-13
Authors: Sara J Becker; Cara M Murphy; Bryan Hartzler; Carla J Rash; Tim Janssen; Mat Roosa; Lynn M Madden; Bryan R Garner Journal: Addict Sci Clin Pract Date: 2021-10-12
Authors: Bryan R Garner; Heather J Gotham; Michael Chaple; Steve Martino; James H Ford Ii; Mathew R Roosa; Kathryn J Speck; Denna Vandersloot; Michael Bradshaw; Elizabeth L Ball; Alyssa K Toro; Christopher Griggs; Stephen J Tueller Journal: Implement Res Pract Date: 2020-09-07