Wenjing Huang1, Sarah B Hunter2, Lynsay Ayer3, Bing Han2, Mary E Slaughter4, Bryan R Garner5, Susan H Godley6. 1. RAND, Santa Monica, CA 90407-2138, United States. Electronic address: whuang@rand.org. 2. RAND, Santa Monica, CA 90407-2138, United States. 3. RAND, Arlington, VA 22202-5050, United States. 4. RAND, Pittsburgh, PA 15213, United States. 5. Research Triangle Institute, Raleigh, NC 27675, United States. 6. Chestnut Health Systems, Normal, IL 61761, United States.
Abstract
BACKGROUND: Measuring the extent a behavioral treatment is sustained in usual care practice settings after initial support for implementation ends is challenging. Oftentimes dichotomous self-report measures are used to measure health program sustainment, but these measures may not adequately capture the quality or extent of behavioral treatment delivery. Using data collected from community based organizations that received support to implement the Adolescent Community Reinforcement Approach (A-CRA), a measure of the extent of sustainment was derived. METHODS: Based on implementation theories and implementation support protocols, a total of 10 core treatment elements were identified to measure the extent of sustainment using information collected from key clinical staff. Item response theory (IRT) and principal component analyses (PCA) were used to further refine the 10 elements into composite measures of sustainment. The association between the 10 elements and the relationships between the comprehensive measures to a dichotomous sustainment measure were also examined. RESULTS: Results from PCA identified two components from the initial 10 elements for measuring the extent of A-CRA treatment sustainment. The two components described different aspects of organizational support for A-CRA treatment implementation, one representing the quality or extent of treatment staffing and delivery and the other representing the quality of or extent of clinical supervision. CONCLUSIONS: Using IRT and PCA, we were able to derive components that could be used to measure the extent of EBT sustainment and also better capture the quality of treatment delivery than the use of simple dichotomous measure. The methodological contribution of our study is that we have demonstrated a general analytic approach that may be applicable for other psychosocial treatments.
BACKGROUND: Measuring the extent a behavioral treatment is sustained in usual care practice settings after initial support for implementation ends is challenging. Oftentimes dichotomous self-report measures are used to measure health program sustainment, but these measures may not adequately capture the quality or extent of behavioral treatment delivery. Using data collected from community based organizations that received support to implement the Adolescent Community Reinforcement Approach (A-CRA), a measure of the extent of sustainment was derived. METHODS: Based on implementation theories and implementation support protocols, a total of 10 core treatment elements were identified to measure the extent of sustainment using information collected from key clinical staff. Item response theory (IRT) and principal component analyses (PCA) were used to further refine the 10 elements into composite measures of sustainment. The association between the 10 elements and the relationships between the comprehensive measures to a dichotomous sustainment measure were also examined. RESULTS: Results from PCA identified two components from the initial 10 elements for measuring the extent of A-CRA treatment sustainment. The two components described different aspects of organizational support for A-CRA treatment implementation, one representing the quality or extent of treatment staffing and delivery and the other representing the quality of or extent of clinical supervision. CONCLUSIONS: Using IRT and PCA, we were able to derive components that could be used to measure the extent of EBT sustainment and also better capture the quality of treatment delivery than the use of simple dichotomous measure. The methodological contribution of our study is that we have demonstrated a general analytic approach that may be applicable for other psychosocial treatments.
Authors: Gregory A Aarons; Amy E Green; Elise Trott; Cathleen E Willging; Elisa M Torres; Mark G Ehrhart; Scott C Roesch Journal: Adm Policy Ment Health Date: 2016-11
Authors: Michael Dennis; Susan H Godley; Guy Diamond; Frank M Tims; Thomas Babor; Jean Donaldson; Howard Liddle; Janet C Titus; Yifrah Kaminer; Charles Webb; Nancy Hamilton; Rod Funk Journal: J Subst Abuse Treat Date: 2004-10
Authors: Stephanie M C Ament; Jeanny J A de Groot; José M C Maessen; Carmen D Dirksen; Trudy van der Weijden; Jos Kleijnen Journal: BMJ Open Date: 2015-12-29 Impact factor: 2.692
Authors: Sarah B Hunter; Melissa Felician; Alex R Dopp; Susan H Godley; Chau Pham; Kathryn Bouskill; Mary E Slaughter; Bryan R Garner Journal: J Subst Abuse Treat Date: 2020-03-07
Authors: Alex R Dopp; Sarah B Hunter; Mark D Godley; Chau Pham; Bing Han; Rosanna Smart; Jonathan Cantor; Beau Kilmer; Grace Hindmarch; Isabelle González; Lora L Passetti; Kelli L Wright; Gregory A Aarons; Jonathan Purtle Journal: Implement Sci Commun Date: 2022-05-13