Jeffrey D Lambert1, Lewis R Elliott2, Adrian H Taylor3, Paul Farrand4, Anne M Haase5, Colin J Greaves6. 1. Department for Health, University of Bath. 2. European Centre for Environment and Human Health, University of Exeter Medical School. 3. Peninsula School of Medicine, Faculty of Health, University of Plymouth. 4. Clinical Education, Development and Research (CEDAR), Psychology Program, University of Exeter. 5. School of Health, Victoria University of Wellington. 6. School of Sport, Exercise and Rehabilitation Sciences, University of Birmingham.
Abstract
OBJECTIVE: Delivery is one of the most common ways of assessing fidelity in behavioral interventions. However, there is a lack of research reporting on how well an intervention protocol reflects its proposed theoretical principles (design fidelity). This study presents a systematic method for assessing design fidelity and applies it to the eMotion web-based intervention targeting physical activity and depression. METHOD: The eMotion intervention comprises of 13 web-based modules, designed according to an underlying intervention map. An independent rater with expertise in behavior change coded the presence or absence of behavior change techniques (BCTs) in the content of eMotion. Results of coding were compared to the intervention designers' a priori specification for interrater reliability. RESULTS: After discussion, the independent rater and the intervention designer had a high agreement for the presence of BCTs relating to behavioral activation (AC1 = 0.91) with "demonstration of behavior" and "monitoring of emotional consequences" having the lowest agreement (AC1 < 0.4). There was also high agreement for the presence of BCTs targeting physical activity (AC1 = 0.88) with "demonstration of behavior" and "monitoring of emotional consequences" having the lowest agreement (AC1 < 0.4). The eMotion description was then amended to align the interrater agreement. CONCLUSIONS: This study presents a novel method for assessing design fidelity. Developers of behavioral (and other multicomponent) interventions are encouraged to develop and refine this method and assess design fidelity in future interventions to ensure BCTs are operationalized as intended. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
OBJECTIVE: Delivery is one of the most common ways of assessing fidelity in behavioral interventions. However, there is a lack of research reporting on how well an intervention protocol reflects its proposed theoretical principles (design fidelity). This study presents a systematic method for assessing design fidelity and applies it to the eMotion web-based intervention targeting physical activity and depression. METHOD: The eMotion intervention comprises of 13 web-based modules, designed according to an underlying intervention map. An independent rater with expertise in behavior change coded the presence or absence of behavior change techniques (BCTs) in the content of eMotion. Results of coding were compared to the intervention designers' a priori specification for interrater reliability. RESULTS: After discussion, the independent rater and the intervention designer had a high agreement for the presence of BCTs relating to behavioral activation (AC1 = 0.91) with "demonstration of behavior" and "monitoring of emotional consequences" having the lowest agreement (AC1 < 0.4). There was also high agreement for the presence of BCTs targeting physical activity (AC1 = 0.88) with "demonstration of behavior" and "monitoring of emotional consequences" having the lowest agreement (AC1 < 0.4). The eMotion description was then amended to align the interrater agreement. CONCLUSIONS: This study presents a novel method for assessing design fidelity. Developers of behavioral (and other multicomponent) interventions are encouraged to develop and refine this method and assess design fidelity in future interventions to ensure BCTs are operationalized as intended. (PsycInfo Database Record (c) 2021 APA, all rights reserved).
Authors: Jeffrey Lambert; Adrian Taylor; Adam Streeter; Colin Greaves; Wendy M Ingram; Sarah Dean; Kate Jolly; Nanette Mutrie; Rod S Taylor; Lucy Yardley; Lisa Price; John Campbell Journal: Int J Behav Nutr Phys Act Date: 2022-09-29 Impact factor: 8.915