OBJECTIVE: To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials. STUDY DESIGN: Currently no validated definition of effectiveness studies exists. We asked the directors of 12 Evidence-based Practice Centers to select six studies each: four that they considered to be examples of effectiveness trials and two considered efficacy studies. We then applied our proposed criteria to test the construct validity using the selected studies as if they had been identified by a gold standard. RESULTS: Based on the rationale to identify effectiveness studies reliably with minimal false positives (i.e., a high specificity), a cutoff of six criteria produced the most desirable balance between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of 0.72. CONCLUSION: When applied in a standardized manner, our proposed criteria can provide a valid and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic reviews can improve when analysts place more emphasis on the generalizability of included studies. In addition, clinicians can also use our criteria to determine the external validity of individual studies, given an appropriate population of interest.
OBJECTIVE: To propose and test a simple instrument based on seven criteria of study design to distinguish effectiveness (pragmatic) from efficacy (explanatory) trials. STUDY DESIGN: Currently no validated definition of effectiveness studies exists. We asked the directors of 12 Evidence-based Practice Centers to select six studies each: four that they considered to be examples of effectiveness trials and two considered efficacy studies. We then applied our proposed criteria to test the construct validity using the selected studies as if they had been identified by a gold standard. RESULTS: Based on the rationale to identify effectiveness studies reliably with minimal false positives (i.e., a high specificity), a cutoff of six criteria produced the most desirable balance between sensitivity and specificity. This setting produced a specificity of 0.83 and a sensitivity of 0.72. CONCLUSION: When applied in a standardized manner, our proposed criteria can provide a valid and simple tool to distinguish effectiveness from efficacy studies. The applicability of systematic reviews can improve when analysts place more emphasis on the generalizability of included studies. In addition, clinicians can also use our criteria to determine the external validity of individual studies, given an appropriate population of interest.
Authors: Antonia F H Smelt; Gerda M van der Weele; Jeanet W Blom; Jacobijn Gussekloo; Willem J J Assendelft Journal: Br J Gen Pract Date: 2010-07 Impact factor: 5.386
Authors: Kevin E Thorpe; Merrick Zwarenstein; Andrew D Oxman; Shaun Treweek; Curt D Furberg; Douglas G Altman; Sean Tunis; Eduardo Bergel; Ian Harvey; David J Magid; Kalipso Chalkidou Journal: CMAJ Date: 2009-04-16 Impact factor: 8.262
Authors: Thomas W Frazier; Eric W Klingemier; Christa J Anderson; Grace W Gengoux; Eric A Youngstrom; Antonio Y Hardan Journal: J Autism Dev Disord Date: 2021-02-08
Authors: Alexandra P Metse; John H Wiggers; Paula M Wye; Luke Wolfenden; Judith J Prochaska; Emily A Stockings; Jill M Williams; Kerryn Ansell; Caitlin Fehily; Jenny A Bowman Journal: Nicotine Tob Res Date: 2016-11-10 Impact factor: 4.244
Authors: L Susan Wieland; Brian M Berman; Douglas G Altman; Jürgen Barth; Lex M Bouter; Christopher R D'Adamo; Klaus Linde; David Moher; C Daniel Mullins; Shaun Treweek; Sean Tunis; Danielle A van der Windt; Merrick Zwarenstein; Claudia Witt Journal: J Clin Epidemiol Date: 2017-02-07 Impact factor: 6.437