Alyssa Zupon1, Craig Rothenberg2, Katherine Couturier2, Ting-Xu Tan2, Gina Siddiqui2, Matthew James2, Dan Savage3, Edward R Melnick2, Arjun K Venkatesh2,4. 1. Yale University School of Medicine, New Haven, Connecticut. 2. Department of Emergency Medicine, Yale University School of Medicine, New Haven, Connecticut. 3. Department of Emergency Medicine, University of California, San Francisco (UCSF) Fresno Medical Education Program, San Francisco, California. 4. Yale New Haven Hospital, Center for Outcomes Research and Evaluation, New Haven, Connecticut.
Abstract
BACKGROUND: Clinical practice guidelines (CPGs) have been published by the American College of Emergency Physicians (ACEP) since 1990 to advance evidence-based emergency care. ACEP clinical policies have drawn anecdotal criticism for bias, yet the overall quality of these guidelines has not previously been quantified. We sought to examine ACEP clinical policies using a recognised, validated appraisal instrument: Appraisal of Guidelines for Research & Evaluation (AGREE II). METHODS: Systematic assessment of current ACEP clinical policies was conducted using the AGREE II instrument, which contains 23 appraisal items (scored on a 1-7 scale) in six domains and two overall assessments. Each policy was independently appraised by five trained appraisers. Primary outcomes were AGREE II ratings for each item, domain and "Overall Assessment," and scores were reported as standardised percentages from all five appraisers. Secondary analyses examined associations between AGREE II ratings and policy publication date, strength of underlying evidence and strength of recommendations. Additional analysis examined relationships between domain and "Overall Assessment" ratings. RESULTS: Twenty guidelines published from April 2007 to November 2017 were included. Of the six domains, "Scope and Purpose" scored highest (mean 90%) and "Applicability" scored lowest (mean 35%). The four remaining domains ("Stakeholder Involvement," "Rigor of Development," "Clarity of Presentation" and "Editorial Independence") had mean scores of 53%-78%. The mean "Overall Assessment" rating was 69% and was not associated with policy publication date, strength of underlying evidence or strength of recommendations. We found positive associations between "Overall Assessment" ratings and two domains: "Rigor of Development" (r = 0.70) and "Clarity of Presentation" (r = 0.70). CONCLUSIONS: Based on validated AGREE II criteria, ACEP clinical policies can be most improved by addressing their application in practice. ACEP clinical policies' overall quality did not improve over the assessed time period and is not explained by the quality of underlying evidence.
BACKGROUND: Clinical practice guidelines (CPGs) have been published by the American College of Emergency Physicians (ACEP) since 1990 to advance evidence-based emergency care. ACEP clinical policies have drawn anecdotal criticism for bias, yet the overall quality of these guidelines has not previously been quantified. We sought to examine ACEP clinical policies using a recognised, validated appraisal instrument: Appraisal of Guidelines for Research & Evaluation (AGREE II). METHODS: Systematic assessment of current ACEP clinical policies was conducted using the AGREE II instrument, which contains 23 appraisal items (scored on a 1-7 scale) in six domains and two overall assessments. Each policy was independently appraised by five trained appraisers. Primary outcomes were AGREE II ratings for each item, domain and "Overall Assessment," and scores were reported as standardised percentages from all five appraisers. Secondary analyses examined associations between AGREE II ratings and policy publication date, strength of underlying evidence and strength of recommendations. Additional analysis examined relationships between domain and "Overall Assessment" ratings. RESULTS: Twenty guidelines published from April 2007 to November 2017 were included. Of the six domains, "Scope and Purpose" scored highest (mean 90%) and "Applicability" scored lowest (mean 35%). The four remaining domains ("Stakeholder Involvement," "Rigor of Development," "Clarity of Presentation" and "Editorial Independence") had mean scores of 53%-78%. The mean "Overall Assessment" rating was 69% and was not associated with policy publication date, strength of underlying evidence or strength of recommendations. We found positive associations between "Overall Assessment" ratings and two domains: "Rigor of Development" (r = 0.70) and "Clarity of Presentation" (r = 0.70). CONCLUSIONS: Based on validated AGREE II criteria, ACEP clinical policies can be most improved by addressing their application in practice. ACEP clinical policies' overall quality did not improve over the assessed time period and is not explained by the quality of underlying evidence.
Authors: Melissa C Brouwers; Michelle E Kho; George P Browman; Jako S Burgers; Francoise Cluzeau; Gene Feder; Béatrice Fervers; Ian D Graham; Jeremy Grimshaw; Steven E Hanna; Peter Littlejohns; Julie Makarski; Louise Zitzelsberger Journal: J Clin Epidemiol Date: 2010-07-24 Impact factor: 6.437
Authors: Edward R Melnick; Jeffrey A Nielson; John T Finnell; Michael J Bullard; Stephen V Cantrill; Dennis G Cochrane; John D Halamka; Jonathan A Handler; Brian R Holroyd; Donald Kamens; Abel Kho; James McClay; Jason S Shapiro; Jonathan Teich; Robert L Wears; Saumil J Patel; Mary F Ward; Lynne D Richardson Journal: Ann Emerg Med Date: 2010-04-03 Impact factor: 5.721
Authors: Pablo Alonso-Coello; Affan Irfan; Ivan Solà; Ignasi Gich; Mario Delgado-Noguera; David Rigau; Sera Tort; Xavier Bonfill; Jako Burgers; Holger Schunemann Journal: Qual Saf Health Care Date: 2010-12