Melissa C Brouwers1, Karen Spithoff2, Kate Kerkvliet2, Pablo Alonso-Coello3, Jako Burgers4, Francoise Cluzeau5, Beatrice Férvers6, Ian Graham7, Jeremy Grimshaw7, Steven Hanna2, Monika Kastner8, Michelle Kho9, Amir Qaseem10, Sharon Straus11, Ivan D Florez12. 1. University of Ottawa, Ottawa, Ontario, Canada. 2. McMaster University, Hamilton, Ontario, Canada. 3. Iberoamerican Cochrane Centre, Biomedical Research Institute Sant Pau (IIB Sant Pau-CIBERESP), Barcelona, Spain. 4. Dutch College of General Practitioners, Utrecht, the Netherlands. 5. Imperial College London, St Mary's Hospital, London, United Kingdom. 6. Département Cancer et Environnement, Centre Léon Bérard, Lyon Cedex 08, France. 7. Ottawa Hospital Research Institute, University of Ottawa, Ottawa, Ontario, Canada. 8. North York General Hospital, Toronto, Ontario, Canada. 9. Institute of Applied Health Sciences, McMaster University, Hamilton, Ontario, Canada. 10. American College of Physicians, Philadelphia, Pennsylvania. 11. Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto, Ontario, Canada. 12. Department of Pediatrics, University of Antioquia, Medellín, Colombia.
Abstract
Importance: Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined. Objective: To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations. Design, Setting, and Participants: A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users. Exposures: Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey. Main Outcomes and Measures: Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach α, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters. Results: A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach α = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76). Conclusions and Relevance: This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.
Importance: Clinical practice guidelines (CPGs) may lack rigor and suitability to the setting in which they are to be applied. Methods to yield clinical practice guideline recommendations that are credible and implementable remain to be determined. Objective: To describe the development of AGREE-REX (Appraisal of Guidelines Research and Evaluation-Recommendations Excellence), a tool designed to evaluate the quality of clinical practice guideline recommendations. Design, Setting, and Participants: A cross-sectional study of 322 international stakeholders representing CPG developers, users, and researchers was conducted between December 2015 and March 2019. Advertisements to participate were distributed through professional organizations as well as through the AGREE Enterprise social media accounts and their registered users. Exposures: Between 2015 and 2017, participants appraised 1 of 161 CPGs using the Draft AGREE-REX tool and completed the AGREE-REX Usability Survey. Main Outcomes and Measures: Usability and measurement properties of the tool were assessed with 7-point scales (1 indicating strong disagreement and 7 indicating strong agreement). Internal consistency of items was assessed with the Cronbach α, and the Spearman-Brown reliability adjustment was used to calculate reliability for 2 to 5 raters. Results: A total of 322 participants (202 female participants [62.7%]; 83 aged 40-49 years [25.8%]) rated the survey items (on a 7-point scale). All 11 items were rated as easy to understand (with a mean [SD] ranging from 5.2 [1.38] for the alignment of values item to 6.3 [0.87] for the evidence item) and easy to apply (with a mean [SD] ranging from 4.8 [1.49] for the alignment of values item to 6.1 [1.07] for the evidence item). Participants provided favorable feedback on the tool's instructions, which were considered clear (mean [SD], 5.8 [1.06]), helpful (mean [SD], 5.9 [1.00]), and complete (mean [SD], 5.8 [1.11]). Participants considered the tool easy to use (mean [SD], 5.4 [1.32]) and thought that it added value to the guideline enterprise (mean [SD], 5.9 [1.13]). Internal consistency of the items was high (Cronbach α = 0.94). Positive correlations were found between the overall AGREE-REX score and the implementability score (r = 0.81) and the clinical credibility score (r = 0.76). Conclusions and Relevance: This cross-sectional study found that the AGREE-REX tool can be useful in evaluating CPG recommendations, differentiating among them, and identifying those that are clinically credible and implementable for practicing health professionals and decision makers who use recommendations to inform clinical policy.
Authors: Chelsea Leversedge; Meghan McCullough; Luis Miguel Castro Appiani; Mùng Phan Đình; Robin N Kamal; Lauren M Shapiro Journal: World J Surg Date: 2022-10-10 Impact factor: 3.282
Authors: John K Noyahr; Oana A Tatucu-Babet; Lee-Anne S Chapple; Christopher Jake Barlow; Marianne J Chapman; Adam M Deane; Kate Fetterplace; Carol L Hodgson; Jacinta Winderlich; Andrew A Udy; Andrea P Marshall; Emma J Ridley Journal: Nutrients Date: 2022-06-23 Impact factor: 6.706
Authors: Jan W van der Scheer; Matthew Woodward; Akbar Ansari; Tim Draycott; Cathy Winter; Graham Martin; Karolina Kuberska; Natalie Richards; Ruth Kern; Mary Dixon-Woods Journal: BMC Med Res Methodol Date: 2021-05-11 Impact factor: 4.615
Authors: Ivan D Florez; Melissa C Brouwers; Kate Kerkvliet; Karen Spithoff; Pablo Alonso-Coello; Jako Burgers; Francoise Cluzeau; Beatrice Férvers; Ian Graham; Jeremy Grimshaw; Steven Hanna; Monika Kastner; Michelle Kho; Amir Qaseem; Sharon Straus Journal: Implement Sci Date: 2020-09-18 Impact factor: 7.327