The AGREE Reporting Checklist: a tool to improve reporting of clinical practice guidelines.

AGREE II has become an internationally accepted standard for evaluation of the methodological quality of clinical practice guidelines

AGREE II items and criteria have been reformatted to create a reporting guide called the AGREE Reporting Checklist

The AGREE Reporting Checklist is intended to improve the comprehensiveness, completeness, and transparency of reporting in practice guidelines

The AGREE Reporting Checklist can be used by practice guideline developers, guideline users, funders, peer reviewers, and journal editors

The international Appraisal of Guidelines, Research and Evaluation (AGREE) research team developed a tool to assess the methodological quality of practice guidelines—the original was released in 2003 (AGREE),1 and the revised and updated version in 2009 (AGREE II).2 AGREE II has become a widely used standard for evaluating the methodological quality and transparency of practice guidelines internationally.3 It was also designed to inform development and reporting requirements for practice guidelines, but these functions have often been disregarded, in part because its formatting and presentation are geared more towards use as an evaluation tool. Similarly, other tools and methods designed to guide the development of high quality practice guidelines (for example, GRADE,4 IOM Standards,5 G-I-N Standards,6 and Guidelines 2.07) are not designed specifically to guide the reporting of practice guidelines and are not generally used as such.

The EQUATOR Network library (www.equator-network.org), a database of health research reporting guides, does not include any reporting checklists or guides specific to the reporting of practice guidelines. In 2003 Shiffman and colleagues released a proposed reporting framework for practice guidelines called the Conference on Guideline Standardization (COGS) checklist, based on existing resources and consensus of the COGS panel.8 This was a fundamental step forward and an asset to the guideline enterprise in providing a minimum reporting dataset for practice guidelines. However, the COGS checklist does not include key items included in the AGREE II evaluation tool that guideline users, guideline developers, and researchers have more recently deemed to be important quality aspects of practice guidelines, and its consensus process included participants from the United States, Canada, and United Kingdom only. For these reasons, the AGREE research team has developed a resource based on AGREE II, called the AGREE Reporting Checklist, specifically to facilitate reporting of practice guidelines. The AGREE Reporting Checklist is a contemporary resource based on a comprehensive review of the literature and consensus among a wider international team of practice guideline stakeholders. This article introduces the AGREE Reporting Checklist, describes its development, and outlines how it can be used.

AGREE Reporting Checklist development methods

To create the AGREE Reporting Checklist, we used the health research reporting development standards proposed by Moher and colleagues.9 Through the process of creating the original AGREE instrument and AGREE II and testing of the AGREE Reporting Checklist draft, all of the requirements in these standards were accomplished.9

The content for the checklist originated from the item generation and reduction stages used to create the original AGREE instrument. Here, a small working group of practice guideline experts reduced a list of 82 candidate items related to quality of practice guidelines, derived from existing appraisal instruments and relevant literature, to 34 items, and these were reduced further with feedback from international experts.1 10 Field testing and validity testing led to the development of the original 23 item AGREE instrument; each item had a definition and specific evaluation criteria to reflect the components of the concept.

In the development of AGREE II, modifications were made and an international sample of users and developers of practice guidelines rated all items as important determinants of the quality and usefulness of a guideline (study 111) and concluded that the instructions were appropriate and easy to apply (study 212). Moreover, the AGREE II domains were also found to be positive significant predictors of the overall quality of practice guidelines, and, with the exception of one, the domains were positive significant predictors of the endorsement of practice guidelines and users’ intention to use the recommendations.11

Across these studies, more than 200 international users and developers of practice guidelines participated. Together, these data show the importance and utility of each of the AGREE II items. For the concepts behind the AGREE II items to be evaluable, a practice guideline developer keen on creating a high quality guideline would need to consider how to embed the ideas in the development process and how to report the information in the guideline itself. Thus, using the content of AGREE II as its foundation, we created the first draft of the AGREE Reporting Checklist. We adapted the evaluation criteria for each item in AGREE II to create reporting criteria. We excluded redundant information and refined the language to match the style of a reporting guide.

To ensure the transferability and adaptability of the AGREE content from an evaluation style manual to a reporting checklist, 15 people with varied experience in developing practice guidelines evaluated the AGREE Reporting Checklist. On a five point scale, participants agreed that its structure was logical (mean=4.6), that its layout allowed for easy application (4.3), and that it facilitated the inclusion of information considered important in the reporting of practice guidelines (4.6). Thirteen respondents reported that they would use the AGREE Reporting Checklist, 14 reported that the appropriate amount of detail had been included in its items, 13 indicated that it would be useful for both new and experienced practice guideline developers, and most reported it would help to remind them about important details to include in their documents.

Scope

The AGREE Reporting Checklist incorporates the content of AGREE II to outline the reporting standards needed to achieve a high quality practice guideline. It comprises 23 items (each with specific reporting criteria) in six domains (see web appendix); the structure and design align with AGREE II. Authors of practice guidelines can use the AGREE Reporting Checklist prospectively during the drafting and final editing stage, to ensure that all necessary information is included, and retrospectively after the guideline is completed, as a quality assurance step. Organizations that develop practice guideline are encouraged to refer to the AGREE Reporting Checklist in their manuals and protocols. Practice guideline funders and research institutions may promote or mandate adherence to the AGREE Reporting Checklist as a means to optimize transparency of required guideline content and to reduce duplication. Finally, journal editors could use the AGREE Reporting Checklist to recommend reporting expectations of submitted manuscripts. The AGREE Reporting Checklist is sufficiently universal that it can be used by practice guideline stakeholders regardless of the more specific protocols or methods used to support guideline development (for example, GRADE,4 IOM Standards,5 G-I-N Standards,6 Guidelines 2.07).

Availability

The AGREE Reporting Checklist is available on the AGREE Enterprise website, a free and open access resource to support the practice guideline field (www.agreetrust.org), and it will be submitted to the EQUATOR Network website (www.equator-network.org/). The AGREE Enterprise website includes access to all AGREE related products, information about AGREE related research activities, the Guidelines Resource Centre, and the My AGREE PLUS interactive online guideline appraisal platform. EQUATOR is an international initiative that seeks to improve the value of published health research literature by promoting transparent and accurate reporting and wider use of robust reporting guides.