Protocol of reporting items for public versions of guidelines: the Reporting Tool for Practice Guidelines in Health Care-public versions of guidelines.

INTRODUCTION: Patient and public versions of guidelines (PVGs) can help with individual decision making and enhance the patient-clinician relationship by providing easily understandable and reliable information. An increasing number of guideline organisations are developing PVGs. However, the reporting of PVGs by different groups and organisations varies widely. This study aims to develop a reporting checklist for PVGs for healthcare. METHODS AND ANALYSIS: We will develop the PVG reporting checklist as an extension of the Reporting Tool for Practice Guidelines in Healthcare (RIGHT) statement. We will build on the methods recommended by the EQUATOR network, which is our starting point. We will conduct a literature review, establish an international multidisciplinary team, run a modified Delphi process to identify the reporting items and pilot test the draft reporting checklist. We plan to update the checklist every 3 years. ETHICS AND DISSEMINATION: Ethics approval and patient consent are not required since this study will not undertake any formal data collection involving humans or animals. The results of this protocol will be submitted to a peer-reviewed journal for publication. TRIAL REGISTRATION: We registered the protocol on the EQUATOR network (http://www.equator-network.org/library/reporting-guidelines-under-development/#84). © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first project on the EQUATOR network to explore how to report a public version of guidelines.

We established an international multidisciplinary consensus working group, including methodologists, content experts and public representatives.

To ensure the completeness of the initial items of interest, we will combine the literature review with feedback from stakeholders. Through the Delphi process and direct discussions, we will finalise the list of most important items.

The list of items might be limited by the lack of extensive literature on patient needs and preferences with respect to PVGs.

Introduction

Informed by a systematic review of evidence and an assessment of the benefits and harms of alternative healthcare options, clinical practice guidelines (CPGs) include recommendations aimed at optimising care delivery and patient outcomes.1 2 CPGs support shared decision-making and help patients participate actively in their care. However, CPGs can be challenging for the public (including patients) to understand. Therefore, guideline-derived materials tailored to the public’s needs and health literacy levels help to inform individual healthcare decisions. Also, patients prefer health professionals to share written materials with them.3

Patients are increasingly using the internet to search for health-related information, but the reliability of such information is unclear. Healthcare workers should guide patients to find reliable and accurate information.4 Patient versions of guidelines (PVGs) are ‘documents that “translate” guideline recommendations and their rationales originally produced for health professionals into a form that is more easily understood and used by patients and the public’.5 PVG that are concise and easy to understand can provide reliable information to patients, help with individual decision-making and enhance the patient–clinician relationship. PVG may also help people to be more reassured and confident about their care,5 and patients value the creation of PVGs.6

Many guideline-producing bodies develop PVGs. For example, the National Institute for Health and Care Excellence, the Scottish Intercollegiate Guidelines Network, the American Academy of Neurology, the National Comprehensive Cancer Network, the US Preventive Services Task Force and the Netherlands Association of Posttraumatic Dystrophy all produce PVGs that are freely available.7–11

From 2011 to 2015, the GRADE (Grading of Recommendations, Assessment, Development and Evaluation) Working Group in its Developing and Evaluating Communication strategies to support Informed Decision and practice based on Evidence (DECIDE) project collaborated with members of the public, patients, carers, specialists in health information communication, journalists and guideline developers to develop frameworks for translating evidence into decisions or recommendations. The group examined what the public knew about guidelines, what they expected from PVGs and how information in PVGs should be presented. This work was summarised in the G-I-N Public Toolkit.12 13 The European Commission Initiative on Breast Cancer has already implemented patient versions.14 In spite of all these initiatives, we could not identify a systematic checklist for PVG reporting, and the content, language style and structure of PVGs from different organisations varied significantly.15 Interviews of guideline developers, shared decision-making experts and patient representatives showed a significant concern about the structure and language of PVGs,16 and that patients prefer guidelines with concise content and non-technical language.

The Reporting Tool for Practice Guidelines in Healthcare (RIGHT) Working Group recently developed and published a reporting checklist for practice guidelines.17 Such reporting checklists can promote transparent and rigorous reporting. PVGs would necessitate its own reporting checklist, however, as they differ from practice guidelines for professionals with respect to the target audience, aims, scope and especially the wording and reporting style.5 18 Such checklists are particularly needed because PVGs from different groups and organisations vary considerately in terms of style and content.15 Proper reporting checklist may improve the overall quality of PVGs, thus promote their utility and effectiveness especially for communication between patients and practitioners.

Objective

The aims of our study are to identify and describe currently published literature on the reporting of PVGs; develop essential reporting items for PVGs; and identify the characteristics of high-quality PVGs.

Methods and analysis

Development process

We will use methods recommended by the EQUATOR network as the starting point,19 refer to the methods used in the RIGHT statement and the Checklist for the Reporting of Updated Guidelines (Check-up)17 20 and adapt these methods as appropriate. Table 1 shows the detailed process with the proposed timeline.

Table 1

Planned steps and project timeline

	Steps	Tasks	Responsible groups	Timeline
Stage 1	1	Identify the need for the checklist	RID group; Secretariat	April 2017
	2	Obtain funding	RID group	May 2017
	3	Identify the participants	RID group; Secretariat	May to October 2017
	4	Generate a list of items for consideration in the Delphi process	RID group; Secretariat	October 2017 to July 2018
	5	Run a modified Delphi process23	DCP group; Secretariat	August to November 2018
	6	Present and discuss results of the Delphi process (teleconference)	DCP group; RID group	December 2018
	7	Draft the checklist	RID group	December 2018
	8	Pilot test and examine the validity	RID group; Secretariat	December to January 2019
	9	Develop the guidance statement and publication strategy	RID group; DCP group	January to February 2019
	10	Develop an explanatory document	RID group; DCP group	February to April 2019
	11	Promote checklist endorsement and adherence	RID group; Secretariat	April to June 2019
	12	Develop a website for RIGHT for PVG	Secretariat	June 2019
	13	Translate and adapt the checklist	Relevant stakeholders	July 2019 on
Stage 2	14	Evaluate PVGs developed using RIGHT-PVG	RID group; Secretariat	From 2020 on
	15	Update the checklist	RID group; Relevant stakeholders	Every 3 years

DCP group, Delphi consensus panellists group; PVG, public version of guidelines; RID group, reporting items development group; RIGHT, The Reporting Tool for Practice Guidelines in Healthcare.

1. Identify the need for the checklist

To identify the need for developing this checklist, we searched the published literature on PVGs and found no reporting checklist. We plan to conduct a search of the relevant literature on patients’ opinions about PVGs and other patient materials. The methods for the literature search are outlined in step 4.

2. Obtain funding

This project is funded by the Open Fund of Key Laboratory of Evidence-based Medicine and Knowledge Translation of Gansu Province, Lanzhou, China. This project is also supported by two National Natural Science Foundation of China programme (The development of evidence grading system for Chinese Integrative Medicine, ID: 81503459; The exploration of critical techniques in guideline development of Chinese Integrative Medicine, ID: 81673825). The funders will have no role in the study design, data collection and analysis, writing of the article or the decision to submit it for publication.

3. Identify the participants

The RIGHT PVG Working Group consists of three subgroups: the secretariat, the reporting items development group (RID group) and the Delphi consensus panellists group (DCP group). Detailed information of members in each group can be found in online supplementary file 1.

The secretariat

The seven members of the secretariat, represent skills and experience covering systematic review and meta-analysis, practice guideline development and implementation, practice guideline interpretation, PVG development, knowledge translation and some clinical areas (eg, laboratory medicine, cardiovascular medicine).

The role of the secretariat group is to: recruit experts; collate documents and email information; arrange meetings; and support the RID group.

Reporting items development group

The RID group includes seven members with research interests covering systematic review and meta-analysis, practice guideline development, implementation and evaluation, PVG, reporting guideline development, public health policy and patients’ values and preferences. The role of the RID group is to draft the protocol and potential items; design questionnaires; organise the Delphi panels; collect and analyse data from the panellists; and draft the final report.

Delphi consensus panellists group

Taking into account diversity of language, gender equality and wide geographic representation, we invited 19 individuals to the DCP group, based on a review of the main authors in the field, as well as the RIGHT members (http://www.right-statement.org/home/member). We included experts with technical expertise in guideline development, PVGs, GRADE, knowledge translation, reporting guidelines (including experts of RIGHT statement) and plain language editing. To reflect the views and perspective of users of PVGs, three representatives of the public were also invited to this group. In addition, the literature on what patients want and need from PVG and other patient materials will also be explored.

The role of the RID group is to: review the protocol and provide comments; contribute to the process of item selection; and decide on the number of items to be included in the final checklist.

4. Generate a list of items for consideration in the Delphi process

In order to identify any existing guidance for conducting and reporting PVGs, we conducted a pilot review of the literature on the standards for reporting guidelines and other related methodological articles to refine the search strategies. In the subsequent, formal search, we will focus on potential sources of checklist items in PVGs.

In addition to searching for guidance specifically on PVGs, we will seek information on developing and reporting of other evidence-based patient tools. Search strategies will be developed with the assistance of an information scientist (Junqiao Chen, University of Oxford). We will search PubMed to identify relevant papers. All search results will be screened in duplicate. Due to the variety of names that can be used to describe PVGs, we will use the snowballing method to conduct reference and citation searches.21 The following sources will be collected for possible items. The third type of study will be used to further support the rationale for a checklist for PVGs (step 1):

Guidance, handbook or studies about the methodology and/or reporting of PVGs.

Guidance or handbooks about the methodology and/or reporting of other evidence-based patient materials, such as decision aids.

Original studies of patients’ needs about the content and/or reporting information of PVG or similar evidence-based patient tools, which could contribute to the checklist from the perspective of patients and the public.

Published PVGs were identified from the study by Santesso and colleague,22 as well as through the reference lists and Google search engine.

By April 2017, we had identified 22 organisations or groups (table 2) who developed PVGs. If the same organisation had developed multiple PVGs using the same methodology and reporting style, we will use convenient sample of one to two PVGs to analyse their reporting characteristics.

Table 2

Organisations that have developed PVGs

No.	Name
1.	American Academy of Neurology (AAN)
2.	American College of Physicians (ACP)
3.	American Cancer Society (ACS)
4.	American Gastroenterological Association (AGA)
5.	American College of Gastroenterology (ACG)
6.	American Society of Clinical Oncology (ASCO)
7.	American Urological Association, Urology Care Foundation
8.	Australia and New Zealand Stillbirth Alliance
9.	British Columbia Medical Association (BCMA)
10.	European Society for Medical Oncology (ESMO)
11.	National Comprehensive Cancer Network (NCCN)
12.	National Institute for Health and Care Excellence (NICE)
13.	Queensland Government
14.	Royal College of Obstetricians & Gynaecologists (RCOG)
15.	Scottish Intercollegiate Guidelines Network (SIGN)
16.	The Endocrine Society
17.	US Preventive Services Task Force
18.	Association of Breast Surgery (ABS)
19.	Cancer Council (CC)
20.	European Society of Human Reproduction and Embryology (ESHRE)
21.	Urology Care Foundation
22.	Canadian Diabetes Association

At this step, all the literature screening and data abstraction will be done by two independent reviewers, and any discrepancy will be solved by discussion.

This step will provide a list of potential items for reporting in a PVG, and the RID group will discuss the items one by one to refine them.

5. Modified Delphi process

To achieve consensus on which items from the list developed in step 4 should be included in the final reporting tool, we will conduct three rounds of a modified Delphi survey,23 24 using a 7-point scale for expressing agreement with each potential reporting item (see figure 1). We will use SurveyMonkey (https://www.surveymonkey.com/) to gather these data. The scale and definitions of agreement and consensus are found in table 3. The process is described in box 1.

Figure 1

The 7-point Likert scale. Not important at all: you are confident that the item should be excluded; Extremely important: you are confident that the item should be included in the final checklist. 1∼7: the important increase in sequence.

Table 3

Definition of consensus

Definition of consensus
Definition of agreement with an item	When 75% or more of participants choose 6–7 on the Likert scale
Definition of disagreement with an item	When 75% or more of participants choose 1–3 on the Likert scale
Definition of ambivalence towards an item	When 75% or more of participants choose 4–5 on the Likert scale
Definition of no consensus within the group	All other types of responses

Analysis of responses: Items with ‘agreement’ are included in the final version and removed from subsequent rounds; Items with ‘disagreement’ are removed; Items which are rated as ‘ambivalent’ or where there is no consensus will be modified to reflect points raised by the DCP group and included in the next Delphi round.

Purpose of the questionnaire

To generate a list of items for consideration based on results of the literature review and analysis of current PVGs.

First round

The following will be sent by email to the Delphi panellists:

An introductory letter and background material including current data and research about proposed items.

The link to the online questionnaire.

A reminder letter and a subsequent telephone call will be made to non-responders after 1 week.

Second round

The following will be sent by email to panellists:

Thank you letter and instructions.

Feedback from panellists and the RID group.

The link to the online questionnaire including items on which consensus was not reached.

A reminder letter and a subsequent telephone call will be made to non-responders after 1 week.

Third round

The following will be sent by email to panellists:

Thank you letter and instructions.

Feedback from panellists and RID group.

The link of the questionnaire including items on which consensus was not reached.

A reminder letter and a subsequent telephone call will be made to non-responders after 1 week.

Only completed questionnaire will be included in the analysis.

During the first round, the panellists will have the opportunity to suggest items that were not included in the initial checklist. Panellists will not be asked to comment on the reasons for including or excluding items after the first round process. In each round, we will include a free text box for suggestions to modify the items or to provide comments. To minimise potential biases, the responses will be analysed anonymously by a biostatistician who is not a member of the panel and who will be blinded to the identities of panel members. After the three-round consensus survey, we will get a checklist of included reporting items for PVGs, which will be discussed in step 6.

6. Present and discuss results of the Delphi process (teleconference)

We will have a teleconference including representatives of the RID and DEP groups to present and discuss the results of the Delphi process, in order to refine the items as indicated.

7. Draft the final checklist

Based on the results of Delphi surveys and teleconference, we will draft the final checklist and send it to the representatives of the RID and DEP groups for their review in order to ensure the accuracy and correctness.

8. Pilot test and examine the validity

The checklist from step 7 will be applied in two to four PVGs that are under development. The developers of these PVGs will be invited to test the checklist, and a questionnaire will be designed for them to seek their comments to refine the checklist. We will also test validity of the checklist through analysing the PVGs. We will randomly select one to two PVGs from each organisation, and two reviewers will use the checklist to assess how well each item was reported (not reported, partially reported, adequately reported or not applicable) in the PVGs.25 The reviewers will also be invited to provide comments and feedback on the ease of use and the completeness and wording of each item.

9. Develop the guidance statement and publication strategy

Based on the pilot and feedback, we will draft the guidance statement and submit it for publication in a peer-reviewed journal. All published documents will be available open access on the RIGHT website (http://www.right-statement.org/).

10. Develop an explanatory document

The RID group will develop a detailed justification and explanation document for the essential reporting items to inform and educate users and facilitate implementation of the checklist. The DCP group will be invited to review and provide comments.

11. Encourage endorsement and adherence

We will disseminate the statement through the following channels:

Submit the report and checklist for potential endorsement by EQUATOR, the GRADE working group, Consolidated Standards of Reporting Trials Group, Cochrane, International Society for Evidence-Based Healthcare the Guidelines International Network (G-I-N) and other relevant networks.

Present the results in international academic conferences, such as the G-I-N annual conference.

Send the statement to interested guideline developers around the world.

12. Develop a website for RIGHT-PVG

As a key implementation strategy, we will create a website for RIGHT-PVG where the checklist will be made available and users can send feedback. We will also link the web site with the EQUATOR Network site.26

13. Translate and adapt checklist

We will welcome and collaborate with other PVG developers who want to translate or adapt this tool for specific contexts.

14. Evaluate PVGs developed using RIGHT-PVG

After releasing RIGHT-PVG, its use will be continuously monitored and its impact evaluated (stage 2). The reporting quality of PVGs will be assessed with RIGHT-PVG. This will help to identify gaps between the current reporting items and practice, thus help with the update of RIGHT-PVG. In addition, the end-users (eg, patients) of PVGs reported referring to the RIGHT-PVG will also be invited to give their feedback on how well the information can meet their need.

Update the checklist

We will review the reporting checklist every 3 years, revising it as indicated, taking into account feedback on the checklist as well as new information and publications in the scientific literature.

Patient and public involvement

We will not directly involve patients in our study, but research about the views and preferences of patients will be analysed to inform the potential items. Three public representatives will act as consensus experts, and they will provide their comments on the reporting items. They will be listed as co-authors in the final publication.

Discussion

The main outcome of this work is a checklist of reporting items for PVGs and an explanatory document. The RIGHT-PVGs will directly address what and how the information should be reported in PVGs. Problems of reporting will be identified through the analysis of existing PVGs from different organisations or groups. Then, together with the literature review and feedback from stakeholders, we will collect all potential items of interest. The Delphi process and teleconference will help finalise items of most importance for reporting PVGs.

The methodology of RIGHT for PVGs will build on the experience and knowledge of the RIGHT Working Group and the Check-up group, that have so far focused on the development and implementation of reporting in practice guidelines. RIGHT for PVGs is one of extensions of the RIGHT checklist17 and is approved by the RIGHT Working Group. The RID group will strictly oversee the process, and appropriate methods will be applied in each step to minimise potential biases.

We will collaborate with guideline developers to implement RIGHT for PVGs, evaluate the utility and obtain feedback so that we can keep the checklist up to date. Several groups of people will be able to benefit from RIGHT for PVGs, including guideline developers, patients and the public, journal editors, and practitioners.

Ethics and dissemination

Ethical issues

Ethics approval and patient consent are not required since this study will not undertake any formal data collection involving humans or animals. The results of this protocol will be submitted to a peer-reviewed journal for publication.

Publication plan

The final report of this project will be submitted to a peer-reviewed journal for publication.