Developing WHO rapid advice guidelines in the setting of a public health emergency.

OBJECTIVES: We describe newly established guidance for guideline developers at the World Health Organization (WHO) on the process and procedures for developing a rapid advice guideline in the context of a public health emergency (e.g., the 2014 Ebola epidemic). STUDY DESIGN AND
SETTING: We based our approach on established rapid review methods, which were incorporated into existing WHO guideline development processes. Guidance was further informed by in-depth discussions of issues related to rapid guideline development with WHO staff (n = 6), who oversee the Organization's response to emergencies.
RESULTS: We discuss criteria for considering if a rapid advice guideline is appropriate and feasible and outline the roles of various contributors across the phases of development. Further, we describe the methods and steps involved in performing rapid reviews, which are more fluid and iterative than for a standard guideline process. In general, rapid advice guidelines involve a shorter timeline, narrower scope, and the use of abbreviated methods for the evidence review.
CONCLUSION: Important differences exist between developing a standard guideline and a rapid advice guideline. However, the core principles for WHO guidelines apply to rapid advice guidelines including minimizing bias, applying transparent processes and the use of explicit methods.

Rapid advice guideline processes used in other contexts have been tailored to WHOs mandate to produce high-quality guidelines in the context of public health emergencies.

The principles underlying rapid advice guidelines are the same as for standard guidelines.

All steps should be tailored to the situation, and some can be appropriately abbreviated from standard processes.

This paper describes the considerations that are relevant to deciding if a rapid advice guideline should be developed in the context of a public health emergency and outlines the processes and methods for developing such guidelines.

To date, WHO has published two rapid advice guidelines based on this approach, both developed in 2014 in the context of the filovirus (Ebola) outbreak.

It is possible to apply rapid review and rapid advice guidelines methods to complex public health interventions in emergency situations where the end users may be very diverse.

WHO has a transparent process for producing evidence-based recommendations in the context of public health emergencies.

Further research is needed comparing rapid advice guidelines to standard guidelines with regard to utility, implementation, and health impact.

Introduction

The World Health Organization (WHO) produces global guidelines for the 193 Member States of the United Nations. WHO defines guidelines broadly, as “any document developed by the WHO containing recommendations for clinical practice or public health. A recommendation tells the intended end user of the guideline what he or she can or should do in specific situations to achieve the best health outcomes possible individually or collectively” [1]. Each guideline developed by WHO (or any organization) needs to best fit the intended purpose and meet the end users' needs, and this determines the methods, resources, and timeline for development, dissemination, and implementation.

The WHO Handbook for Guideline Development, 2nd Edition (2014) (“WHO Handbook”), outlines four main types of guidelines: standard guidelines, consolidated guidelines, interim guidelines, and guidelines developed in response to a public health emergency or urgent need such as a natural disaster, warfare, biologic or chemical exposures, or an unforeseen disease epidemic (Box 1 ) [1]. In the context of such emergencies, WHO must at times provide global leadership within hours to days. Such guidelines are termed as “emergency (rapid response) guidelines” and processes and methods for producing such guidelines are currently under development at WHO [1]. However, if a public health emergency continues and as response efforts evolve into recovery and rebuilding, guidelines are needed that are developed using more rigorous methods and generally with a somewhat longer timeline: perhaps 1 to 3 months. These are termed “rapid advice guidelines” and are the focus of this paper.

There are four main types of guidelines produced by WHO that comprise a broad spectrum of products that vary mainly in terms of the following features:

Purpose;

Scope;

The point in time at which the guideline is being developed relative to the life span of an intervention;

The organizations or entities developing the guideline;

The presence in the guideline of new vs. previously published recommendations; and

The timeline.

Standard guidelines

Purpose—to provide recommendations on a specific topic or condition;

Scope—focused or comprehensive;

Developer—WHO technical staff;

New or existing recommendations—usually new; may contain existing recommendations if they have been evaluated and updated as appropriate;

Development period—6 months to 2 years.

Consolidated guidelines

Purpose—to aggregate all the existing guidance on a disease or condition;

Scope—comprehensive;

Developer—WHO technical staff;

New or existing recommendations—existing recommendations that have been evaluated and found to be up to date; may contain some new recommendations;

Development period—1 to 2 years.

Interim guidelines

Purpose—to provide guidance when new interventions, exposures, or diseases arise or when new evidence becomes available or data are likely to be incomplete;

Scope—focused;

Developer—WHO technical staff;

New or existing recommendations—new;

Development period—6 to 9 months.

Guidelines in response to an emergency or urgent need

There are two basic types:

Emergency (rapid response) guidelines

Purpose—produced when public health emergencies may necessitate a response from WHO within hours to days;

Further guidance on this type of guideline is under development at WHO.

Rapid advice guidelines

Purpose—to meet an emergent or urgent public health need when the short timeline mandates a modified process;

Scope—focused;

Developer—WHO technical staff;

New or existing recommendations—usually new; may contain existing recommendations if they have been evaluated and updated as appropriate;

Development period—usually 1 to 3 months.

Abbreviation: WHO, World Health Organization.

Alt-text: Box 1

Rapid advice guidelines must meet minimum standards, the recommendations must be based on evidence and they are subject to an internal quality control and assurance process. The process and methods used for their development may be modified from those of standard guidelines, to meet the accelerated timeline necessitated by Member States' needs in the context of the emergency [1].

The objective of this paper is to describe the criteria that WHO staff use to assess the need for developing a rapid advice guideline and to outline the steps and methods for developing such a guideline in the context of a public health emergency. This article is a synopsis of more detailed methods for developing rapid advice guidelines, which can be found in the WHO Handbook on Guideline Development (2nd Edition, 2014) [1] (see Chapter 11).

Our purpose is to advance transparency of WHOs guideline development process and to provide external organizations with a description of an approach that might be useful and applicable to their contexts. To date, WHO has published two rapid advice guidelines based on this approach, both developed in 2014 in the context of the filovirus (Ebola) outbreak: one examining hand hygiene and the use of chlorine; [2] the other on the effectiveness of various components of personal protective equipment for healthcare workers [3]. Although we began development of methods specific to WHO rapid advice guidelines before the Ebola crisis emerged, it is a compelling example of how we were able to apply these newly developed methods and finalize them based on lessons learned in the context of Ebola (see Box 2 ).

Personal protective equipment in the context of filovirus disease outbreak response: Rapid advice guideline (2014). World Health Organization.

Context

A public health emergency of international concern.

Issue

Healthcare workers caring for individuals with Ebola were at an increased risk of contracting Ebola virus disease during the outbreak in West Africa starting in 2013.

There was uncertainty in the field as to the most effective types of personal protective equipment (PPE).

Development of a WHO rapid advice guideline

A rapid review (RR) was conducted over 7 weeks to inform the recommendations [4].

Initially, the RR focused on the comparative effectiveness and disadvantages of PPE (gloves, gowns, and face protection) for healthcare workers working with Ebola patients. However, only noncomparative studies were identified.

Concurrent with the RR, a survey of values and preferences was administered to expatriated healthcare workers over a 3-week period, which helped to inform recommendations.

The noncomparative data from the RR, the survey data, and information from experts in virology and bloodborne pathogens and materials science formed the basis for the recommendations which were formulated at an expert meeting.

Significance

Produced over a 12-week time frame [3], this marked the first rapid advice guideline produced by WHO following the approaches outlined herein.

Abbreviation: WHO, World Health Organization.

Alt-text: Box 2

WHO has issued guidelines labeled as “rapid” in the past. A 2007 publication describes the production of a guideline where most all of the standard steps and methods were executed within an 8- to 10-week time frame [5]. Since that publication, the term “rapid” has been used in the title of several WHO guidelines. However, none of these guidelines was, in fact, produced rapidly, and none reported using unique or modified approaches: rather they described standard approaches in the context of efforts to produce the guideline rapidly. The current work builds on that prior work, focusing on when standard guideline methods can and should be abbreviated to meet the needs of Member States in a timely manner.

Methods

This guidance on rapid advice guideline development is based upon an existing rapid review approach, which was modified to meet WHOs needs and to allow integration with the organization's existing approach to developing standard guidelines. The rapid review approach [6] and typology [7] were developed by the Knowledge Synthesis Group at the Ottawa Hospital Research Institute. This approach consists of an eight-step process based upon widely accepted systematic review methods, particularly those of the Cochrane Collaboration [6]. This approach has been used to develop rapid reviews for a variety of types of decision makers and has undergone modifications as needed to optimize the approach.

In addition, this guidance on how to develop rapid advice guidelines was informed by discussions with WHO staff involved in emergency response (n = 6), including staff from the Global Influenza Programme, Department of Food Safety and Zoonoses, Global TB programme, HIV Department, Emergency Risk Management Department, and the WHO Headquarters Library. The primary purpose of these informal dialogues was to become more familiar with the current WHO guideline process and to understand staff roles, experiences, and needs with regard to development of rapid advice guidelines. This was not considered research as there was no formal structure to the discussions and no data collection, analysis, or reporting. These discussions therefore did not require research ethics approval.

Assessing the need for a rapid advice guideline

The first step when planning the development of any guideline, including a rapid advice guideline, is to search for relevant, high-quality existing guidelines. If such a guideline already exists, it may be adopted or adapted by WHO staff at headquarters or in regional or country-level offices or at the subnational and facility level. However, if no relevant guidelines are identified, there are a number of important considerations when deciding to develop a rapid advice guideline vs. a standard guideline or to defer development of a guideline altogether.

What is the type of emergency and the risk to public health?

The first step is to examine the public health event that is driving the request for a rapid advice guideline. Emergencies may be classified as natural, technological, or conflict related and may be of sudden onset (e.g., earthquakes, tsunamis, chemical crises) or more gradual onset (e.g., deteriorating situations in armed conflict, progressive disease outbreaks, drought, or food insecurities). All types of emergencies can evolve into protracted situations.

WHO and the Member States of the United Nations use the Rapid risk assessment of acute public health events manual to assess “any outbreak or other rapidly evolving situation that may have negative consequences for human health and requires immediate assessment and action” [8]. Risk is characterized by level and is based on broad descriptive definitions of likelihood and consequences, represented in the form of risk matrices. The WHO Emergency Response Framework describes WHOs roles and responsibilities between the initial alert of an event and its subsequent classification [9]. WHO categorizes emergencies from grade 1 (those with minimal expected public health consequences) to grade 3 (those involving events in one or more countries and having significant public health consequences that call for a substantial regional and/or international response).

Is the event novel?

WHO staff may consider producing a rapid advice guideline in the face of either a new situation (e.g., a new strain of influenza, the Middle East respiratory syndrome coronavirus, or an earthquake) or an event encountered previously but causing problems in a different context (e.g., a change in disease pattern such as the Ebola virus disease outbreak in West Africa in 2013 or a prolonged armed conflict compounded by a disease outbreak). If the event is not novel, high-quality relevant guidelines may already exist and a new guideline may not be needed.

Does uncertainty need to be urgently addressed?

Guidelines are indicated when there is uncertainty about what to do in a specific situation. WHO staff may be uncertain about what advice to provide or there may be uncertainty in the field, with different stakeholders having different viewpoints and approaches. In determining if a rapid advice guideline is appropriate, the key question is how quickly the uncertainty needs to be dealt with.

What is the anticipated time frame for the event?

Rapid advice guidelines can generally be developed within 1 to 3 months. If an event is likely to persist beyond 6 months, a rapid advice guideline may not be optimal and a standard guideline may be the best approach. On the other hand, if the emergency is likely to be transient, then existing guidelines should be reviewed for applicability, with the production of emergency (rapid response) guidelines as appropriate. It is important to weigh the impact of developing recommendations using standard processes and timelines vs. producing a guideline that may be prone to serious limitations under an accelerated timeline.

Will the rapid advice guideline be rapidly implemented?

Rapid advice guidelines should only be developed if a mechanism is either already in place or likely will be in place for disseminating and implementing the recommendations in the guideline in the context of the emergency. Various factors need to be carefully considered: the existence of functioning health systems; adequate health workforce; necessary infrastructure; the acceptability of the proposed intervention; the training requirements; and resource availability.

Steps to developing a rapid advice guideline

The basic steps for developing a rapid advice guideline are depicted in Table 1, Table 2 and are generally the same as those that apply to standard guidelines. There are, however, some differences and additional considerations when developing a rapid advice guideline.

Table 1

Steps in the development of rapid advice guidelines—phase 1 (planning)

Primary contributor	Step	Key points for rapid advice guidelines
Phase 1. Planning
Member State, WHO country office, or public/private entity	Request(s) for guidance on a topic.	The request is in the context of a public health emergency.
WHO technical unit	Determine if a guideline is needed; review existing WHO and external guidelines.	The technical unit must determine if a rapid advice guideline is needed or if a standard or interim guideline would be more appropriate.
	Discuss the process with GRC Secretariat and with other WHO staff with experience developing guidelines.	The planned guideline is discussed with the Secretariat when it first becomes a possibility.
	Form the Steering Group.	All relevant departments at WHO headquarters and in the regional offices must be involved.
	Identify sufficient resources.
	Determine the timeline.
Steering Group	Draft the scope of the guideline.Begin preparing the planning proposal.	The literature is scoped through a brief review. The guideline's scope must be narrow and feasible.
	Identify potential members of the GDG and the chair.	Issue invitations early; involve the GDG in determining the scope and key questions.
	Obtain DOIs and manage any COIs among potential GDG members.	The process for rapid advice guidelines and standard guidelines is identical.
Steering Group and the Guideline Development Group (GDG)	Formulate key questions in PICO format.Prioritize outcomes.	Key questions (in PICO format) include only those of the highest priority and must be focused and narrow. Background questions are not addressed in a rapid advice guideline.
WHO Steering Group	Finalize the guideline planning proposal.	The process is the same as for a standard guideline.
Guidelines Review Committee (GRC)	Review and approve the planning proposal.	The GRC uses an accelerated process for review and disposition.

Abbreviations: COI, conflict of interest; DOI, declaration of interest; PICO, population, intervention, comparator and outcome; WHO, World Health Organization.

Table 2

Steps in the development of rapid advice guidelines—phase 2 (development) and phase 3 (publishing and updating)

Phase 2. Development
Systematic review (SR) team	Perform SRs of the evidence for each key question with the potential of abbreviating the SR process (i.e., perform an RR).	The contractor needs to be identified from the outset and involved in the scoping and development of key questions: they can advise on what is feasible in the given time frame.
	Evaluate evidence quality for each important outcome, using GRADE as appropriate.	The process is the same as for a standard guideline.
Steering Group	Convene a meeting of the GDG.	Meeting place and participants need to be identified at the beginning of the development process. The meeting has a similar format and agenda as for the development of a standard guideline.
Guideline Development Group (GDG)	Formulate recommendations using the GRADE framework.	The general methods are the same as for a standard guideline. The evidence may be sparse, so other factors that inform the recommendations must be transparent and based on indirect evidence when possible, and on equity, human rights and gender considerations.
Steering Group	Draft the guideline document.	The document should be concise and tailored to the end user.
External review group	Conduct targeted external peer review.	External peer review is recommended for rapid advice guidelines but may not be feasible in some situations.
Phase 3. Publishing and updating
Steering Group and editors	Finalize the guideline document. Perform copy editing and technical editing. Submit the final guideline to the GRC for review and approval.	This step will have to be performed in an accelerated manner. Editorial staff needs to be identified early in the process.
WHO Guidelines Review Committee (GRC)	Review and approve the final guideline.	The GRC uses an accelerated process for review and disposition.
Steering Group and editors	Finalize the layout. Proofread.	This step needs to be accelerated and perhaps abbreviated from the standard processes.
	Publish (online and in print, as appropriate).
WHO technical unit and program manager	Disseminate, adapt, implement, evaluate.
WHO technical unit	Update.	From the outset, the technical unit must consider the likely shelf life of the rapid advice guideline and whether a standard guideline will follow and when.

Abbreviations: GRADE, Grading of Recommendations Assessment, Development and Evaluation; RR, rapid review.

Consult the WHO Guidelines Review Committee Secretariat

Once the relevant WHO technical unit determines that a guideline is needed, the unit contacts the Guidelines Review Committee Secretariat whose remit is to support the WHO Guidelines Review Committee, which is responsible for setting the standards, developing the methods, and assuring the quality of all guidelines issued by WHO [1]. The Secretariat will assist the technical unit in deciding if the topic is suitable for a rapid advice or other type of guideline and will provide technical support if the unit moves ahead with guideline development.

Formulate the various groups involved in developing a rapid advice guideline

When developing a rapid advice guideline, four key groups need to be established quickly. First is the internal WHO Steering Group whose primary responsibility is to oversee the rapid advice guideline development process. Second, the review team will produce a rapid, yet comprehensive and objective synthesis of the evidence to inform each recommendation. A methodologist with expertise in guideline development processes and methods is also identified early in the development process. Third, the external review group contributes diverse and real-world perspectives at the peer-review stage. Fourth, the Steering Group assembles the Guideline Development Group, which provides input on the scope and content of the rapid advice guideline, and is primarily responsible for formulating the recommendations. The Guideline Development Group must include a broad range of relevant clinical and public health technical and programmatic expertise, as well as representation from key stakeholders such as persons who will be affected by the recommendations in the guideline. The Guideline Development Group must have geographic representation from all WHO regions and must be gender balanced to the extent possible (Fig. 1 ).

Fig. 1

Contributors to the development of rapid advice guidelines issued by WHO. GRADE, Grading of Recommendations, Assessment, Development and Evaluation; WHO, World Health Organization.

It is critical to include individuals with expertise in ethical, social, and legal dilemmas on the Guideline Development Group, as well as expertise in issues related to equity, gender, and human rights. Although these issues may be considered by some to be peripheral to the urgent health problem being addressed (e.g., an outbreak of a disease), critical human rights issues often emerge in the context of a public health emergency, and they must be addressed in the initial stages of a response.

Scope the rapid advice guideline and define the key questions

Once the need for a rapid advice guideline has been established, the WHO Steering Group continues to redefine the scope of the guideline and develop key questions in population, interventions, comparators, outcomes (PICO) format. A rapid advice guideline will most likely provide recommendations on the benefits and harms of interventions. However, recommendations on diagnostic tests, prognosis, and risk factors may also be needed.

With the assistance of an experienced information specialist, a scoping exercise should be conducted quickly to provide a general sense of the depth of the relevant literature. This is not a systematic search of all potential sources, but rather a focused search for the best available, relevant literature, including high-quality systematic reviews and key primary studies. The resources most applicable to the topic should be examined briefly (e.g., MEDLINE, The Cochrane Library, Scopus, etc.) in addition to looking for any information or guidance published by WHO in the early stages of the public health emergency. This scoping exercise, including synthesis of the evidence retrieved, should take no longer than 1 or 2 days, and a brief summary of the results should be prepared.

Prepare and maintain the planning proposal

A detailed planning proposal akin to a review protocol should be prepared for all guidelines, including rapid advice guidelines. At WHO, all planning proposals are reviewed by the Guidelines Review Committee, and in the context of a rapid advice guideline, the primary issue for the Guidelines Review Committee is to determine if there is adequate justification for applying an accelerated and abbreviated process. The planning proposal for rapid advice guidelines has the same content, level of detail, and format as for standard guidelines, describing the planned processes and procedures, the results of the scoping review, the methods for the rapid review, and the approach for translating the evidence into recommendations.

The planning proposal serves as a point of reference for all contributors, and therefore, it must be detailed and kept up to date, even when operating under a compressed timeline. This is particularly important as contributors to the guideline may change as WHO staff members are deployed to the field during the guideline development process. As described below, the rapid review process is often more fluid and iterative than that of a standard systematic review, and thus, the planning proposal is a living document, amended as needed, including the rationale for any changes. Complete and accurate documentation ensures transparency and greatly facilitates the drafting of the final guideline document.

Performing rapid reviews and developing summaries of the evidence

What are rapid reviews?

When rapid advice guidelines are deemed necessary, conducting a systematic review de novo may not be feasible. Rapid reviews have emerged as a streamlined approach to identifying and synthesizing evidence, typically for the purpose of assisting expeditious decision making by state and local governments or by healthcare providers. For the purposes of this guidance, we define “rapid review” as a type of evidence review that is produced using accelerated and/or modified systematic review methods [6].

How do rapid reviews compare with systematic reviews?

The core principles of evidence searching and retrieval for standard systematic reviews apply to rapid reviews, including thoughtful scoping and formulation of the review questions, transparency, reproducible methods, careful assessment of the quality of the information incorporated into the review, efforts to minimize bias at every stage, and the clear presentation of information focused on the intended users' needs. However, there are important differences: the rapid review may have a more limited scope and fewer outcomes of interest, more restricted search criteria, looks to existing high-quality systematic reviews as the first line of evidence, involves a more targeted and iterative procedure for screening the literature and for data analysis and synthesis, places less emphasis on meta-analyses, and involves a concise and abbreviated report. In addition, in a rapid review, the search process is more iterative and hierarchical, depending on the findings at each step: the types of publication and study designs included and the bibliographic databases searched may change as the evidence is explored. Other efficiencies may be achieved by, for example, adding more resources so that reviewers can work in parallel.

Types of reviews that underpin rapid advice guidelines may be categorized into two basic types: a standard systematic review performed rapidly or a rapid review involving a variety of abbreviated methods, which may include only existing systematic reviews; primary studies and existing systematic reviews; or only primary studies (Table 3 ) [7].

Table 3

Types of rapid reviews used to inform recommendations in rapid advice guidelines

Types of rapid reviews [7]	Traditional systematic review (conducted rapidly)	Rapid review of systematic reviews	Rapid review of systematic reviews plus primary studies	Rapid review of primary studies only
Time frame	Up to 16 weeks	Up to 12 weeks	Up to 12 weeks	Up to 12 weeks
Methods
Question types	Clinical effectiveness, clinical efficacy; safety/harms; diagnostic or screening test accuracy; cost-effectiveness; health systems, education, public health, policy/programs, or prevention interventions
Number of questions	Multiple (targeted and narrow in scope)	1 primary question (targeted)
Literature search	No restrictions	Restrictions (e.g., date, study design, language, setting)
Number of databases searched	No restrictions (comprehensive)	2–3 databases
Use of systematic reviews	Systematic reviews and primary studies	Systematic reviews only	Systematic reviews plus primary studies	Primary studies only
Gray literature	Yes, as appropriate	Limited (e.g., key web sites)
Screening	2 reviewers	2 reviewers: second reviewer may only review excluded studies at title/abstract phase of screening
Types of study designs included	RCTs and observational studies as appropriate	Systematic reviews and guidelines only (highest quality)	Systematic reviews and guidelines plus RCTs or observational studies (highest quality)	RCTs or observational studies only (highest quality)
Data extraction	Complete verification	Selected verification
Outcomes	Restricted to four critical outcomes or fewer	2–4 critical outcomes only: more if data are available
Assessment of risk of bias at the individual study level	Yes (using validated instruments when available)
Assessment of the quality of the body of evidence	GRADE for critical outcomes as appropriate	Reliance on GRADE as reported in the included systematic review(s); or perform de novo for each systematic review		GRADE for critical outcomes as appropriate

Abbreviations: GRADE, Grading of Recommendations Assessment, Development and Evaluation; RCTs, randomized controlled trials.

Steps in the rapid review process

Select the types of evidence to be collected and identify the appropriate sources

Depending on the nature of the question being asked, the purpose of the rapid review, and the magnitude of the literature on the topic, various types of evidence may be targeted. In most cases, the emphasis will be placed on locating and summarizing evidence from relevant and high-quality “off-the-shelf” systematic reviews or guidelines. In the absence of such systematic reviews, high-quality and/or recent primary studies may be included. Landmark papers may be included for reference, and high-quality quasiexperimental or observational studies may be considered, depending on the key question and the volume of the available evidence.

Usually, no more than two to three of the most relevant databases are searched (e.g., MEDLINE, The Cochrane Library, EMBASE, Scopus). However, depending on the review topic and access to research databases, additional databases including topic specific and regional databases may be examined (e.g., PsychINFO, CINAHL, ERIC, African Index Medicus, International Clinical Trial Registry Platform, ClinicalTrials.gov). A WHO information speciality should be involved in the selection of the priority information sources, as regional databases and local sources may be the richest source of relevant information.

Develop search strategies

In a standard systematic review, the aim is to maximize both recall, which is the ability to identify all relevant articles (sensitivity) and precision, which is the ability to exclude nonrelevant articles (specificity). However, for a rapid review, the aim may be to maximize precision rather than recall. Several common eligibility restrictions should be considered to optimally balance recall and precision (Box 3 ) [1]. Potential restrictions should be discussed among WHO Steering Group members and with the review team information specialist.

Sources

Usually, search no more than two or three key bibliographic databases.

If time and resources permit, additional resources may be added.

Language

Language restrictions are frequently applied, as translation is time consuming and resource intensive.

Limitations by language of publication need to be assessed for each topic, with consideration given to the distribution of the disease or condition being addressed and the likely languages of the relevant publications.

Accessible studies

Publication status is limited to full text only (abstracts are not usually included).

To maximize efficiency, articles should be electronically available through e-journal subscriptions available to the rapid review team.

Articles should be purchased directly from a journal only under special circumstances, namely when the paper is deemed essential and is not available through other means.

Gray literature

The utility of the gray literature is assessed for each topic.

Web sites of relevant organizations may be examined, depending on the subject under review.

Year (search dates)

Publication dates are limited (e.g., only the most recent decade is searched).

When applying a year limit, a rationale for the time frame must be provided.

Region

Restrictions may be placed on the geographical locations of the included studies.

A rationale should be provided to explain why citations from certain regions, rather than from the global literature, are targeted.

Alt-text: Box 3

Search strategies for a rapid review will generally have language restrictions because translation is time consuming. The languages of inclusion should be carefully selected based on the guideline topic. For example, a rapid review on personal protective equipment for health workers in Ebola treatment centers [4], engendered by the Ebola virus disease outbreak that became widespread across parts of West Africa in 2014, included only literature in English and French owing to the geographic distribution of the outbreak and the opinion of experts that most of the relevant literature was in those two languages. Citations in nonselected languages are generally included during the citation screening phase but may be excluded from further analyses if the full text is difficult to access or insufficient time or resources are available for translation.

Search terms should include both medical subject headings (MeSH) and text words. Validated search filters may be useful (see Chapter 8, WHO Handbook [1]), such as those related to study type and design (e.g., randomized controlled trial, systematic review, or meta-analysis). The draft search strategy must be reviewed by at least one other member of the rapid review team, one or more content experts, and a WHO information specialist. A limited search for gray literature should be considered (e.g., relevant data may be quickly identified and retrieved from the web sites of relevant organizations).

The search approach and restrictions used, and their rationale and potential limitations should be reported in the planning proposal, the review report, and the guideline document. A list of potentially relevant citations identified during the search but excluded from the analysis due to language restrictions or other reasons should be included as an appendix in the rapid review report.

Consider other strategies for identifying relevant literature

In the context of a new situation or event, the best (and perhaps only) data might come from the analysis of emerging information in real time. In the Ebola virus disease outbreak in West Africa in 2014, essentially, no relevant data were obtained through a rapid review of the published literature comparing various types of personal protective equipment in the context of Ebola or related viruses [4]. Therefore, a survey of repatriated healthcare workers was rapidly implemented to gather information on experiences with various types of personal protective equipment [10]. If time permits, the reference lists of all included studies should be scanned for additional relevant studies to ensure that key publications have not been overlooked.

Screen and select studies

Standard systematic review methods apply to the process of screening the records retrieved via the searches. Records should be imported into reference management software to facilitate record management and citation screening.

Study selection involves a two-step process. First, either two people independently screen titles and abstracts of all potentially relevant records or one person reviews all titles and abstracts, whereas the second reviewer examines only the citations excluded by the first. Second, two reviewers examine the full-text publications to determine their eligibility. As for a standard systematic review, consensus on the included studies should be achieved, with involvement of a third reviewer if necessary.

To keep the scope of a rapid review within the bounds dictated by timelines and resources, initially, the evidence is often limited to that found in systematic reviews. A decision to include primary studies must be justified in the planning proposal and reflected in the timelines and budget. Further restrictions (e.g., by outcomes or study quality) may be considered to accommodate the inclusion of primary studies.

Records that are not available electronically are generally excluded because the timeline of a rapid advice guideline is not compatible with the delays involved in interlibrary loans. Even if the full text cannot be obtained or translated, the abstract may provide valuable information, particularly when evidence is sparse.

Extract data and synthesize evidence

Once the included studies are finalized for each critical outcome, outcome data can then be extracted, including key study demographics, effect estimates (e.g., odds ratios, mean differences, or summary effect [i.e., a meta-analysis]), and their corresponding confidence intervals. A standard extraction form should be developed and pilot tested to facilitate accurate data collection. Usually, one reviewer extracts data, and a second verifies all extracted data. If this is not feasible, a random sample of at least 10% of the included studies should be independently checked to provide some measure of quality assurance.

The rapid review team will finalize the data analysis plan in consultation with the WHO Steering Group. Quantitative syntheses of primary studies (i.e., meta-analyses) may not be feasible for rapid reviews unless time and resources permit; however, the results of previously published meta-analyses should be reported. Fig. 2 provides details of the various steps and decisions involved in selecting the type of evidence and the approach to data synthesis (see Chapter 11, WHO Handbook [1]).

Fig. 2

Approaches to a rapid review of the evidence. GRADE, Grading of Recommendations Assessment, Development and Evaluation; PICO, population, intervention, comparator and outcome; RR, rapid review; SR, systematic review.

Assess the quality of the body of evidence

The risk of bias should be assessed for each included study to facilitate appropriate interpretation of the review findings. For rapid reviews particularly, the assessment of the risk of bias may be used to select the studies included in the review, once initial criteria based on study design have been applied.

The quality of the body of evidence for each outcome that is critical for decision making should generally be assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework [11]. The focus is on health outcomes and not on intermediate, surrogate, or other types of outcomes. However, exceptions may be made when data are sparse, and decisions may need to be based on indirect evidence, including intermediate outcomes.

Rapid reviews often necessitate the inclusion of existing systematic reviews over primary studies. However, when using an existing systematic review that did not use the GRADE framework to assess the quality of the body of evidence (or which does not supply all of the necessary information for this assessment), it may not be feasible to examine the individual studies included in the review to assess their risk of bias and to develop GRADE profiles de novo. In this case, ROBIS [12], a tool for assessing the risk of bias in systematic reviews (rather than in primary studies) and where appropriate the relevance of a review to the research question at hand, could be applied. Further, to determine a review's quality, A Measurement Tool to Assess systematic Reviews [13] could also be used. Although limited in the ability to assess quality in terms of certainty of the effect estimates, this will help to identify areas of potential concern to help judge overall risk of bias and the quality of conduct across included reviews. If only primary studies are identified, then it will be important to assess risk of bias at the individual study level (See Box 4 ) applying the GRADE framework.

Introduction

Brief description of the rationale for the rapid review and of the context for the guideline.

Duration of the rapid review process (with accompanying dates).

Indication that this is a rapid review and should be interpreted in that light.

Methods

Final key questions in PICO format.

How critical and important outcomes were selected.

Study inclusion and exclusion criteria.

Search strategies and databases searched.

Approach to screening citations and identifying the final set of included studies.

Data extraction process.

Assessment of the risk of bias at the individual study level.

Use of GRADE or other approach to assess the quality of the body of evidence for each critical outcome.

Description of the data synthesis process.

Results

Complete documentation of the search results, including a PRISMA flow diagram [14].

A summary table of results for each key question.

GRADE evidence profiles (or modified versions thereof) for each key question.

Discussion

The strengths and limitations of the review process, focusing particularly on how the methods differed from those of a standard systematic review and the potential risk of bias introduced by the rapid review process.

Future research needs.

Information page

Acknowledgments.

List of authors and collaborators.

How the rapid review should be cited.

Declaration of interests of the report authors.

Sources of funding of the rapid review.

Disclosure statement regarding the limitations of the rapid review process.

Reference list

Appendices (as appropriate)

List of studies fulfilling inclusion criteria, with citations.

List of publications excluded at the full-text screening stage, with citations.

List of non-English language or selected foreign language studies that may fulfill inclusion criteria.

Data extraction tables.

Risk of bias summary tables.

GRADE evidence profiles.

Abbreviations: GRADE, Grading of Recommendations Assessment, Development and Evaluation; PICO, population, intervention, comparator and outcome; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses.

Alt-text: Box 4

Develop the rapid review report

The rapid review report should transparently and succinctly summarize the methods used and the results of the review. Suggested components of the rapid review report are listed in Box 4. The rapid review methods should be reported at a level of detail that will allow them to be replicated by interested organizations and readers. A PRISMA flow diagram [14] gives the reader an overview of the rapid review process and a snapshot of the evidence identified. All rapid reviews should include a narrative summary of the evidence, generally organized around the PICO framework. A brief section on the gaps in the evidence and future research needs may be very useful, particularly when data are sparse. A written disclosure should be provided that the rapid review is not intended to be a gold standard systematic review and that its results should be interpreted with caution and viewed within a specific context.

Formulate recommendations and draft the guideline

The Steering Group needs to plan early for the Guideline Development Group meeting where recommendations will be formulated. Recommendations can be developed via a virtual meeting, although in-person meetings are preferred, even in the context of a rapid advice guideline.

The GRADE approach for formulating recommendations should be followed when developing rapid advice guidelines (see Chapter 10—WHO Handbook [1]). It will seldom be feasible to collect primary data or to perform a review of the resource implications of the intervention or of the values and preferences surrounding the outcomes of interest. However, data that can be readily obtained should be collected (e.g., the cost of gloves in the 2014 guideline on personal protective equipment in the context of Ebola virus disease) [3].

Implementation and the importance of context should also be considered when developing a rapid advice guideline as most research evidence was likely generated in settings and populations that differ from that of the public health emergency at hand. Thus, the degree to which such evidence can be directly applied to the current context may be limited. It is important to consider how contextual factors can modify the benefits and harms of an intervention and how various barriers and facilitators can affect implementation and impact. For a rapid review that relies heavily on evidence from systematic reviews, the synthesis should be tailored to the local context for the emergency throughout all stages of the guideline development process.

The process and resources needed to draft the final rapid advice guideline document are the same as for standard guidelines, and a writer should be identified early and involved throughout the development process. It is particularly important to describe how the rapid advice guideline differs from a standard guideline, and the potential biases that may have been introduced. In addition, the shelf life of the document should be clearly indicated; for example, if the rapid advice guideline constitutes interim guidance because new information is anticipated in the foreseeable future, this should be clearly indicated to the user.

External peer review and publication

A draft of the rapid advice guideline draft should be peer reviewed by key individuals, both internal and external to WHO. Three to six potential peer reviewers should be identified early and their interest, availability, and commitment to a quick turn-around time (e.g., 48–72 hours) discussed. Governmental or nongovernmental organizations that are involved in the public health emergency should also be asked to review the draft document to promote engagement and buy-in during dissemination and implementation and to raise issues and concerns before publication. At an absolute minimum, all relevant departments at WHO and in the regional offices must be given the opportunity to provide substantive input into the final document.

Publication of the final rapid advice guideline involves the same steps as for a standard guideline. Electronic means will usually be used for initial dissemination, followed by print circulation as required in the local context.

Conclusion

WHO must produce high-quality, evidence-informed guidelines in the context of public health emergencies when there are no existing guidelines for Member States to implement. We have outlined the processes and methods by which WHO can produce rapid advice guidelines in this context.

The development of a rapid advice guideline differs in important ways from that of a WHO standard guideline. A rapid advice guideline has a very narrow scope to make development feasible within the given time frame. Moreover, WHO staff and external experts need to be identified and engaged early in the guideline development process, and the Guidelines Review Committee Secretariat should be contacted to put in place the required expedited processes and to provide technical support.

Rapid review methods may differ from those of a traditional systematic review, including constraints in searching bibliographic databases and other sources of information; the need for a more fluid and iterative approach to establishing study inclusion/exclusion criteria, data extraction, and evidence synthesis; and the abbreviated nature of the review report. These differences, in turn, may affect the credibility of the review and the validity of the review's conclusions. Given that interest in rapid reviews has increased and there is great variability in the approaches and level of reporting [15], [16], [17], [18], [19], future research needs to address how rapid reviews compare with standard systematic reviews in terms of bias and credibility, with further guidance developed on when and how to conduct a rapid review. Rapid reviews have become an area of new methodological development for several health research organizations. The US Agency for Healthcare Research and Quality has established a rapid reviews workgroup [20]. Cochrane, the world's largest producer of high-quality systematic reviews of effectiveness, recently established the Cochrane Methods Rapid Reviews Group [21]. Further, due to the increased interest of public authorities and clinicians, the Guidelines-International-Network established a working group dedicated to the methods for developing guidelines in an accelerated time frame [22].

Few data exist on the ways in which rapid advice guidelines are developed and implemented, how they differ from standard guideline development methods, and the impact of rapid advice guidelines on health outcomes. Nevertheless, the core principles and standards for WHO guidelines apply: minimize bias; apply transparent processes and explicit, reproducible methods; acknowledge potential limitations; and attend to the target audience's needs and to the interests of the individuals and populations affected by the recommendations. Applying these principles and meeting these standards in the face of an emergency involves trade-offs, as well as expertise in both guideline development methods and the guideline topic. Further, guideline developers at WHO need to commit to updating these guidelines in a timely manner when new data become available. When warranted, rapid advice guidelines need to be converted to standard ones so that WHO recommendations are robust and the organization is prepared for continuing public health emergencies or for recurrent events.