Improving the Applicability and Feasibility of Clinical Practice Guidelines in Primary Care: Recommendations for Guideline Development and Implementation.

OBJECTIVE: To give recommendations for the development of primary care clinical practice guideline (CPG) to improve applicability and feasibility of primary care CPGs in China.
DESIGN: A two-round Delphi survey.
METHODS: A two-round Delphi survey including guideline development methodologists and clinical practitioners from six countries was conducted. In round one, participants were asked to raise special considerations for the development of primary care CPGs through open-ended questions. In round two, participants were asked to rate the level of agreement on each recommendation item generated by round one and to raise additional recommendations. Opinions from participants were reviewed by thematic analysis. Integrated results from the Delphi survey were validated by participants.
RESULTS: The necessity of developing recommendations for the development of primary care CPGs were consistently recognized by participants. The main recommendations of guideline development were generated as follows: (1) considering the context of primary care institutions and the applicability of existing guidelines for primary care in planning guideline; (2) involving primary care practitioners and patients in guideline groups; (3) considering the variation of health-care resources between primary care settings when developing recommendations; (4) presenting the difference of recommendations between primary care CPG and general CPG; (5) implementing more active education and training; and (6) considering the changing of primary care medical resource when updating guideline.
CONCLUSION: In this study, we present recommendations to inform the development of clinical practice guidelines in primary care settings. Next steps will include merging these recommendations with general guideline development methods to inform the development of guidelines for primary care.

Plain Language Summary

We provide recommendations on nine phases of guideline development for primary care through a Delphi survey.

The two-round Delphi survey based on a systematic search of handbooks for guideline development allowed us to seek opinions and perspectives from stakeholders based on existing methods for guideline development.

The opinions from the Delphi survey were reviewed in parallel by members from the working group, and the consistency of each member’s item reduction was evaluated by an expert member from working group.

Although we tried to cover a wide geographical area by inviting experts from six countries, the applicability of the recommendations may still have limitations when applied worldwide.

Introduction

A good primary health-care system should occupy a central role in health care and take on a gate-keeping role to reduce cost burden arising from uncontrolled and inappropriate use of expensive hospital services.1 However, many countries including China are faced with serious shortage of well-trained primary care practitioners and have a deeply entrenched habit of seeking help from tertiary hospitals due to lack of public confidence in the primary health-care system.2,3

Appropriate clinical practice guidelines can assist practitioners and patient decisions on adequate health care for specific clinical circumstances.4 Our previous survey, however, revealed a huge gap between the needs and the availability of appropriate clinical practice guidelines (CPGs) for primary care practitioners in China.5 Similar result was found in previous study from other country, current CPGs did not meet the clinical needs in primary care.6

Although guidelines on guideline development and implementation provide straightforward pathways and methods, the applicability and feasibility of CPGs for primary care are still problematic.16–18 An assessment of guidelines on hepatitis B at primary care level found that none of the included guidelines met all criteria of appropriateness for remote primary health-care settings, indicating that guidelines need to recognize the difficulties of rural and remote practice, and hence to present practical alternatives to urban-centred recommendations.19 Even though guidelines are appropriate for primary care, the dissemination and implementation still need to be improved.7 A cross-sectional study from Sweden showed that 42% of clinicians in primary care were unfamiliar with the content of evidence-based guidelines,20 and guideline adherence in primary care settings was poor.21–23

This study aims to identify specificities and further recommendations to improve the applicability, feasibility and implementation of CPGs in primary care in China by consultation with guideline development methodologists and clinical practitioners based on systematic review of previous guidelines of guideline development.

Methods

We used a web-based Delphi survey and a validation check to achieve formal group consensus, maximizing dialogue through structured feedback. To facilitate the survey, a working group including methodologists, clinicians and pharmacists was established in January 2016. As the first step, the working group conducted a comprehensive search of previous guidelines and handbooks of guideline development and implementations to identify common flows and methods in guideline development and implementation.8 Then, the working group held two face-to-face meetings to raise potential specialties in primary care CPG development and implementation, and formed a questionnaire for the Delphi survey (Figure 1). The questionnaire included 41 open-ended questions on steps of guideline development and implementation including planning guideline, setting up guideline group, declaration and management of interests, formulating questions, choosing outcomes, evidence retrieval, evidence assessment, developing recommendations, producing and publishing guideline, implementation and evaluation, and updating guidelines.

Figure 1

Overview of consultation to develop process recommendations and guiding principles for primary care CPG development.

As the second step, a two-round Delphi survey of guideline development methodologists and clinical practitioners was conducted.9,10 For selection of the sample, the working group generated a list of candidates for Delphi survey through the corresponding author information of previous guidelines and handbooks of guideline development and implementations,8 and inquired whether the authors’ were willing to participate by email. A maximum variation sampling approach was used with the balance of expert region and professional background and final sample size was based on reaching data saturation. In the first-round Delphi survey, participants were asked firstly to rate the applicability of previous guidelines, as well as the necessity of developing recommendations for primary care CPGs by a five-level Likert scale (1, completely inapplicable/unnecessary, to 5, completely applicable/necessary). Then, participants were asked to raise special considerations and recommendations for primary care CPGs following the questionnaire. Opinions from participants were reviewed in parallel by two members of the working group (LNZ and YJD) by steps as follows: (1) separating compound responses into individual items; (2) line-by-line coding each item and summarizing into concise items; and (3) categorizing concise items by steps of guideline development and implementation. An expert in the working group (LLZ) then evaluated the consensus of each member’s item reduction and led a discussion to achieve consensus in case of discrepancies.

In the second-round Delphi survey, the participants were asked to rate the level of agreement on each recommendation generated by the first-round survey using a five-level Likert scale (1, strongly disagree, to 5, strongly agree). Participants were encouraged to explain on disagreement or to raise additional recommendations. The working group only included items with an average score of ≥4. Mirroring the process in the first-round, explanations and additional recommendations were summarized and categorized.

In the third phase, the working group invited the participants to validate the results by email. Change was allowed based on the feedback from the participants and discussion within the working group.

Results

A total of 16 experts from six countries participated the first-round Delphi survey. One of them did not continue to the second-round due to schedule conflict. Another two experts joined the second-round Delphi ().

Attitude Towards “The Applicability of Previous Guidelines of Guideline Development for Primary Care CPGs”

Fifteen of the 16 experts rated the applicability of previous guidelines of guideline development and the necessity of developing recommendations for primary care CPGs. The average scores (3.1 and 3.5, respectively) revealed fundamental applicability of previous guidelines, and necessity of developing recommendations to improve the feasibility and applicability of primary care CPGs and CPGs which aim to be implemented in primary care.

Planning Flow for Developing Primary Care CPGs

Before generating a project of developing primary care CPGs, the flow for considering whether to adopt previous guideline or to de novo develop guideline for primary care is summarized in Figure 2. For adaptation of guidelines, experts recommended to use previous approaches such as ADAPTE.11,12 The following recommendations focused on the de novo development of primary care CPGs and of CPGs which aim to be implemented in primary care (Table 1).

Figure 2

Recommendations for the planning flow of developing primary care clinical practice guidelines.

Table 1

Recommendations for the de novo Development of Clinical Practice Guidelines (CPGs) in Primary Care

Phase	Task	Recommendations
Planning guidelines	Planning and scoping guideline	1. To understand the current situation of primary care institutionsThe current situation of primary care institutions should be understood by the guideline group at the beginning of planning step by the following recommended steps.1.1 Information gatheringInformation on the organizational context of primary care institutions (eg. organizational structure, service procedure, health resource, payment for medical service of primary care institutions), disease distribution in primary care (eg. prevalence of disease, characteristics of patients) and the evidence-practice gaps (eg. the availability of guidance, perceived needs for evidence-informed guidelines) should be gathered firstly from existing sources (eg. literature and Health Statistics Yearbook).1.2 Multidisciplinary discussionA multidisciplinary discussion including primary care health practitioners as the main information providers should be performed. A survey covering different regions and types of primary care institutions is not necessary if the group is representative of all the diversity to be addressed by the guideline.2. Appraising existing guidelines2.1 A systematically search for relevant guidelines should be performed. Limitation on publication period, institution and language etc could be considered.2.2 Existing guidelines could be assessed by evaluating the quality, currency, content and applicability of guidelines. The tools and criteria for assessment could be seen in CAN-IMPLEMENT©.2.3 For the assessment of applicability, multidisciplinary discussion or health practitioner interview is recommended. Large-scale survey is usually not feasible due to constrained time and resources.2.4 The variations among different levels and types of primary care institutions should be considered in the assessment. But the guideline group should keep in mind that the idea is to produce a general guideline rather than a too specific one.3. TimelineFor an experienced guideline group, the de novo development of primary care CPG usually needs 1–2 years.
Setting up guideline groups	Setting up steering group, development group and external review group	1. The composition of guideline groups1.1 The opinion of all stakeholders in the development of CPG for primary care should be listened, including primary care practitioners, patients from primary care, academic organizations and developers of general CPGs etc. Of note, opinions can also be sought by other means than membership in guideline groups.1.2 Usually, the proportion of primary care membership should be more than 20% or higher. The exact proportion could depends the content of the guideline, the knowledge and skills of primary care practitioners. Primary care practitioners should be included in steering group, development group and external review group. The specialty of primary care practitioners could include clinicians, nurses, pharmacists, and administrative staff etc, depending on the content of guidelines.2. The role of primary care practitioners in de novo guideline developmentPrimary care practitioners should participate in all phases of guideline development including: scoping the guideline, formulating questions, assessment of the applicability of current guideline, assessment of the current situations of primary care institutions, developing recommendations, drafting the guideline, external review, and implementation and evaluation.
Declaration and management of interests		The same as general guideline.
Formulating questions and choosing outcomes	Formulating questions	1. To formulate and refine questions1.1 Drafting the questions for de novo development could be done in the step of planning the guideline, when it is decided to go with the de novo development of guideline for primary care. Questions should be focused on the areas that are valued by primary care providers, and where there are known mismatches between evidence and practice and variation in practice. In addition, the question formulation should be matched with the orientation, service capacity and workflow of primary care institutions.1.2 Questions should be refined after the assessment of existing guidelines and literature.2. The number of questionsThe number of questions depends on the aim and resources (eg, budget, timeline) of guideline.
	Choosing and rating outcomes	1. OutcomeThe outcomes should include both patient outcome (health outcome) and health system outcome. The patient outcome should be of importance to patients from primary care and the health system outcome should be in accordance with the orientation of primary care.
Evidence retrieval	Evidence retrieval and synthesis	1. Evidence retrievalFor the de novo guideline development, a comprehensive search is mandatory.
Evidence assessment	Evidence assessment	1. The applicability of evidenceThe applicability of evidence (results of studies) for primary care could be assessed by questionnaire, expert consulting (primary care providers included).2. Presentation of evidence sourceThe settings where the evidence was produced could be presented in guideline (eg, in primary care institutions, tertiary hospital).
Developing recommendations	Interpreting the evidence to make recommendations and prioritising recommendations	1. Factors considered in recommendation developmentThe accessibility and resource cost (including the health insurance coverage) of recommendations in primary care institutions should be considered in judgment of cost and benefits.The applicability for different regions and levels of primary care institutions should be considered in judgment of equity.
Producing and publishing guideline	Writing guideline	1. FormatBoth full and summary versions of guidelines should be considered. The summary version should mainly include recommendations. A link between full and summary versions of guidelines should be established.2. Difference between primary care guideline and general guidelineThe difference between primary care guideline and general guideline could be presented in primary care guideline.3. LanguageThe language of the primary care guideline should be plain and clear so as to be easily understood by primary health-care practitioners.
	External review	1. External reviewerThe primary care practitioners (target audience of the guideline), academic organizations (endorsement bodies), and developers of general guidelines (relevant guideline developers) should be included in external review process.2. External review contentThe language of guideline should be reviewed by primary care practitioners prior to release.
	Publishing guideline	1. DisseminationFree access of guideline should be advocated to facilitate the dissemination of guidelines, since most of primary health-care practitioners do not have access to a paid database.
Implementation and evaluation	Implementation and evaluation of guidelines	1. Guideline implementationGuideline implementation should be consistent with the access that is available for primary care providers.Making guidelines available at the time of decision making should be considered (eg, computerized decision support). If the guideline relates to patient decisions then patient decision aids could be developed.2. Education and trainingRelevant education and training of guideline is strongly suggested to make primary care practitioners aware of the guideline.
Updating guideline	Updating guideline	1. Updating cycleThe updating cycle should be decided based on topic areas, evidence updating speed and changing rate of medical resources in primary care institutions. Usually, 3–5 years is suggested.

Notes: General guideline refers to guideline developed for all levels of medical institutions. Primary care guideline refers to guideline developed for primary care institutions.

Planning Guideline

The situations and needs in primary care are usually ignored at the very beginning of guideline development. Understanding situations in primary care through literature or health statistics review, and multidisciplinary discussion is crucial at the planning step. Primary care health practitioners as the main information providers should be involved in the multidisciplinary discussion. The following information in primary care should be gathered: organizational context (eg, organizational structure, service procedure, health resource, payment for medical service of primary care institutions) and disease distribution (eg, prevalence of disease, characteristics of patients). Moreover, the evidence-practice gaps (eg, the availability of guidance, perceived needs for evidence-informed guidelines) should be known to highlight planning and scoping guidelines.

Setting Up Guideline Group

Primary care practitioners are not often involved in guideline development, even in guidelines for primary care. We recommend that primary care practitioners should be included in steering group, development group and external review group. For primary care CPGs, membership of primary care practitioners should be 20% or higher. Alternatively, their opinions should be sought by other means.

Formulating Questions and Choosing Outcomes

For guidelines whose target users include primary care practitioners, the applicability and feasibility of interventions for primary care should be considered as a question when formulating questions. For guidelines developed specially for primary care, the known mismatches between evidence and practice, or variation of practice in primary care should be considered as a question.

Since the orientation of primary care is usually different from that of the tertiary hospitals, the health system outcomes from the aspect of primary care should be considered in addition to patient outcomes (health outcomes).

Evidence Assessment

For evaluation of effectiveness and safety, magnitude of effect should be the same for primary care guidelines and general guidelines. Difference might lie in cost or cost-effectiveness, burdens and resource requirement. However, these evaluations from the perspective of primary care are usually ignored, causing inapplicability of guidelines in primary care. Given the evidence of burdens, resource requirement and cost in primary care, recommendations might change from “for” to “against” (or opposite) due to the change of net benefit.

We recommend evaluation of issues which might be different in primary care from the perspective of primary care, and to consider the difference in making recommendations.

Developing Recommendations

For guidelines whose target users include primary care practitioners, contextualized recommendation could be considered, hence making different recommendations for primary care and other levels of health care is reasonable. The difference in recommendations should be presented in primary care guidelines.

Producing and Publishing Guidelines

For writing guideline, the language of primary care guidelines should be plain and clear to be easily understood by primary care practitioners. We recommend guidelines which aim to be implemented in primary care to have a summary version including major recommendations, which are better accepted in primary care.

For external reviewer, primary care practitioners should be included in the external review process. And for guidelines developed specially for primary care, both primary care practitioners and developers of general guidelines on the same topic should be included in the external review process.

For dissemination, lack of access to guidelines is a major barrier for their implementation. Therefore, we recommend making guidelines free access, and disseminated not only by biomedical literature database, but also by other localized access which are available for primary care practitioners.

Implementation and Evaluation of Guideline

Being unaware of guidelines is one of the major barriers for implementation of guidelines in primary care. It is easier for practitioners in tertiary hospitals to be aware of new or updated guidelines through academic conferences, literature, clinical decision support system (eg, Uptodate), or colleagues. But practitioners in primary care have limited access to these sources. Therefore, we recommend making the primary care practitioners aware of guidelines first through local and individual access, for example, information system of primary care institutions, emails or short messages. Second, organizing small conferences or workshops in primary care institutions to facilitate the participation of primary care practitioners. Then, making guidelines available at the time of decision-making should be considered. This is usually done by computerized decision support in tertiary hospitals. For primary care, dissemination of a hard copy of the guidelines is an alternative approach to remind practitioners when making decisions.

Discussion

Summary of Findings

Developing clinical practice guidelines for primary care settings is an extensive and iterative process that involves several phases of input and consultations.13–15 This study aims to establish recommendations for the development of CPGs for primary care. Using Delphi survey methodology allowed us to seek opinions and perspectives from stakeholders based on existing methods for guideline development. In our study, recommendations on nine phases of guideline development for primary care were generated. These recommendations refer to specific issues rather than comprehensive procedures of the development of primary care guidelines.

Findings of Similar Studies

One of the barriers in the development of CPGs for primary care is the gap between published evidence and information needed for guideline development. Models developed by Habbema et al tried to bridge the gaps by projecting outcomes for the conditions for which the guideline is intended.24 They used colorectal and breast cancer screening as examples to show the utility of models. Moreover, resource limitation is another common situation in the development of CPG for primary care. Alper et al developed the RAPADAPTE method which extended “guideline adaptation” to “evidence source adaptation” and shortened the time of identifying, appraising, and synthesizing evidence in the de novo development of guideline.25 By using the RAPADAPTE method, they developed a guideline with 90 recommendations within six months.26 Further, implementation plans that address the concerns and complexities of everyday practice are essential for the promotion of primary care CPGs. The PARiHS framework developed by McKillop et al could help encompass the complex nature of evidence implementation by identifying positive and negative indicators of supports and inhibitors in everyday clinical practice.27–29

Limitation

The main limitation of our study lies in the variation of primary care settings between different countries. Participants involved in the Delphi survey may only consider the primary care special issues based on the situations in their own countries. Actually, this is likely the main reason that the coefficient of variation on each recommendation is not all small in the second-round Delphi survey. For remedy the limitation, we introduced primary care situation in China at the beginning of survey, and invited the participants to validate the results by email in the third phase to reduce the difference in results due to difference in understanding.

We focus on CPG development process, including CPG development and implementation. The reasons for lack of acceptance and guideline implementation are not our main aim. Patients were not included because they found it difficult to comment on CPG development process, but patients’ opinions need to be taken into account when developing recommendations for future CPGs in primary care.

Conclusions

In this study we present recommendations to inform the development of clinical practice guidelines in primary care. Next steps will include merging these recommendations with general guideline development methods to inform the development of guidelines for primary care.