Implementation of an evidence-based guideline on fluid resuscitation: lessons learnt for future guidelines.

INTRODUCTION: There is little experience with the nationwide implementation of an evidence-based pediatric guideline on first-choice fluid for resuscitation in hypovolemia.
METHODS: We investigated fluid prescribing behavior at (1) guideline development, (2) after guideline development, and (3) after active implementation and identified potential barriers and facilitators for guideline implementation. In order to minimize costs and to optimize implementation effect, we continuously developed and adjusted implementation strategies according to identified barriers. Implementation success was evaluated using questionnaires, pharmaceutical data, and data from medical records. DISCUSSION: The most remarkable change occurred after guideline development and dissemination: Normal saline use by neonatologists increased from 22-89% to 100% and by pediatric intensivists from 43-71% to 88-100%, and synthetic colloid use by pediatric intensivists declined from 29-43% to 0-13% with a reduction in albumin use by neonatologists from 11-44% to 0%. After active guideline implementation, most of specialist's management behavior was according to the guideline.
CONCLUSION: Stakeholders involved in the developmental process are of great importance in disseminating recommendations before active implementation. Therefore, to successfully implement guidelines and reduce costs of active implementation, any guideline development should consider implementation right from the beginning. Implementation strategies should target identified barriers and will therefore always be guideline specific.

Introduction

Clinical practice guidelines can facilitate translation of research into clinical practice and are seen as powerful tools to achieve effective care, reduce variability in daily practice, and may reduce costs [14]. However, many guidelines are not used in daily practice unless they are actively implemented pursuing consolidation of behavior change [12]. So far, there is little experience with the implementation of pediatric evidence-based guidelines, and the specific strategies favoring success are not well known.

In 2000, a Dutch multidisciplinary national committee developed an evidence-based guideline on the first-choice fluid for resuscitation of hypovolemic shock due to dehydration, sepsis, trauma, and hemorrhage in critically ill neonates and children up to 18 years. In 2004, it was updated. The guideline recommends normal saline as the first-choice treatment for all forms of hypovolemic shock [4]. In 2004, we started an implementation project with two objectives. Our first goal was to successfully implement the guideline’s recommendations. Our second aim was to learn some important lessons for the implementation of future pediatric guidelines. In this paper, we describe (1) pre- and postimplementation questionnaire surveys to investigate first-choice fluid use in various pediatric departments, (2) implementation barriers and facilitators encountered, (3) the tailored implementation strategies that were used, (4) quality-of-care indicators that we used to evaluate the implementation success, and (5) lessons learnt for the implementation of future pediatric guidelines.

Materials and methods

Questionnaires, barriers, and facilitators and implementation strategies

Preguideline questionnaire survey

Questionnaire A was developed (2000) to investigate pediatric practice before guideline development and to identify barriers and facilitators for guideline implementation. Questions concerned the first-choice plasma volume expanders in various forms of hypovolemic shock. It was sent to all heads of Dutch pediatric (PICU) and neonatal (NICU) intensive care units; all of them were also involved in the guideline development process (Fig. 1).

Fig. 1

Flow diagram of surveys by questionnaire to different specialists before guideline development, after guideline development, and after active implementation

Postguideline questionnaire survey

Two questionnaires were developed: questionnaire B for guideline development committee members and questionnaire C for all specialists not involved in the guideline development process but involved in treatment of hypovolemic children. Questionnaire B evaluated if the guideline had been implemented on the departments of the committee members. Implementation was defined as “the methods used to promote the uptake of the guideline on pediatric departments with the purpose of obtaining a structural place in medical care”. Identified barriers and facilitators were used to develop effective implementation strategies (Table 1). Questionnaire C evaluated fluid resuscitation practice in 151 randomly chosen specialists from all academic and all general hospitals (in general hospitals, the head of the department; in academic centers, the specialist on call): one pediatrician from all general hospitals, one pediatric anesthesiologist from all academic and burn centers and some general hospitals, one pediatric surgeon from all academic and burn centers, and one neonatologist and pediatric intensivist from all academic centers. Implementation barriers and facilitators were identified in order to develop effective implementation strategies for each different specialism. Questionnaire C contained ten questions exploring awareness and use of the existing national guideline, first-choice plasma volume expanders, and frequency of use.

Table 1

Reported implementation barriers and facilitators and implementation strategies

Barriers

Uncertain applicability of evidence from adults to children

Fear of inducing hypernatremia in neonates when using normal saline

Existence of a local protocol

Ongoing recommendation to use albumin or synthetic colloids by international “opinion leaders”

“Old habits and routines” that seem to work well

Unclear accessibility of the new guideline, lack of detailed knowledge about guideline’s recommendations, reported “unclear recommendations”, and “extensive guideline”

Facilitators

“Evidence-based” guideline, relevant to practice, with clear recommendations

Motivated members of multidisciplinary committee

Digital free accessibility of the guideline

Strategies

At guideline development onset

Personal interviews with academic specialists aimed at identification of potential barriers and facilitators for implementation

Formation of multidisciplinary guideline development committee consisting of all stakeholders’ representatives

During guideline development

Use of evidence-based guideline development process

Potential barriers to implementation addressed in guideline

Involvement of all stakeholders in formulating the final guideline recommendations

Local implementation by stakeholders on their departments and in regional hospitals

After guideline development

Endorsement of guideline by the Dutch Association of Pediatrics

Educational visits of academic specialists

Information stand at Dutch annual pediatric meeting; checking participants’ recommendation knowledge offering a reward (plastic duck filled with bath salts)

Proposed incentive: printed copy of full-text guideline for those who fill out questionnaires

Dissemination of the guideline’s recommendation as a pocket-size plasticized card

Digital free access to the full text guideline

Publication of the guideline’s recommendation in the newsletter of the Dutch Association of Pediatrics and Dutch Journal of Pediatrics

Interactive lessons about guideline recommendations for all Dutch pediatric residents

Implementation strategies

As the frequency of fluid resuscitation is the highest in academic centers, extra attention was given as an implementation strategy. One staff member (the specialist on call, not the committee member) of each of the academic pediatric specialist units was visited by the guideline development and implementation group (MT). During this visit, questionnaire C was filled out and barriers and facilitators were identified (Table 1). To specialists working in general hospitals (n = 121), the questionnaires were sent by postal mail to the head of the department with a stamped addressed return envelope included. An accompanying letter explained the rationale for the survey and offered an incentive, proposed as “a valuable document for your hospital’s library”. After 6 weeks, nonresponders were called.

In 2005, all involved clinicians received a printed version of the guideline. In addition, subdivision chairs of the Dutch Association of Pediatric Surgery, Pediatric Anesthesiology, Neonatology, Pediatric Intensive Care, and General Hospital Pediatric Practices received it and were asked to inform their members about the recommendations. Every visitor of the yearly congress of Dutch pediatricians received a pocket-size plasticized card with the guideline recommendations (n = 1,000) and were asked to answer three questions about the guideline at our “implementation stand”. In return, they received an incentive. During 2005, every 2 months, a newsletter about this implementation project was published in the journal of the Dutch Pediatric Society. Guideline’s recommendations were also published in the Dutch Journal of Pediatrics [26]. From 2001 on, all consecutive Dutch first-year pediatric residents (n = 720) were taught interactively about the guideline by the chair of the guideline development committee (NB).

Evaluation of implementation success

The following quality-of-care indicators were developed: percentage of specialists aware of the guideline, percentage of specialists reportedly following the guideline, and change (% liters) of annual use of volume expanders (i.e., albumin, synthetic colloids, and crystalloids). Quality-of-care indicators were measured using questionnaires, hospital pharmaceutical data, and data from medical records. Successful implementation was defined as a 10% change in quality-of-care indicators after active implementation. Details on how these changes were measured are given below.

Reported behavior

After analyzing the results of all questionnaires (2006), we defined two target groups to evaluate the success of active implementation: (1) general hospital pediatric departments that had reported not to always follow the recommendations (27 out of all 95, 28%) and (2) all academic departments of pediatric anesthesiology because of their lowest use of crystalloids and because their frequency of fluid use is relatively high. One member of the guideline development and implementation group (MT) called these practices (n = 35) and asked to speak to the pediatrician/anesthesiologist on call, being not the one already visited in 2004. Questions concerned (1) awareness of the guideline, (2) whether the guideline’s recommendations were always followed, and (3) first-choice volume expanders. The actual questionnaires can be provided upon request from the authors.

Pharmaceutical data

We tried to collect data on the annual volume in liters of different plasma solutions used per department for the indication of hypovolemia in children before guideline development onset in 2000, in 2004, and after active implementation in 2006. Given that the highest amount of intravenous fluids was prescribed in academic hospital centers, data from all academic pharmacies (n = 8) were collected (Fig. 2).

Fig. 2

a Proportion first-choice fluid in dehydration before (2000), after guideline development (2004), and after active implementation (2006) according to specialty. b Proportion first-choice fluid in sepsis before (2000), after guideline development (2004), and after active implementation (2006) according to specialty. c Proportion first-choice fluid in trauma before (2000), after guideline development (2004), and after active implementation (2006) according to specialty. d Proportion first-choice fluid in hemorrhage before guideline development (2000), after guideline development (2004), and after active implementation (2006) according to specialty. Data were not obtained for all years for each different specialty (see “Materials and methods”). Academic specialists: NICU—n = 10 in 2000, n = 8 in 2004, neonatologists do not encounter trauma as a cause for hypovolemia in neonates; PICU—n = 7 in 2000, n = 8 in 2004; pediatric anesthesiology departments—n = 8 in 2004 and 2006; pediatric surgery departments—n = 6 in 2004. General hospital pediatricians—n = 95 in 2004

Medical records

Initial plasma volume expander used was extracted from the medical records of 120 consecutive patients who were treated for hypovolemia in January and February 2006 on three academic neonatal and three pediatric intensive care units (n = 6): (1) one with the highest amount of albumin use, (2) one with the highest amount of nonhuman colloids use, and (3) one with moderate use of all plasma volume expanders compared with other academic centers. With a number of 20 medical records per academic hospital unit concerning 20 hypovolemic children in all of whom saline 0.9% is prescribed as first-line treatment, one can conclude with a 95% confidence interval of 84–100% that for all hypovolemic children on that department, normal saline is used as first-line treatment. Figure 3 shows the chain of events on a time scale.

Fig. 3

Chain of events on a time scale

Data analysis

Statistical analysis included descriptive statistics and frequency analysis. All analyses were performed using SPSS 12.0 (SPSS, Chicago, IL, USA). Microsoft Excel was used for analyzing pharmaceutical data.

Results

Questionnaire surveys

Response rate

Response rates to questionnaires A and B and C were 95% (17 of 18), 77% (20 of 26), and 99% (150 of 151), respectively. In case of missing answers, respondents were approached. Finally, all questionnaires were fully completed.

Barriers to and facilitators for guideline implementation and implementation strategies used

Table 1 shows barriers and facilitators to implementation reported during the whole process of guideline development and implementation and all implementation strategies used.

First-choice fluid therapy before and after guideline development

Figure 2a–d shows reported use of first-choice volume therapy for all forms of hypovolemic shock in different clinical settings before (in 2000) and after guideline development (in 2004) and after active implementation (in 2006). Active implementation started in 2004. Only a minority used other crystalloids instead of normal saline. Although normal saline was recommended in the guideline, for ease of reading and interpretation, we report “total crystalloid use”.

Before guideline development, first-choice fluid therapy on the NICUs for different forms of shock was crystalloid in 22–89%, albumin in 11–44%, and synthetic colloid in 0–11%. After guideline development, reported use was 100%, 0%, and 0%, respectively. Before guideline development, it was on the PICU’s crystalloid in 43–71%, albumin in 0–14%, and synthetic colloid in 29–43%. After guideline development, it was 88–100%, 0%, and 0–13%, respectively. After guideline development, but before active implementation, reported first-choice fluid therapy in general hospital pediatric departments was a crystalloid in 84–100%, albumin in 0–1%, and synthetic colloid in 0–2%; in academic pediatric surgical departments, it was 33–100%, 0–17%, and 0–17%, respectively, and in academic pediatric anesthesiology departments, it was 50–88%, 0%, and 13–38%, respectively. This last group was evaluated after active implementation: 75–100% now reported crystalloids as first choice, albumin 0%, and synthetic colloids 0–25%. Of all participants, 68% reported never to use colloids for volume therapy in hypovolemic shock and 32 % sporadically, i.e., 90% of these 32% less than once a month.

Local implementation and barriers and facilitators met by members of the multidisciplinary guideline development committee

Up to 95% of respondents of the guideline development committee reportedly already implemented the guideline after 2001. Only 15% (three of 20) reported barriers during their implementation consisting of “a too extensive guideline text” and “routines or old habits”.

Frequency of prescribing fluids for volume therapy

On average, academic pediatric specialists prescribe fluid therapy daily or several times a week. General hospital pediatricians prescribe fluid therapy monthly to once in every 3 months. General pediatric anesthesiologists prescribe initial fluid therapy once in 3 months.

Awareness of guideline by pediatric specialists not involved in the guideline development

Before active implementation, 52% (78 of 150) of all practitioners, mainly general hospital pediatricians 66% (63 of 95), were aware of the guideline.

Adherence to the guideline

Of academic specialists, 64% (nine of 14) always followed the recommendations, 29% (four of 14) inconsistently, and 7% (one of 14) never (“own protocol”). Main reason given for inconsistent use was “unclear recommendations” in 50% (two of four) and “insufficient knowledge of these recommendations” in 50% (two of four). Many specialists complained that the guideline was too long and too extensive. Reporting to questionnaire C, 57% (36 of 63) of general pediatricians who were aware of the guideline claimed to “always follow the recommendations” and 43% (27 of 63) “inconsistently”. Lack of detailed knowledge about the recommendations was mentioned by 57% as main reason for inconsistent use.

Evaluation using indicators

Before active implementation, only 13% (one of eight) of academic pediatric anesthesiologist were familiar with the guideline compared to 50% (four of eight) after active implementation. Before active implementation, 43% (27 of 63) of general hospital pediatricians aware of the guideline did not always follow the guideline. After active implementation, 89% (24 of 27) claimed to “always” follow the recommendations. Eleven percent (three of 27) was not aware of the guideline, but two of these three hospitals already used normal saline as first-choice treatment.

Response rate of academic pharmacists was 100%. All pharmacists could only provide data on total fluid use (in liters) per year for 2004 and 2006. No electronic data were available for 2000. They were unable to specify data: (1) per indication, (2) on the number of children receiving fluid therapy, and (3) for each department separately. Therefore, we dismissed all results.

A total of 120 medical records were included. This showed that 109 of 120 (92%) of hypovolemic children received normal saline and that five out of six departments used normal saline in all 20 consecutive cases for initial treatment. This was after active implementation and in keeping with the answers given in the questionnaire.

Discussion

We observed a remarkable change in prescribing practice by Dutch physicians. Usually, changes in care after guideline development and implementation are not that overwhelming, with a median 10% improvement of care [12, 15]. Some important lessons can be learnt about pediatric guideline development and implementation in general. We will discuss the state of evidence for the various strategies, strengths, and weaknesses of implementation study designs in general and give recommendations for future guideline implementation.

Evidence and marketing in fluid prescribing behavior

Several surveys have shown that, despite publication of systematic reviews [1, 2, 7], this evidence has not changed clinical practice and most physicians still use colloid products [24, 25]. One survey among pediatric anesthesiologists showed that albumin continues to be the most frequently used volume expander in neonates, whereas alternative colloids are used in older children [25]. This is quite surprising since pediatric clinical research with hetastarch and gelatins is almost nonexistent. Several authors have shown that physicians may change their prescribing practice as a result of contact with drug retailers, irrespective of scientific evidence [8, 19, 22]. Marketing strategies by the industry involves regular visits of stakeholders and identifying barriers. We copied this approach in our implementation study.

Guideline development and implementation

A guideline should be developed according to evidence-based principles which facilitate the acceptance and effective use in the target group [6]. The following guideline characteristics have been shown to contribute to their use in practice: inclusion of specific recommendations, sufficient supporting evidence, a clear structure, an attractive layout, and short summaries [16]. Guideline endorsement by a physician’s own specialty organization is associated with improved physician’s confidence in a guideline [17, 23]. Surveys of pediatrician’s attitudes about guidelines report barriers to adherence such as lack of awareness, lack of agreement with specific recommendations, or lack of agreement with the concept of guidelines in general [9]. Identification of these barriers can foster opportunities to improve physician adherence to a guideline. Physician participation in guideline development has been shown to be useful in addressing barriers owing to lack of agreement [18]. Recent graduates are more likely to find guidelines helpful and reported change of behavior more often than in seasoned practitioners [9]. Our guideline was developed using evidence-based guideline development principles [4, 20]. All stakeholders were involved in the entire process of guideline development and implementation.

Fluid prescribing behavior after guideline development

After guideline development and before active implementation, all neonatologists used normal saline and 88% of pediatric intensivists. Albumin use had declined to zero. Our survey before active guideline implementation showed that most of the pediatric specialist’s management behavior was already in keeping with the guideline’s recommendations. We presume that stakeholders (i.e., key clinician leaders in the field of hypovolemia) who were involved in the developmental process were disseminating the recommendations in their region as 95% of the guideline development committee members reportedly had implemented the guideline in their own departments (see “Local implementation and barriers and facilitators met by members of the multidisciplinary guideline development committee” in “Results” section)

Implementation strategies and study design

Several systematic reviews have showed that there is no “magic bullet” for implementation success [3, 12, 21]. Interventions that consistently have shown effectiveness include interactive educational meetings, educational outreach visits, reminders, and multifaceted interventions. Interventions with variable effectiveness include audit and feedback, use of local opinion leaders, local consensus processes, and patient-mediated interventions. Interventions with little or no effect are didactic educational meetings and educational materials [3, 11, 12].

There is a strong call for multiple arm cluster randomized controlled trials to investigate the (cost-) effectiveness of various different implementation strategies [10, 13]. An important advantage of this design is the high internal validity, meaning that a causal relation between implementation strategies and observed change in practice can be demonstrated. Disadvantages are the fact that they are expensive, time-consuming, and, generally, will have low external validity and applicability outside the context of the guideline at hand. First of all, the success of implementation has been shown to be very much dependent on the guideline development process. Second, the effect of different implementation strategies like outreach visits will all highly depend on the commitment and skills of the persons executing these strategies. Their effectiveness and importance cannot be overstated, but, unfortunately, cannot be quantitated as typical aspects of an intervention. All these issues will be guideline specific and we therefore believe that the results of randomized implementation trials will have a limited applicability to future guidelines in other specialties and other countries. An alternative design is “interrupted time series”. This design is useful to detect whether the intervention (i.e., implementation strategy) has had an effect that is greater than what would be expected by the underlying secular trend. Data are collected at multiple time points before and after the intervention. A disadvantage of this design is the fact that it is necessary to collect sufficient data points, which is time- and resource-consuming, and the fact that analysis of data and its interpretation is usually complicated [10].

We decided it would be most efficient to continuously develop and adjust implementation strategies according to identified barriers in this particular case.

Evaluation of implementation success by indicators

No patient-oriented outcomes could be used because evidence showed no difference between treatments. In our case, the percentage change in amount of volume expanders used was the main indicator. According to the questionnaires, active implementation was successful as well. As self-reported measures potentially reveal an idealized version of actual behavior, we developed two more objective indicators: pharmaceutical data and medical records.

Pharmaceutical data appeared to be invalid (see “Results”). Data from medical records showed that 92% of hypovolemic children actually received normal saline. This was after active implementation of the guideline and in keeping with the answers given in the questionnaire. We therefore believe we can rely on the answers obtained from the questionnaires.

Limitations

Our study has several limitations. A disadvantage of our uncontrolled design is the fact that a causal relationship between implementation strategies and observed prescribing changes cannot be considered proven, and therefore, the results should be interpreted with caution. Changes in prescribing behavior could also be a result of the publication of Cochrane reviews and the Saline versus Albumin Fluid Evaluation (SAFE) trial. In our survey, all participants were asked to motivate their decision on first-choice plasma volume expanders in different types of hypovolemic shock. Yet, none of them mentioned the SAFE trial. Therefore, guideline development and implementation were important factors in the change of fluid prescribing behavior. Another possible limitation is the fact that we evaluated postactive implementation success in a limited group of specialists. Because most of physician’s prescribing behavior was already in keeping with the guideline, it was not cost-effective to extensively evaluate fluid prescribing behavior in all specialists after active implementation. Yet, we obtained data from medical records that showed that indeed 92% of children in university hospitals received normal saline.

A clear limitation may be the use of a self-report measure (questionnaires) in this study. We tried to minimize this bias assuring physicians anonymity. Furthermore, data collection differed between academic centers and general hospitals (personal interview vs. mail survey). Possible bias toward perceived acceptable responses in personal interviews is therefore possible. However, the results obtained from the medical records were in keeping with the results from the questionnaires.

Finally, we demonstrate early successful implementation, which does not necessarily ensure longer-term effectiveness [5]. Therefore, monitoring by indicators at regular intervals remains necessary.

Conclusions

Stakeholders are of great importance in the process of guideline formulation and implementation. Furthermore, we believe that specifically targeting implementation strategies to identified barriers has played an important role in our observed change in practice. As these barriers will always be guideline specific, every implementation project should be tailor made.