Mind the gap: Mapping variation between national and local clinical practice guidelines for acute paediatric asthma from the United Kingdom and the Netherlands.

BACKGROUND: Clinical practice guidelines (CPGs) aim to standardize clinical care. Increasingly, hospitals rely on locally produced guidelines alongside national guidance. This study examines variation between national and local CPGs, using the example of acute paediatric asthma guidance from the United Kingdom and the Netherlands.
METHODS: Fifteen British and Dutch local CPGs were collected with the matching national guidance for the management of acute asthma in children under 18 years old. The drug sequences, routes and methods of administration recommended for patients with severe asthma and the tone of recommendation across both types of CPGs were schematically represented. Deviations from national guidance were measured. Variation in recommended doses of intravenous salbutamol was examined. CPG quality was assessed using the Appraisal of Guidelines for Research and Evaluation (AGREE) II.
RESULTS: British and Dutch national CPGs differed in the recommended drug choices, sequences, routes and methods of administration for severe asthma. Dutch national guidance was more rigidly defined. Local British CPGs diverged from national guidance for 23% of their recommended interventions compared to 8% for Dutch local CPGs. Five British local guidelines and two Dutch local guidelines differed from national guidance for multiple treatment steps. Variation in second-line recommendations was greater than for first-line recommendations across local CPGs from both countries. Recommended starting doses for salbutamol infusions varied by more than tenfold. The quality of the sampled local CPGs was low across all AGREE II domains.
CONCLUSIONS: Local CPGs for the management of severe acute paediatric asthma featured substantial variation and frequently diverged from national guidance. Although limited to one condition, this study suggests that unmeasured variation across local CPGs may contribute to variation of care more broadly, with possible effects on healthcare quality.

Introduction

The increasing volume of new clinical evidence hinders the ability of individual clinicians to incorporate large amounts of constantly updated information into their decision-making [1]. Clinical practice guidelines (CPGs) are documents that systematically distil medical evidence into treatment recommendations to inform patient care [2, 3]. CPGs support the provision of evidence-based medicine and aim to optimize the quality and cost-effectiveness of care [1, 4–6]. CPGs are produced by numerous leading clinical institutions, including professional bodies and national societies like the National Institute for Health and Care Excellence (NICE) in the United Kingdom (UK) and the Dutch Federation of Medical Specialists [7-9]. These organisations systematically review the scientific literature, condensing the best available evidence into practice-oriented recommendations for various conditions [4, 10–13]. Expert opinion and consensus judgment may be used in the absence of conclusive evidence [10].

Despite their stated goal of standardising the quality of clinical care [3], CPGs are not a matter ‘of one size fits all’ [14]. It may not always be possible for clinicians at a given hospital to adopt national recommendations without adaptation to their local resources and needs, for example, differences in the availability of medications, various levels of local expertise or financial constraints [12, 14]. In contrast to the well-defined methodologies used to design national CPGs, the production of local CPGs remains a largely ad-hoc process, driven by small groups of clinicians with various levels of familiarity with guideline development [10, 15]. In addition to drawing on national CPGs, local CPGs may incorporate alternative sources of information, including locally available expert opinion and scientific literature published more recently than that incorporated in national CPGs [1, 16].

Although national CPGs and their locally adapted counterparts have become a staple of day-to-day care, variation in clinical practice continues to undermine healthcare outcomes [17, 18]. An example of this is provided by the management of acute paediatric asthma (APA). Asthma is the most common chronic paediatric condition in the UK and the Netherlands [19, 20]. There are approximately 25,000 annual emergency hospital admissions for APA in the UK [20]. Despite APA being extensively covered in national and local clinical guidance [21-23], variation in the emergency clinical management of APA is widespread [20, 24–26]. Significant variation in regional mortality from asthma, including APA, has been reported in recent years [27]. Variation in clinical practice has been related to differences in CPGs, including those produced by individual hospitals [28-30]. The nature and extent of local CPG variation, however, remains poorly understood. This study examines variation in the treatment recommendations for severe APA outlined in national guidance from the UK and the Netherlands and by a sample of local CPGs from British and Dutch hospitals.

Methods

Guideline and data sampling

National guidelines on APA produced by the British Thoracic Society/Scottish Intercollegiate Guideline Network (BTS/SIGN) in 2019 and the Dutch Paediatric Society (NVK) in 2012 were collected [21, 23]. Both documents were chosen as the clinical guidelines from the relevant learned bodies of the UK and the Netherlands for the management of acute paediatric asthma. A convenience sample of local CPGs for APA management were obtained between 1 January and 1 February 2019 from Dutch and British tertiary-level hospitals. Local CPGs were retrieved online from hospital websites where available, or were obtained from clinicians working at each hospital. The most recent version of each local CPG was used. CPG recommendations were subdivided according to the age of the patient, to the clinical severity of APA at presentation and according to a patient’s response to treatment. Treatments recommended in children two to eighteen years old were examined. The criteria outlined by each CPG to assess asthma severity at presentaton were collected. Pharmacological interventions recommended for managing patients with the most severe form of asthma at presentation and by a lack of response to consecutive treatments were examined. Hereafter, this is referred to as the ‘most severe’ treatment pathway. For each CPG, the dose, route (e.g. intravenous) and method (e.g. a bolus) of administration of each recommended drug was extracted, alongside the order in which it was suggested to be administered. The tone of each recommendation was noted and categorized as either mandatory (‘give’) or optional (‘consider giving’).

Variation in drug choice, sequence and tone

Differences in drug choice, sequence and route of administration across the sampled local CPG treatment pathways were measured using the relevant national CPG as a reference. Local CPGs were awarded one point at each step if they recommended a drug type or an administration route that differed from their respective national CPG at that position or if additional alternatives were offered. Differences in methods of administration or in the tone of recommendation were not scored. Differences in drug choice, sequence, route and method of administration and tone of recommendation across the sampled treatment pathways were also represented in a common schematic diagram. Design idioms were borrowed from familiar urban transport maps. Each guideline was represented as a single line featuring the sequence of drugs recommended for a patient with most severe APA. Where a guideline offered clinicians the choice between two drugs with no distinct preference, both choices were represented. Drugs recommended for treating comorbidities, differential diagnoses or side effects resulting from the primary APA treatment were not represented.

Variation in drug dosage

Beyond differences in recommended drug choice, sequence and tone of recommendation, variation may also exist in medication dosing. For APA, this has, for instance, been shown to be the case for continuous salbutamol infusions [31]. As a case study, the upper and lower limits of recommended salbutamol infusion dose ranges were compared across the sample.

Guideline quality assessment

The quality of the sampled national and local guidelines from the UK and the Netherlands was scored independently by two reviewers (CK and RN) using the Appraisal of Guidelines Research & Evaluation (AGREE) II instrument [32]. AGREE II is a standardized tool that allows to assess the methodological quality of CPGs based on six domains: 1) scope and purpose, 2) stakeholder involvement, 3) rigour of development, 4) clarity of presentation, 5) applicability and 6) editorial independence of the guideline. Each domain is comprised of several items that can each be scored on a 7-point scale taking into account predefined criteria. Discrepancies of over 3 points per item across reviewers were discussed and scores were revised as felt to be reasonable. Scaled domain scores were defined in line with AGREE II, by summing up item scores within each domain for each reviewer and standardizing it as a percentage of the maximum possible score [32].

Results

Guideline description

Of the sampled local CPGs, seven came from British tertiary-level hospitals and seven from Dutch tertiary-level hospitals, including two CPGs from different units of the same Dutch hospital (NL1a and NL1b). One additional Dutch hospital had no local CPG and used national (NVK) guidance (NL5). The basic characteristics of the sampled CPGs and the criteria used by each CPG to characterise APA severity at presentation are shown in Table 1. All CPGs stratified patients into two (‘moderate’ and ‘severe’) or three (‘moderate’, ‘severe’, and ‘life-threatening’) categories. In line with NVK guidance, most local Dutch CPGs distinguished two groups at presentation using a 94% oxygen saturation (SpO2) threshold. Local British CPGs mostly aligned with British national (BTS/SIGN) guidance, and defined three groups based on a 92% SpO2 threshold alongside other diagnostic parameters including the presence of signs of life-threatening APA.

Table 1

Description of sampled acute paediatric asthma CPGs from the UK and the Netherlands.

Guideline	Last updated	Scale of application	Setting	Online access	Authors	Description of methods	Number of stratified patient groups at presentation	Assessment parameters for most severe APA at presentation
BTS/SIGN	2019	National	Hospital including PICU	Yes	Paediatricians, respiratory consultants, pharmacists, researchers, nurse specialists & practictioners	Yes	Three	spO2 < 92% plus any of the following: PEF < 33% of best or predicted,1 silent chest, cyanosis, poor respiratory effort, agitation, confusion
NVK	2012	National	Hospital, not specified	Yes	Paediatricians, paediatric respiratory consultants	Yes	Two1	spO2 ≤ 94%
UK1	2014	Local	Hospital excluding PICU	No	Asthma Steering Group, not further specified	No	Three	spO2 < 92% plus any of the following: PEF < 33% of best or predicted value1 silent chest and/or cyanosis, poor respiratory effort, altered consciouness
UK2	2016	Local	Not specified	No	Paediatrician	No	Three	spO2 < 92% plus any of the following: PEF < 33% of best or predicted value,1 silent chest, cyanosis, poor respiratory effort, agitation or exhaustion, altered consciousness
UK3	2018	Local	Ward excluding PICU	Yes	Paediatric respiratory consultant, paediatrician, clinical pharmacist	Yes	Three	Any of the following: spO2 < 92%, silent chest, cyanosis, poor respiratory effort, fatigue or exhaustion, agitation or reduced consciousness
UK4	2015	Local	Not specified	No	Paediatric respiratory consultant, paediatrician, paediatric intensive care consultant, paediatric medical pharmacist	Yes	Three	spO2 < 92% plus any of the following: silent chest, cyanosis, poor respiratory effort, agitation or altered consciousness, exhaustion, increased pCO2 or hypotension3
UK5	2017	Local	Hospital including PICU	No	Paediatric respiratory consultant, paediatrician, paediatric medical pharmacist	Yes	Three	Any of the following: spO2 < 92%, PEF < 33% of best or predicted value,1 silent chest, poor respiratory effort, agitation or altered consciousness, heart rate ≥ 140/min (2–5 years) or > 125/min (≥ 5 years)
UK6	2017	Local	Hospital, excluding PICU	Yes	Paediatric respiratory consultant, paediatricians, senior staff nurse	No	Two1	Any of the following: spO2 < 92%, PEF < 33% of predicted value,1 silent chest, cyanosis, exhaustion with poor respiratory effort, confusion, maximal accessory muscle use/recession, unable to talk, coma, hypotension, marked tachycardia
UK7	2018	Local	Not specified	No	Paediatric respiratory consultant, paediatrician, paediatric medical pharmacist	No	Three	spO2 < 92% plus any of the following: PEF < 33% of best or predicted,1 silent chest, cyanosis, poor respiratory effort, fatigue or exhaustion, altered consciousness
NL1a	2019	Local	PICU	No	Unspecified	No	Two1	Not specified
NL1b	2016	Local	Hospital, incl. A&E	No	Paediatric respiratory consultants, paediatrician,	No	Two1	spO2 ≤ 94%
NL2	2017	Local	Hospital incl. PICU	No	Paediatrician, Paediatric intensive care consultant, paediatric respiratory consultant	No	Two1	spO2 ≤ 94%
NL3	Unknown	Local	Hospital incl. PICU	No	Paediatric respiratory consultant	No	Two1	spO2 ≤ 94%
NL4	2017	Local	PICU	No	Unspecified	No	Two1	spO2 ≤ 94%
NL5	Uses national CPG
NL6	Unknown	Local	Hospital	No	Unspecified	No	Two 1	spO2 ≤ 94%
NL7	2011	Local	Hospital	No	Paediatrician, paediatric intensive care consultant	No	Two	spO2 ≤ 94%

A&E: Accidents & Emergency; APA: acute paediatric asthma, BTS/SIGN: British Society/Scottish Intercollegiate Guidelines Network, NL: Netherlands, NVK: Nationale Vereniging voor Kindergeneeskunde (Dutch Society for Paediatrics), PEF: peak expiratory flow, PICU: Paediatric Intensive Care Unit, SpO2: oxygen saturation, UK: United Kingdom.

1 Applicable for children above 5 years.

The recommended drug treatment pathways for each CPG are represented schematically in Fig 1. Differences in drug choice, sequence and route of administration across the sampled CPGs were quantified in Table 2.

Fig 1

Variation across treatment pathways for most severe acute paediatric asthma across British and Dutch CPGs.

Drug recommendations proceed from left to right. Iv: intravenous, neb: nebulized.

Table 2

Quantitative assessment of deviations from national guidance across sampled local British and Dutch CPGs.

Step	Drug 1	Drug 2	Drug 3	Drug 4	Drug 5	Drug 6	Drug 7	Drug 8	Number of deviating steps (%)
United Kingdom
BTS/SIGN	O2	Nebulized salbutamol	Nebulized ipratropium	Oral CS	Nebulized MgSO4	Intravenous MgSO4	Intravenous salbutamol	Intravenous aminophylline	First-line drugs	Second-line drugs	Across the CPG
UK1	0	0	0	1	1	0*	0*	0*	2(25%)	0*	2(25%)
UK2	0	0	0	1	1	0*	0*	0*	2(25%)	0*	2(25%)
UK3	0	0	0	0	1	0*	0*	0*	1(13%)	0*	1(13%)
UK4	0	1	0	1	1	0*	0*	0*	0	0*	3(38%)
UK5	0	0	0	1	1	0*	0*	0*	2(25%)	0*	2(25%)
UK6	0	0	1	1	1	0*	0*	0*	3(38%)	0*	3(38%)
UK7	0	0	0	0	0	0*	0*	0*	0	0*	0
No. of deviating steps across local British CPGs (%)	0	1(14%)	1(14%)	5(71%)	6(86%)	0	0	0	13(37%)	0*	13(23%)
Netherlands
NVK	O2	Nebulized salbutamol	Nebulized ipratropium	Oral CS	None	Intravenous MgSO4	Intravenous salbutamol	No drug recommended	First-line drugs	Second-line drugs	Across the CPG
NL1a	0	0	1	0	-	0	0	1	1(25%)	1(33%)	2(29%)
NL1b	0	0	0	0	-	0	0	0	0	0	0
NL2	0	0	0	1	-	0	0	1	1(25%)	1(33%)	2(29%)
NL3	0	0	0	0	-	0	0	0	0	0	0
NL4	0	0	0	0	-	0	0	0	0	0	0
NL5	0	0	0	0	-	0	0	0	0	0	0
NL6	0	0	0	0	-	0	0	0	0	0	0
NL7	0	0	0	0	-	0	0	0	0	0	0
No. of deviating steps across local Dutch CPGs (%)	0	0	1(14%)	1(14%)		0	0	2(29%)	2(6%)	2(11%)	4(8%)

*: Local British guidelines received zero points at that position given the absence of a rigid sequence outlined by BTS/SIGN. BTS/SIGN: British Society/Scottish Intercollegiate Guidelines Network; CPG: clinical practice guideline; CS: corticosteroids; MgSO4: magnesium sulfate. NL: Netherlands; O2: oxygen UK: United Kingdom.

The differences across nationally recommended pathways were examined. The NVK recommended six drugs in total, all of which were also recommended by BTS/SIGN. Alongside the drugs recommended by the NVK, BTS/SIGN also recommended nebulized magnesium sulfate and intravenous aminophylline. In both CPGs, the sequence and tone of recommendations differed more for second-line drugs than for first-line treatments. BTS/SIGN and the NVK recommended the same mandatory sequence of first-line treatments (oxygen, nebulized salbutamol, nebulized ipratropium bromide, oral corticosteroids) up to nebulized magnesium sulfate. The latter drug was advised as an optional treatment by British national guidance. National recommendations on second-line drugs diverged in both sequence and tone. While the NVK recommended considering intravenous magnesium sulfate before giving intravenous salbutamol, BTS/SIGN recommended that clinicians consider administering any of three recommended second-line drugs (intravenous magnesium sulfate, intravenous salbutamol and intravenous aminophylline) in the order they deemed most appropriate based on their assessment of the risks and benefits involved.

Local CPG recommendations for most severe APA were compared to the relevant national CPGs. All Dutch and British local CPGs aligned with their respective national CPG on four drugs. Three drugs were first-line treatments (oxygen, nebulized salbutamol, corticosteroids) and one was a second-line intervention (intravenous salbutamol). One British CPG (UK4) recommended nebulized terbutaline as an alternative for nebulized salbutamol without justifying this deviation from national guidance (see S1 File on the local considerations behind various sequences of second-line treatments). Six out of seven local British CPGs departed from BTS/SIGN recommendations by omitting nebulized magnesium sulfate. None of them provided a rationale for this omission. Six Dutch local CPGs recommended the same drugs as the NVK. Differing with national recommendations, one Dutch local CPG omitted ipratropium bromide (NL1a), while two Dutch local CPGs included intravenous theophylline as their last recommended intervention (NL1a and NL2). These departures from national guidance were justified within the relevant local CPG.

Looking at the advised routes and methods of administration for recommended drugs, two Dutch and six British local CPGs deviated from national CPGs by recommending intravenous corticosteroids instead of, or following the administration of oral corticosteroids. A further two Dutch and six British local CPGs recommended a bolus of intravenous salbutamol before starting a continuous salbutamol infusion.

Apart from recommending nebulized terbutaline (UK4), and omitting nebulized magnesium sulfate (all CPGs but UK7), local British CPGs advised the same sequence of first-line drugs as BTS/SIGN. While they did not strictly differ from national guidance given the lack of a defined second-line drug sequence, local British CPGs, by contrast, recommended four distinct sequences of second-line treatments encompassing the same three drugs. The most common sequence was intravenous magnesium sulfate followed by intravenous salbutamol followed by intravenous aminophylline (UK2, UK4, UK5). By contrast, five out of seven local Dutch CPGs recommended the same treatment sequence as the NVK for both first-line and second-line drugs.

Overall, local British CPGs deviated from the drug choice and order recommended in national guidance (where an order was declared) for 23% of their recommended interventions compared to 8% across local Dutch CPGs (see Table 2). Deviations from national guidance were more frequent for second-line treatments than for first-line drugs in Dutch local CPGs (11% compared to 6% deviating steps). This was not the case in British local CPGs given the absence of a rigidly defined second-line drug sequence in BTS/SIGN guidance. Five local British CPGs and two Dutch local CPGs differed from their respective national CPG for multiple treatment steps. Local CPGs from both countries featured more variation in second-line interventions than first-line treatments.

Both Dutch and British local CPGs adopted a more flexible tone of recommendation for second-line interventions than for first-line drugs. This was especially true for British local CPGs. Local CPGs from both countries outlined more rigid drug sequences than national CPGs, with more drug recommendations communicated using an imperative tone (‘give’) and fewer alternatives provided.

Variation in intravenous salbutamol dosage

Recommended dose ranges for continuous salbutamol infusions across the sampled CPGs are shown in Fig 2. Both the recommended starting doses and maximum doses varied considerably across guidelines. Dutch CPGs advised significantly lower starting doses (in most cases 0.1 μg/kg/min) than British CPGs (1–2 μg/kg/min) and allowed for higher maximum doses (10 μg/kg/min), compared to British CPGs (5–8 μg/kg/min). The salbutamol dosages recommended in two of seven British local CPGs and six of seven Dutch local CPGs aligned with their respective national CPGs.

Fig 2

Recommended dose ranges for continuous salbutamol infusions across sampled CPGs from the UK and the Netherlands.

Hatching indicates starting dose ranges. The arrow indicates the absence of a maximum dose recommended by the concerned guideline. UK2 and UK5: maximum doses calculated for a child of 10 kilograms based on the absolute maximum dosages recommended by each CPG.

Guideline quality

As described in Table 3, national guidelines from the UK and the Netherlands scored highest on rigour of development (92% and 78% respectively for BTS/SIGN and NVK guidance) and lowest on applicability (71% and 10% respectively for BTS/SIGN and NVK guidance). Local British guidelines and Dutch guidelines scored highest on scope & purpose (mean score of 48% across local British CPGs and 43% across local Dutch CPGs) and clarity of presentation (40% across local British CPGs and 45% across local Dutch CPGs). Local CPGs from both countries scored lowest on editorial independence (mean scores of 0% and 2% respectively for local British and Dutch CPGs). On average, local British guidelines scored lower than BTS/SIGN guidance across all AGREE II domains. Local Dutch guidelines on average scored similarly to the NVK guideline for two domains, namely clarity of presentation and applicability. Average scores for local Dutch and British CPGs were similar across all domains.

Table 3

Scaled AGREE II domain scores of sampled national and local APA guidelines from the Netherlands and the UK (%).

Guideline	1.	2.	3.	4.	5.	6.
	Scope & purpose	Stakeholder involvement	Rigour of development	Clarity of presentation	Applicability	Editorial independence
NVK	47	42	78	44	10	13
BTS/SIGN	78	83	92	72	71	83
NL1a	19	0	20	42	2	0
NL1b	50	6	9	44	13	0
NL2	69	19	2	61	15	0
NL3	50	36	7	50	10	0
NL4	36	0	5	36	4	0
NL5	47	42	78	44	10	13
NL6	31	0	0	33	8	0
NL7	42	6	6	47	10	0
Mean score local Dutch CPGs	43	14	16	45	14	2
UK1	36	8	2	28	15	0
UK2	39	11	7	31	13	0
UK3	53	22	15	42	8	0
UK4	56	22	16	56	13	0
UK5	53	14	5	42	29	0
UK6	44	17	4	36	15	0
UK7	53	42	14	50	27	0
Mean score local British CPGs	48	19	9	40	25	0

APA: acute paediatric asthma; BTS/SIGN: British Society/Scottish Intercollegiate Guidelines Network; CPGs NL: Netherlands; NVK: Nederlandse Vereniging voor Kindergeneeskunde (Dutch Society for Paediatrics); UK: United Kingdom. NL5 corresponds to the NVK guideline.

Discussion

This study demonstrated the presence of considerable variation across local CPGs from British and Dutch hospitals for the treatment of children with severe asthma, including numerous deviations from national guidance. Identified discrepancies included the choice, sequence and recommended methods of administration for the drugs advised for severe APA. Local recommendations for second-line drugs varied more than for first-line interventions, a finding that may reflect the paucity of evidence on effective treatments for severe APA [22]. Differences were also observed in the tone of recommendations made by national CPGs based on this limited evidence. The high level of agreement between national CPGs and their mostly imperative tone on first-line drugs may reflect the ease with which the available evidence could be distilled into a single treatment pathway. Faced with less certain evidence on second-line interventions [33], Dutch national guidance recommended a rigid sequence of interventions while the British national guideline took a more permissive stance, describing various available treatment options without being prescriptive. Possibly as a result of these differences, local British CPGs deviated from national CPGs and from each other, more so than local Dutch CPGs. Substantial variation was also observed in recommended continuous salbutamol infusion rates. The presence of such variation between dosing regimens for one single drug suggests that variation across the sampled CPGs extends beyond the features measured in this analysis.

The AGREE II scores obtained by national guidelines from the UK and the Netherlands were comparable to the ones assigned by Ruszczyński et al. to national and professional guidelines for the diagnosis and management of paediatric asthma [34]. The finding of higher scores for scope & purpose and on clarity of presentation and lower scores for editorial independence are in agreement with prior studies of local CPGs [35]. Lower scores may partly result from limited reporting from guideline developers for a given domain [35].

In the presence of multiple sources of clinical guidance [10, 36], local CPGs play a unique role in the translation and dissemination of evidence into local care [14]. In contrast to national CPGs, which are often a careful summary of the best available evidence including all its uncertainties, the design of local CPGs caters to their use as direct guides for action by those clinicians with less experience and time to appraise the evidence themselves as part of their decision-making [37, 38]. As such, local CPGs ought to be immediately intelligible and implementable by these clinicians. This requires simplification of the detailed knowledge contained within national CPGs and the introduction of other sources of knowledge, including expert opinion, if evidence is lacking [1, 39, 40]. Given the slowercycle of national CPG development, and the focus of national guidelines on higher-grade evidence [10, 11], local CPGs may also constitute a preferred vehicle to incorporate new evidence alongside expert insights derived from local practice [16]. The development of local CPGs, furthermore, entails adaptations in the recommendations of national CPGs in order to align with a hospital’s resources, patient populations and working processes [12, 14].

Although the intended use of local CPGs may justify deviations from national guidance, some of the variation measured across the sampled local CPGs may not arise from a lack of evidence or the need to adapt national guidance to a given hospital setting [12, 41, 42]. The examined local CPGs, in fact, often failed to declare the methods by which they were produced, or to identify or explain their deviations from national guidance. In the absence of justification such deviations may be considered a source of unwarranted variation in clinical care [42-44].

Research and policy implications

While local CPGs are widely relied upon as guides to action in a given care setting [14], research into the development of clinical guidelines has focussed on those authored by national and professional bodies. This study highlights the need to further investigate the origins of variation between local CPGs and the effects of CPG variation on variation in clinical care.

Different priorities may characterize national and local CPG development [14], and result in inconsistencies between national and local guidance. Consequently, where local guidelines deviate from national ones, justification for these deviations should be provided to help clinicians navigate this inconsistency

This study demonstrates that the treatment received by a child with acute asthma may be partly dictated by the hospital they present to. Variation in the treatment of APA, as other conditions, has been linked to poor clinical outcomes [17, 18, 20]. The extent to which this variation represents an appropriate adaptive response to local circumstances remains unknown. One driving factor in the widespread development and use of local guidelines, as revealed by this study, may be the inability of national guidance to achieve sufficient clarity to be used effectively by clinicians [45].

The widespread variation in the recommended treatment of a high-risk condition identified in this study, must be accompanied by open sharing of local CPGs. If excess harm arising from varying treatments for APA (amongst other conditions) is to be reduced, a culture of transparent, inter-organisational learning from local practice should be established. Alongside transparency of recommendations, the outcomes of local centres should be shared to enable the evaluation of the relationship between differences in treatment and differences in clinical outcomes. By sharing both local CPGs and outcomes data between providers, national bodies would be better informed of the use of these critical documents and clinicians would be better equipped to assess the potential benefits of variations to their practice.

Strengths and limitations

In this study, we applied a novel methodology to analyse and represent variation across a type of clinical guidance that remains understudied. This method could be similarly applied to guidelines for other conditions. This study had several limitations. Because local CPGs tend not to be shared beyond the hospital in which they were developed [14], local guidelines were obtained through convenience sampling. The APA guidelines obtained were from seven out of the eight tertiary paediatric hospitals in the Netherlands and from tertiary hospitals across the UK. As local CPGs were obtained from specialist paediatric centres, they may not be representative of the CPGs used in other less specialised paediatric centres.

While some of the measured variation may be assigned to the varied care settings for which the sampled local CPGs were developed (ranging from emergency departments to critical care units), it is unlikely that the breadth of variation that was measured was attributable to setting alone. In order to represent variation in a legible way across numerous guidelines, differences across local CPGs were described with reference to their respective national guidance. This was done in the absence of a clearly established customary hierarchy between national and local guidelines in the UK or in the Netherlands [45].

Several local British CPGs were last updated prior to the publication of the version of British national guidance that was examined. This may explain some of the deviations that were found. Upon comparison however, the three most recent versions of BTS/SIGN guidance on asthma, published in 2014, 2016 and 2019, did not differ in their recommendations for treating most severe APA [23, 46, 47]. The analysis focused on a single treatment pathway, and did not encompass all grades of APA severity. The drug pathway for most severe APA was chosen as the most extensive drug sequence that could be compared across the sampled guidelines, including all available APA therapies up to maximal escalation. Although this pathway was associated with severity assessment parameters that did not fully align across guidelines, the chosen pathway ensured comparability across the sample for the sickest APA patients. Whereas this study describes substantial variation across local CPGs and between local CPGs and their relevant national guidance, it does not explore the process by which this variation arises or the impact such variation has on clinical outcomes. Lastly, the scores obtained as part of the AGREE II assessment of guideline quality cannot be interpreted as a proxy for ease of implementation nor adherence to a CPG [36].

Conclusions

Local British and Dutch CPGs varied substantially in their treatment recommendations for severe acute paediatric asthma and scored poorly on quality. Although limited to a subset of guidelines, these findings may indicate the presence of considerable variation in local CPGs across medical conditions. Variation in the content and quality of local guidelines may contribute to variation of care more broadly and potentially undermine healthcare quality.

(PDF)

Click here for additional data file.

17 Jan 2022

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Maria G Grammatikopoulou

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Thank you for stating the following in the Acknowledgments Section of your manuscript:

"This article is independent research supported by grants from the National Institute for Health Research (NIHR) Imperial Patient Safety and Translational Research Centre (PSTRC) PSTRC_2016_004. Infrastructure support for this work was provided by the NIHR Imperial Biomedical Research Centre (BRC) 1215-20013. JC acknowledges support from EPSRC grant EP/N014529/1 supporting the EPSRC Centre for Mathematics of Precision Healthcare and the Wellcome Trust 215938/Z/19/Z. Funding organisations were not involved in the design of the study and collection, analysis, and interpretation of data or in writing the manuscript."

We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Acknowledgments section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form.

Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows:

"Two authors (NA and CK) had support from the National Institute for Health Research (NIHR, accessible from https://www.nihr.ac.uk/) Imperial Patient Safety and Translational Research Centre (PSTRC) PSTRC_2016_004. Infrastructure support for this work was provided by the NIHR Imperial Biomedical Research Centre (BRC) 1215-20013. JC acknowledges support from the Engineering and Physical Sciences Research Council (EPSRC, accessible from: https://epsrc.ukri.org/) grant EP/N014529/1 supporting the EPSRC Centre for Mathematics of Precision Healthcare and from the Wellcome Trust (accessible from: https://wellcome.org/), grant 215938/Z/19/Z. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the

manuscript."

Please include your amended statements within your cover letter; we will change the online submission form on your behalf

Additional Editor Comments (if provided):

Dear authors,

thank you for your submission. Please find attached the reviewer's comments, we are looking forward to your revision!

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: N/A

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The authors performed a study evaluating the variation between national and local clinical practice guidelines for acute paediatric asthma in two countries. The study is well written and the topic is interesting. However, I have some methodological concerns regarding the sampling method.

The authors used a convenience sampling method to obtain local and national guidelines. How do they handle the importing biases of this technique?

Please include all the abbreviations in footnote of Table 1.

It would be interesting also to create a new column in Table 1 with the AGREE II overall score of each guideline. Apart from variations in treatment recommendations, the quality and reporting of practice guidelines is of great importance, as it will aid pediatric asthma practitioners to select the highest quality guidelines.

The following review on quality appraisal of preschool wheezing and asthma guidelines in children should be cited and discussed: https://pubmed.ncbi.nlm.nih.gov/32816386/

Authors should include the convenience sampling method as a limitation of their study.

Reviewer #2: The study's central idea is interesting, but the sample is limited and chosen by convenience, which weakens the study. It is a local study, of little interest on a global scale, and may be submitted to a journal of regional interest. The text can be improved, making it more objective and fluid, making it easier to read. The discussion can also be deepened, carrying out a critical analysis and proposals for changes.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Marina de Barros Rodrigues

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

21 Mar 2022

Dear Editor and Reviewers,

Thank you very much for your consideration of our manuscript ‘Mind the gap: Mapping Variation between National and Local Clinical Practice Guidelines for Acute Paediatric Asthma from the United Kingdom and the Netherlands’. We have provided responses to each reviewer comment in turn below. Reviewer 1’s important suggestion we include AGREE II scores for each guideline has led to an increase in word count to describe the methods used, results obtained and to discuss their significance. We have attempted to offset this increase in word count with a tighter edit to other sections of the manuscript.

Reviewer #1:

1) The authors performed a study evaluating the variation between national and local clinical practice guidelines for acute paediatric asthma in two countries. The study is well written and the topic is interesting. However, I have some methodological concerns regarding the sampling method. The authors used a convenience sampling method to obtain local and national guidelines. How do they handle the importing biases of this technique? Authors should include the convenience sampling method as a limitation of their study.

Thank you for your helpful review of our study and your kind comments.

We agree that the method of obtaining local guidelines is subject to selection bias. In the case of our study, the only local guidelines available to the authors were from specialist (tertiary) paediatric centres. It is potentially the case that these local CPGs are not representative of those in use in other less specialized paediatric units. We have acknowledged this as a limitation in the ‘Strengths and limitations’ section of the discussion.

Local CPGs are not usually publicly available. As researchers we sought to draw upon guidelines that were available to us to begin what we feel is an important discussion about local CPG variability and transparency. We have also raised in the discussion the need for wider transparency in local CPGs to facilitate further study of variation in practice without the unavoidable selection bias of our study.

The national guidelines used in the study are the two widely used guidelines produced by the respective learned bodies of the UK and Netherlands and were therefore chosen as a single reference point for national recommendations.

2) It would be interesting also to create a new column in Table 1 with the AGREE II overall score of each guideline. Apart from variations in treatment recommendations, the quality and reporting of practice guidelines is of great importance, as it will aid pediatric asthma practitioners to select the highest quality guidelines. The following review on quality appraisal of preschool wheezing and asthma guidelines in children should be cited and discussed: https://pubmed.ncbi.nlm.nih.gov/32816386/

Thank you for this very helpful comment. In light of your suggestion, we have conducted a review of each national and local CPG according to the AGREE II instrument and have provided scores and interpretation for each domain of the AGREE II, now summarized in Table 3 of the revised manuscript. We have additionally discussed and cited the very helpful suggested review. Rather than use overall AGREE II scores for the study, we have instead reported on individual domains of AGREE II as we did not want to mask variation in scores within domains that may be insightful for the reader.

3) Please include all the abbreviations in footnote of Table 1.

Thank you for this. The abbreviations were added in a footnote of Table 1.

Reviewer #2:

1) The study's central idea is interesting, but the sample is limited and chosen by convenience, which weakens the study.

Thank you for this comment. We agree that selection bias in the inclusion of local guidelines is a limitation of this study which may impact upon the validity and generalizability of findings. We have made changes to the discussion of the manuscript to reflect this. This is also expanded upon in response to the first comment of Reviewer 1.

2) It is a local study, of little interest on a global scale, and may be submitted to a journal of regional interest.

Thank you for this comment. We acknowledge that this study is somewhat limited in its focus upon two countries with largely well-resourced health systems. However, in our decision to submit to PLoS One, we were conscious of its policy to evaluate research independently of its perceived regional or global significance.

Additionally, previous work by our group (doi: 10.1186/s12916-021-01963-0) has identified variation in national clinical practice guidelines to be a global phenomenon, yet work examining the variation in local clinical practice guidelines is very limited considering the scale of their use in everyday clinical practice. This study provides a potential framework by which other researchers may examine local CPGs in their country or clinical specialty.

3) The text can be improved, making it more objective and fluid, making it easier to read.

Thank you for this comment. All sections of the manuscript have been revised to improve the fluency of the text.

4) The discussion can also be deepened, carrying out a critical analysis and proposals for changes.

The discussion was revised in light of the findings from the AGREE II guideline quality assessment suggested by Reviewer 1, including suggestions for improving current practices around national and local clinical practice guideline development.

Best wishes,

Charlotte Koldeweij, on behalf of all the authors

Submitted filename: Response to reviewers PLoS One 17-3-22.docx

Click here for additional data file.

11 Apr 2022

Mind the gap: Mapping variation between national and local clinical practice guidelines for acute paediatric asthma from the United Kingdom and the Netherlands

PONE-D-21-30920R1

Dear Dr. Koldeweij,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Maria G Grammatikopoulou

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #1: All comments have been addressed

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: (No Response)

Reviewer #2: (No Response)

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: Yes: Marina de Barros Rodrigues

9 May 2022

PONE-D-21-30920R1

Mind the gap: Mapping Variation between National and Local Clinical Practice Guidelines for Acute Paediatric Asthma from the United Kingdom and the Netherlands

Dear Dr. Koldeweij:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Maria G Grammatikopoulou

Academic Editor

PLOS ONE