Practice variation across five European paediatric emergency departments: a prospective observational study.

OBJECTIVES: To compare paediatric healthcare practice variation among five European emergency departments (EDs) by analysing variability in decisions about diagnostic testing, treatment and admission. DESIGN AND POPULATION: Consecutive paediatric visits in five European EDs in four countries (Austria, Netherlands, Portugal, UK) were prospectively collected during a study period of 9-36 months (2012-2015). PRIMARY OUTCOME MEASURES: Practice variation was studied for the following management measures: lab testing, imaging, administration of intravenous medication and patient disposition after assessment at the ED. ANALYSIS: Multivariable logistic regression was used to adjust for general patient characteristics and markers of disease severity. To assess whether ED was significantly associated with management, the goodness-of-fit of regression models based on all variables with and without ED as explanatory variable was compared. Management measures were analysed across different categories of presenting complaints.
RESULTS: Data from 111 922 children were included, with a median age of 4 years (IQR 1.7-9.4). There were large differences in frequencies of Manchester Triage System (MTS) urgency and selected MTS presentational flow charts. ED was a significant covariate for management measures. The variability in management among EDs was fairly consistent across different presenting complaints after adjustment for confounders. Adjusted OR (aOR) for laboratory testing were consistently higher in one hospital while aOR for imaging were consistently higher in another hospital. Iv administration of medication and fluids and admission was significantly more likely in two other hospitals, compared with others, for most presenting complaints.
CONCLUSIONS: Distinctive hospital-specific patterns in variability of management could be observed in these five paediatric EDs, which were consistent across different groups of clinical presentations. This could indicate fundamental differences in paediatric healthcare practice, influenced by differences in factors such as organisation of primary care, diagnostic facilities and available beds, professional culture and patient expectations. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Large European study on paediatric practice variation in emergency departments (EDs) including the entire range of paediatric presentations.

Information on presenting complaint available.

Adjustment for important patient characteristics and markers of disease severity.

No data on differential diagnosis after assessment by ED physician or outcome.

No specific data on referral status available.

Introduction

Variability in healthcare delivery can indicate appropriate use, overuse and underuse of resources. Differences in patient characteristics, including severity and nature of presenting problems, result in differences in diagnostic and therapeutic management.1 This resulting variation in management is warranted, because different clinical problems require different management to achieve the best patient outcome.2–4

Yet variation can also arise from other factors, like differences in practice guidelines and adherence, medical tradition, patient expectations or healthcare organisation.5–9 In these instances, both deviations in management to the lower and higher end of the spectrum and higher and lower resource use can be associated with poorer outcomes or lower cost efficiency, depending on the underlying factors. Studying practice variation has therefore been acknowledged as an important tool to identify areas with potential for improvement of patient care.

Several studies have observed practice variation in the paediatric emergency setting, for specific presentations,10 11 such as minor head injury or respiratory symptoms. Other studies have focused on variability in resource use in paediatric emergency departments (EDs) in low acuity presentations. These studies reported that physician training background was associated with resource use and that diagnostic testing and procedures were less frequent in the low acuity group.12 13 Many studies have been conducted in the North American setting and not all were able to adjust for differences in patient characteristics, such as disease severity.14 Large scale European studies are scarce.

This large multicentre study aimed to compare paediatric healthcare practice among five European EDs. We wanted to analyse variability in decisions about diagnostic testing, treatment and admission, after adjustment for patient characteristics, across subgroups of presenting problems covering the broad spectrum of paediatric ED presentations.

Method

Study design, data source and study population

This study was embedded in the TrIAGE project (Triage Improvement Across General Emergency departments for paediatric patients), a prospective observational study and followed from observations in previous analyses. The study design has been described in detail elsewhere.15 In brief, during this project electronic health record data of all ED visits of children <16 years were prospectively collected in five different hospitals in four different countries. The five participating hospitals were: Erasmus Medical Centre, the Netherlands; Maasstad Hospital, the Netherlands; St. Mary’s hospital Imperial College Healthcare National Health Service Trust, UK; Hospital Prof. Dr. Fernando Fonseca, Portugal; Vienna general hospital, Austria. In the latter ED, only low urgent trauma cases presented, because the majority of patients with trauma were seen in the traumatology department.

Study sites were diverse in their catchment area and complexity of the patient population, number of visits and organisation of healthcare. Data were obtained by questionnaires obtained from the participating EDs (online supplemental appendix 1). Four EDs were paediatric EDs, and one was mixed adult-paediatric. The supervising physician was a paediatrician in all EDs, and in one site a paediatric emergency physician. The enrolment period varied from 8 to 36 months between 2012 and 2015, during which 119 209 consecutive ED visits were included. The differences in patient load account for differences in enrollment time to include sufficient patients. Also practical reasons, such as availability of staff to help in high quality data collection, played a role.

Nurses at the participating EDs were informed about the study and encouraged to be complete in their registration of routine medical data.15

Children with incomplete triage data were excluded from the analysis. Complex comorbidity has been linked to a higher use of diagnostics and therapeutic interventions at the ED.16 Children with known complex comorbidity were therefore excluded if patient-level information was available. This was the case for hospitals with high proportions of comorbidity: Erasmus MC, St Mary’s and Vienna general hospital (10%–38% comorbidity). Maasstad Hospital and Hospital Fernando Fonseca reported an estimated total comorbidity of less than 10%, and much lower proportions of complex comorbidity, and did not provide patient-level information. Comorbidity was defined according to the Paediatric Medical Complexity Algorithm.17 18

Main outcome measures

We evaluated ordering of diagnostic tests (laboratory testing and imaging at the ED), administration of intravenous medication or fluids and hospital admission. Laboratory testing included tests and cultures in blood, urine, faeces, and cerebrospinal fluid. Imaging included X-ray, ultrasound, CT and MRI. Admission was defined as admission from the ED to the general ward or paediatric intensive care unit.

Confounders

Patient characteristics (age, gender), physiological parameters (heart rate, respiratory rate, oxygen saturation, temperature), presentational flow chart and urgency according to the Manchester triage system (MTS), and presentation during office hours or during out-of-office hours were considered as potential confounding variables. Office hours were defined as Monday until Friday, between 08:00 am and 05:59 pm, and all other time points were defined as out-of-office hours. Vital signs and age were included as continuous variables.

In all participating hospitals, the MTS was routinely used for triage of presenting children. The MTS consists of 53 presentational flow charts that cover almost all presentations to EDs.19

The triage nurses are trained to select the most specific presentational flow chart. Only if there is no defining symptom at presentation the nurse will select an aspecific flow chart, like unwell child or crying baby. To ensure sufficient standardisation of triage, triage nurses using the MTS are well trained.

Presentational flow charts in turn consist of signs and symptoms that classify patients into five urgency categories, indicating the time to first contact with the treating clinician. These categories were assigned to three groups: MTS emergent or very urgent (<10 min waiting time), MTS urgent (<60 min waiting time), and MTS standard (60–120 min) or non-urgent (120–240 min waiting time).

To create subgroups of comparable presenting symptoms, we used MTS presentational flow charts. These were grouped into nine categories as defined in our previous publications: cardiac, dermatologic, ear/nose/throat, gastrointestinal, neurologic/psychiatric/intoxications, respiratory, trauma/muscular, unwell and urinary/gynaecological.15 20 Heterogeneous presentations with low frequency were grouped together as ‘other’ (online supplemental appendix 2).

In addition to the subgroups of presenting symptoms based on MTS presentational flow charts, we defined a subgroup of infectious presentations, because a suspected infection is an important reason for presentation at the ED. We defined this subgroup as children <5 years old, who had been assigned to the presentational flow chart shortness of breath or vomiting/diarrhoea or had presented with fever (defined as temperature ≥38.5°C on presentation or MTS discriminator hot child).

Statistical analysis

We evaluated ordering of diagnostic tests, initiation of treatment and hospital admission across centres, using multivariable logistic regression models, adjusting for identified confounders. In this analysis, the Maasstad hospital was (randomly) selected as the reference. Differences between EDs are expressed as adjusted ORs (aORs), relative to practice in the Maasstad hospital, with 95% CIs.

Patient characteristics and all other included variables are presented using descriptive statistics with absolute numbers, proportions, ranges and medians as appropriate. Vital signs are presented as proportion abnormal, based on the Advanced Paediatric Life Support reference values, with fever defined as a temperature ≥38.5°C.21

To assess whether ED was significantly associated with management when adjusted for confounding factors, the fit of regression models based on all variables with and without ED as explanatory variable was compared using the generalised likelihood ratio test statistic. Patients were then stratified according to categories of MTS presentational flow charts and separate regression analyses were performed within those strata. Because the ED of General hospital Vienna only treated a small proportion of trauma patients, this hospital was excluded from the analysis in the category trauma/muscular. Results of the presentational flow chart category ‘other’ are not presented, because of the inherent heterogeneity of this category.

Missing data for vital signs were imputed 25 times using the MICE algorithm in R (V.3.6.3). These missing data were assumed to be missing at random, conditional on other variables in the database. The imputation model included all predictors and outcome measures and additional descriptors of case mix: patient age and sex, date and time of arrival, and triage characteristics.15 22 Analyses were performed with IBM SPS statistics, V.25 (IBM).

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

Study Group

Of all 119 209 ED visits of patients 16 years or younger included in the TrIAGE cohort, 5706 were excluded because of complex comorbidity, leaving 113 503 who met the inclusion criteria. A total of 1581 presentations had to be excluded because of missing presentational flow chart (n=1578 presentations) or missing time of arrival (n=3 presentations), resulting in a study group of 111 922 presentations (94%).

Across the 5 EDs, the median age at presentation ranged from 3.8 to 5.7 years, and 42%–48% of children were female (table 1). Most children presented with general malaise or because of parental concern, trauma or injuries, gastrointestinal or respiratory complaints. Between 11% and 33% of children had tachypnoea at presentation an, 11%–18% tachycardia and 4%–9% had a recorded temperature of ≥38.5°C. In concordance with differences in frequency of abnormal vital signs, the casemix of patients differed among EDs with respect to MTS urgency and presentational complaint. In Erasmus and Maasstad hospital, for example, 46%–47% of patient were triaged as urgent, compared with 18%–24% of patients presenting at the three other hospitals (table 1).

Table 1

Baseline characteristics

	Emergency department
	Maasstad	Erasmus	Fernando	St Marys	Wien	Total
N	10 484	13 968	53 175	15 027	19 268	111 922
Patient characteristics
Age in yrs (median, IQR)	5.7 (1.9–11.6)	4.1 (1.3–9.8)	4.7 (2.0–9.5)	3.8 (1.5–8.7)	3.9 (1.6–8.3)	4.4 (1.7–9.4)
Gender, n % female	43.3	42.2	47.9	44.2	47.5	46.2
Abnormal vital signs (95 th percentile APLS 2017)*
Tachypnea (%)	32.9	20.3	10.8	16.9	22.3	16.9
Bradypnea (%)	1.9	5.2	7.5	1.3	4	5.3
Tachycardia (%)	18.2	12.3	12.9	14.1	10.8	13.1
Bradycardia (%)	4.4	7.9	6	4.3	10.3	6.6
Oxygen saturation <94% (%)	1.9	1.8	1.5	1.4	1	1.5
Fever (temp ≥38.5°C (%))	8	9.3	4	6.4	6.6	5.8
No of abnormal vital signs (%)
0	53.9	61.7	67	69.9	59.4	64.2
1	33.8	30.1	27.8	23.1	33.3	28.9
2	11.6	7.7	4.9	6.4	7	6.4
3	0.7	0.6	0.3	0.6	0.3	0.4
MTS urgency (%)
Emergent| very urgent	15.7	14	11.9	10.6	5.4	11.2
Urgent	47.4	45.7	20.4	24.3	18.1	26.2
Standard| non-urgent	36.8	40.3	67.7	65.1	76.5	62.5
Time of presentation (%)
Office hours	39.8	47.3	42.3	36	43.6	42.1
Out of office hours	60.2	52.7	57.7	64	56.4	57.9
Presentational flow chart categories
Cardiac	0.4	1	1.2	0.8	1.8	1.2
Dermatologic	8.5	11.8	14.3	9.9	14	12.8
ENT	1.6	3	14	4.4	14	10.2
Gastrointestinal	10	12.7	16.2	11.5	21.1	15.4
Neurologic/psychiatric	2.4	7.5	3.1	2.8	4	3.7
Respiratory	12.1	8.1	11.2	11.2	16.6	11.8
Trauma/muscular	44.3	29.9	14.7	23.2	3.3	18.6
Unwell	16.2	20.3	19	30.9	17.1	20.1
Urinary/gynaecological	1.2	2.8	2.3	1.5	2.3	2.1
Other	3.4	2.9	3.9	3.9	6	4.1
Missing values	Maasstad	Erasmus	Fernando	St Marys	Wien	Total
Heart rate	60.9% (n=6380)	51.1% (n=7138)	35.9% (n=19106)	19.6% (n=2940)	61.4% (n=11830)	42.3% (n=47394)
Respiratory rate	83.1% (n=8712)	68.2% (n=9531)	35.9% (n=19106)	23.6% (n=3544)	86.8% (n=16715)	51.5% (n= 57608)
Oxygen saturation	61.2% (N=6418)	69.4% (n=9694)	34.4% (n=18279)	19.8% (n=2973)	61.2% (n=11799)	43.9% (n=49163)
Temperature	57.9% (n=6069)	47.4% (n=6626)	12.1 % (n=6431)	32.4% (n=4872)	1% (n=194)	21.6% (n= 24192)

*Presented as percentage of measured values. Percentage of missing values of vital signs is displayed above.

ENT, ears, nose and throat; MTS, Manchester Triage System.

Management differences across EDs

Management also varied among EDs, with Vienna performing lab tests in 36% of presentations against 9.2% in St Mary’s. Likewise, imaging was performed in 24%–37% of presentations in Maasstad, Erasmus and Fernando, while in only 7.2% of patients presenting in Vienna. Differences in therapy were less pronounced but, with regard to admission, high admission rates (20%–23%) were observed in Erasmus and Maasstad, while only 4.6%–9.6% of patients were admitted in the other hospitals (table 2). Inclusion of ED as confounding variable in the multivariable regression model improved model fit for all management measures (p<0.001), indicating that management differed depending on the ED of presentation.

Table 2

Management per ED

	Maasstad	Erasmus	Fernando	St Marys	Wien	Total
N	10 484	13 968	53 175	15 027	19 268	111 922
Diagnostic
Lab any (%)	20	28.5	13.1	9.2	35.8	19.1
Imaging any (%)	37.2	24.9	23.7	14.2	7.2	21
Therapy
Intravenous medication or fluids (%)	12.8	9.5	7.5	4.1	4	7.2
Admission
General admission/ICU admission (%)	23.4	20.3	5.2	9.6	4.6	9.3
ICU admission (% of total)	0.2	2.2	0.3	0.1	0	0.4

ED, emergency department; ICU, intensive care unit.

Management differences within presentational flow chart categories

Because management will be guided by presenting complaint, we assessed differences in management across EDs in children with comparable presenting complaints. The size of presentational flow categories relative to total presentations varied per hospital. The MTS urgency within categories also differed, with higher MTS urgency in Maasstad and Erasmus, indicating differences in patient populations between EDs (table 1, figure 1).

Figure 1

aOR for management according to presentational flow chart categories. aOR, adjusted OR.

In most presentational flow chart categories we observed, after adjusting for patient characteristics, time of presentation and markers disease severity, that patients presenting in Vienna and, for some categories, Erasmus MC, were more likely to receive lab testing. Patients presenting in Fernando were more likely to receive imaging in the majority of categories, followed by Maasstad and Erasmus MC (figure 1). Intravenous administration of medication or fluids was more likely in Maasstad hospital and, in some categories, in Erasmus MC and Fernando, compared with other hospitals. Admission was more likely in Maasstad hospital, followed by Erasmus MC. The chance of admission was consistently lower elsewhere after adjustment for other parameters, with the exception of smaller categories with broader CIs. One ED had an overall average or lower likelihood of medical interventions (St Mary’s), but for other EDs, instead of overall high or low resource use, there were specific interventions that were performed more or less likely within EDs (figures 1 and 2). The likelihood of administration of intravenous medication and admission seemed to vary in parallel directions.

Radar charts presenting aOR for management outcome measures in the five largest presentational flow chart categories. aOR, Adjusted OR.

Subanalysis in infectious children

An additional regression analysis was performed in the subgroup of young children with suspected infectious diseases. Similar patterns of variability in management across EDs were observed (table 3). Lab testing was more likely in Vienna and in Erasmus MC, imaging more likely in Fernando, intravenous medication and admission more likely in Maasstad hospital, followed by Erasmus MC. This means that, in this more homogeneous group of children, there was no apparent lower variability in m anagement among different EDs.

Table 3

aOR for infectious children <5 years

aOR for management in infectious children (n=23695)	Any lab tests		Any imaging		Intravenous medication or fluids		Admission (ICU and general)
	aOR	95% CI	aOR	95% CI	aOR	95% CI	aOR	95% CI
Maasstad	Reference		Reference		Reference		Reference
Erasmus	2.64	(2.33 to 2.99)	3.66	(2.93 to 4.56)	0.63	(0.53 to 0.73)	0.75	(0.65 to 0.85)
Fernando	0.89	(0.79 to 1.00)	6.91	(5.63 to 8.48)	0.43	(0.38 to 0.50)	0.19	(0.17 to 0.22)
St Marys	0.36	(0.30 to 0.41)	1.99	(1.57 to 2.52)	0.25	(0.20 to 0.30)	0.33	(0.29 to 0.38)
Wien	2.88	(2.54 to 3.27)	2.85	(2.28 to 3.57)	0.36	(0.30 to 0.43)	0.17	(0.15 to 0.20)

Based on MTS flow chart ‘diarrhoea and vomiting’ or ‘shortness of breath’, or based on presence of fever (MTS discriminator hot child/adult or temp ≥38⁰C)). ORs are adjusted for age, gender, MTS urgency category, heart rate, respiratory rate, oxygen saturation, temperature and time of presentation.

aOR, adjusted OR; ICU, intensive care unit; MTS, Manchester Triage System.

Discussion

In this large observational study of paediatric practice variation across five European EDs, management was associated with ED of presentation. We observed ED-related patterns of variability in the likelihood of diagnostic testing, intravenous medication and admission, which remained stable across groups of clinical presentations, after correcting for several general patient characteristics and markers of disease severity known to be associated with management. Though one ED had overall low resource use, there were large differences across other EDs in likelihood for imaging or laboratory testing, after adjusting for the differences in disease severity and presenting symptoms that were observed between hospitals.

Other unmeasured medical and non-medical factors are likely to play a role in hospital-specific patterns of variability. The proportion of self-referred patients differed greatly among hospitals (online supplemental appendix 1). Reasons for primary care physicians to refer to an ED include available diagnostic facilities, request for a professional opinion, or expected need for in-hospital treatment.6 This means that disease characteristics of referred and non-referred presentations are likely to differ. These factors could partly be adjusted for by the measures of disease severity and presenting symptoms.

Prior out-of-hospital diagnostics and treatment will also influence management at the ED. The higher rate of referrals by primary care physicians in Maasstad hospital and Erasmus MC could account for the higher likelihood of admission to these hospitals, as has been reported previously.7 23 Parent and patient expectations regarding management differ between self-referred and referred patients. Presentation at ED without prior consultation of the primary care physician can be triggered by parental perceptions of disease severity and the expectation that specific diagnostic facilities or treatment available at the ED are required.6 24–27 This can also stimulate healthcare providers to perform additional testing or influence their treatment decisions.28 However, referral status only cannot explain the variability in management that was observed in the three hospitals with comparably low referral rates.

A myriad of other factors has been linked to clinical management. Financial incentives embedded in the organisation of healthcare systems could differ across EDs. National or local professional culture, standard of care and facilities might partly account for the observed variability, such as preferences for lab testing, imaging, and the availability thereof.29–31 Differences in practice guidelines, reflecting these differences in professional culture and diagnostic options, could also be of influence. These are neither harmonised across European countries, nor is adherence likely to be comparable across EDs. Holding varying guideline recommendations regarding lab tests and imaging partly responsible for the observed patterns would reflect international differences in the general value placed on specific diagnostic tests, regardless of disease presentation, as the differences in additional testing were rather consistent and independent of presenting complaint.

Parent and patient expectations and preferences regarding healthcare are affected by cultural and socioeconomic factors. These, in turn, influence management decisions and could represent another non-medical factor contributing to the observed variability.28 32 Professional education and training have been reported to be associated with management, where paediatric specialty training was linked to a lower amount of diagnostic testing.5 33 34 However, in our study there was no difference in respect to those factors among hospitals with higher and lower likelihood of testing.

Strengths/limitations

A major strength of this study is that we could adjust for several relevant patient characteristics and markers of disease severity, due to the availability of triage urgency data, presentational flow chart, vital signs and basic patient characteristics. We could include a large sample of patients from different European countries. This is an advantage, because these differences can help in identifying relevant factors responsible for practice variation, but also represents a limitation, since individual effects could not be disentangled. Hospitals differed in multiple characteristics, such as the availability of primary care physicians, rate of self-referrals, and patient casemix. Patient-specific data on referral were not available for all hospitals, and referral status could therefore not be included in the regression analyses. In addition, the availability of resources, including staffing and beds, could vary during the project, but exact data were missing for our analysis.

We used the selected MTS presentational flow chart as a proxy for presenting symptoms. In the course of the evaluation at the ED, the initial impression will have changed in a proportion of children, due to the elucidation of other signs and symptoms, which could lead to adjustments to the differential diagnosis and changes in subsequent management steps. Because we had no data on differential diagnosis and final diagnosis, we could only stratify according to presenting symptoms. The remaining heterogeneity of patients within categories and between EDs will have contributed to the observed variability in management. We did not have patient outcome measures available, therefore the consequences of deviations, compared with the benchmark, could not be assessed in terms of effects on outcomes.

Implications

Our analysis revealed substantial variability in management, even after adjustment for relevant patient characteristics and markers of disease severity. We acknowledge that not all practice variation is unwarranted or problematic, because contextual and patient-related factors such as those described above can cause variation that is not associated with lower quality care.35

However, we believe that our findings of consistently higher likelihood of lab testing or imaging in some hospitals, compared with others, are sufficient reason to further study underlying reasons for these patterns. In that sense, ours can serve as a pilot study. As a starting point, deviations from the benchmark should prompt a general exploration of potential explanations, and how these deviations might affect patient outcome. In a second step, a review of recent guidelines and review syntheses, combined with an assessment of adherence to guidelines, could provide further insights. An accessible and feasible approach could be to increase awareness of practice guidelines during handover and rounds on a case level. Both by following recommendations with a strong evidence base for a well-defined population in favour of providing healthcare actions, and by following recommendations against certain practices because of insufficient added value, quality of care will be improved and variation will be reduced.

A related study focusing on febrile children found that admission varied across European EDs, after adjusting for explanatory variables comparable to the ones in our study but also for management at the ED, pointing to other factors than disease characteristics.36 Factors related to organisation of healthcare and local culture of care will likely play an important role. Though more difficult to influence, comparing and learning from differences in organisation and medical culture can be a first step to long-term changes, to ensure an sustainable healthcare system. The number of EDs required for a study searching to assess the importance of these factors depends on the heterogeneity of the EDs and healthcare systems, and on the research question. Such evaluation should preferentially involve patient important outcomes and prior out-of-hospital management, to assess the entire trajectory of care and to produce suggestions for improvements.

Conclusion

In this analysis of paediatric healthcare practice among five European ED distinctive hospital-specific patterns in variability of management could be observed, which were consistent over different groups of clinical presentations. This pattern in variability could indicate fundamental differences in paediatric healthcare practice across countries, influenced by factors such as organisation of primary care, diagnostic facilities and available beds, professional culture and patient expectations.