Assessment of documented adherence to critical actions in paediatric emergency care at a district-level public hospital in South Africa.

INTRODUCTION: The provision of high-quality care is vital to improve child health and survival rates. A simple, practice-based tool was recently developed to evaluate the quality of paediatric emergency care in resource-limited settings in Africa. This study used the practice-based tool to describe the documented adherence to critical actions in paediatric emergency care at an urban district-level hospital in South Africa and assess its relation to clinical outcomes.
METHODS: This study is a retrospective observational study covering a 19-month period (September 2017 to March 2019). Patients <13 years old, presenting to the emergency centre with one of six sentinel presentations (seizure, altered mental status, diarrhoea, fever, respiratory distress and polytrauma) were eligible for inclusion. In the patients' files, critical actions specific for each presentation were checked for completion. Post-hoc, a seventh group 'multiple diagnoses' was created for patients with more than one sentinel disease. The action completion rate was tested for association with clinical outcomes.
RESULTS: In total, 388 patients were included (median age 1.1 years, IQR 0.3-3.6). The action completion rate varied from 63% (polytrauma) to 90% (respiratory distress). Participants with ≥75% action completion rate were younger (p < 0.001), presented with high acuity (p < 0.001), were more likely to be admitted (adjusted OR 2.2, 95%CI: 1.2-4.1), and had a hospital stay ≥4 days (adjusted OR 3.4, 95%CI: 1.5-7.9).
CONCLUSION: A high completion rate was associated with young age, a high patient acuity, hospital admission, length of hospital stay ≥4 days, and the specific sentinel presentation. Future research should determine whether or not documented care corresponds with the quality of delivered care and the predictive value regarding clinical outcome.

African relevance

Presents a first step in the assessment of documented paediatric emergency care within resource-limited settings

Highlights the importance of the quality of paediatric emergency care

Highlights deficiencies in the documentation of local health-care, from which other resource-limited settings might learn

Highlights the importance of good record-keeping

Introduction

Worldwide, the burden of paediatric emergency care is high as children represent a quarter of all emergency centre visits in well-resourced and low-resourced countries [[1], [2], [3], [4], [5]]. The provision of high-quality care is vital to improve child health and survival rates [ [1,[3], [4], [5]]. Evidence exists that paediatric-specific standards of care improve paediatric emergency care in resource-limited settings [6,7]. Minimum standards required to provide safe and effective care for acutely ill children in emergency centres have been established by international organisations such as the World Health Organization and the International Federation of Emergency Medicine [8,9]. These standards apply to any emergency care system in any setting and do not mandate the need for highly specialised equipment, staff or facilities.

Few studies have reported on the quality of paediatric emergency care. Those studies have focused on metrics used for quality assessment, the quality of triage, timeliness of the initial assessment, specific diseases, available resources or observed changes in care after implementation of guidelines or training [10,11].

A simple, practice-based tool was recently developed to evaluate the quality of paediatric emergency care in resource-limited settings in Africa [11]. Critical actions for the initial management of severely ill children presenting with one of six sentinel conditions requiring emergency care were identified by a group consensus process [11]. Actions were selected that, if absent, would reflect a modifiable gap in the delivery of quality care. The six selected presentations (active seizure, fever, altered mental status, respiratory distress, diarrheal illness, polytrauma) occur frequently and carry high mortality rates in Africa. With this practice-based tool, this study aimed to describe the documented adherence to critical actions in paediatric emergency care and assess the relation to clinical outcomes at an urban district-level hospital in Cape Town, South Africa.

Methods

Study design

This is a retrospective observational study covering a 19-month period (September 2017 to March 2019) of consecutive paediatric presentations to the emergency centre of Khayelitsha Hospital in Cape Town, South Africa. The study was approved by the Health Research Ethics Committee of Stellenbosch University, and a waiver of consent was obtained. The STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) checklist was used to guide the report [12].

Setting

Khayelitsha Hospital is a district-level hospital situated in the partially informal settlement of Khayelitsha, 30 km outside of Cape Town. The hospital experiences a high poverty-related disease burden, including HIV, tuberculosis and malnutrition [13., 14., [15]]. The general emergency centre has a resuscitation area with four beds (for adults and children) and one infant warmer. The paediatric specific area within the general emergency centre has three acute care cots, with an additional six cots in an adjacent ward. Patients are attended by medical officers with supervision from a specialist emergency physician, managing between 700 and 1200 children aged <13 years per month. Patients are triaged into different categories using the paediatric version of the South African Triage Scale (SATS) [16]. Colours represent the acuity of cases along with the proposed urgency of management required: ‘Red’ – Emergent (immediate), ‘Orange’ – Very urgent (within 10 min), ‘Yellow’ – Urgent (within 1 h), and ‘Green’ – Non-urgent (>1 h) [16]. Severely ill children (typically triaged as Orange or Red) are managed within the resuscitation area. The mortality rate of those managed in the resuscitation area is 85 per 1000 live births [17].

Practice-based tool

The tool was developed to evaluate the quality of paediatric emergency care within the first hour of presentation [11]. Each of the six sentinel presentations (active seizure, fever, altered mental status, respiratory distress, diarrheal illness, polytrauma) has a list of critical actions requiring completion. The algorithm contains a shortlist of actions which are consistent with existing guidelines and are near-universally indicated for the presentation. The actions do not represent the minimum standards of care but were selected to ensure use in various environments by minimally-trained practitioners present in the room [11]. The tool was adjusted to local epidemiology by removing testing for malaria as this is not endemic.

Participants

The patient registers in the resuscitation and paediatric areas were searched for potential participants for the defined study period. Patients were eligible if they were <13 years old at presentation to the emergency centre, triaged as urgent or higher (Yellow, Orange or Red) and presenting with at least one sentinel presentation. The defining criteria for each sentinel presentation are presented in Appendix B. Exclusion criteria were no signs of life on arrival, missing patient folder, or absence of notes of an initial assessment. Patients with non-sentinel diseases or triaged as non-urgent (green) were not counted.

Data collection and management

Data was collected using a standardized form (Appendix C). A pilot study was completed using five patient folders per presentation (30 in total) to train the chart reviewers adequately and to adapt the data capture form where needed; data were included. Data collection was performed by one reviewer (EAB). Independently from this reviewer, a second reviewer (EE) recollected data of a 10% random sample. The data collection of the first reviewer was used for analysis.

Critical actions were marked completed when specifically documented. The first documented date and times were extracted. Completion of ‘assess breathing’ and ‘perform physical exam – of at least 3 systems’ was assumed when any note pertaining to the system was made. The action, ‘expose patient’, was marked completed when the examination of multiple body areas was noted or drawn within the given body picture on the admission form. Intravenous (IV) access was assumed with documentation of the administration of IV medication or fluids. A full septic workup consisted of blood culture, full blood count, urine testing and lumbar puncture [18].

Data analysis

A seventh group was created post hoc. Participants presenting with more than one sentinel presentation were deemed to represent a more complex group than those with a single sentinel presentation and were included in a ‘Multiple diagnosis’ group.

The distribution of continuous data was analysed with histograms. If normality was present, mean and standard deviation (SD) were calculated. Median and interquartile ranges (IQR) are presented if data were non-normally distributed. Log-transformation for data was used to achieve normality, if this was not achieved, data were dichotomized. The denominator for the action completion rate was the total number of critical actions required per presentation. In participants with multiple presentations, the denominator was the total of all actions required for those presentations combined. Low adherence was defined as <75% of critical actions completed.

All seven groups were compared using the Fisher's exact or chi-square test for binary outcomes and student's t-tests for continuous outcomes. The inter-rater agreement between reviewers was measured using κ statistics with 95% confidence intervals [19]. Multivariable regression analysis was used to identify associations between the action completion rate (determinant) and patient outcomes (disposition from emergency centre, length of hospital stay, length of Paediatric Intensive Care Unit (PICU) stay, in-hospital mortality). The first 24 h of admission were deemed as day 0. High-risk factors of poor outcomes were identified in the literature and include patients who are young, present after hours (17 h00–08 h00), have a high acuity or experience long waiting times [[20], [21], [22], [23]]. To assess the effect of these potential confounders and effect modifiers, variable relating to age, time of presentation, triage category, waiting time before seen by a physician, and specific sentinel presentation were included in the regression analysis. Confounding was deemed present if 10% or more difference in the regression coefficient (β) was observed when the variable was added to the regression model. Effect modification was present if interaction terms were significant. The odds ratio (OR) with 95% confidence interval (CI) was used as the measure of association. Significance was considered at a 5% level.

Analyses were performed using SPSS Statistics for Windows, version 23 (IBM Corp. Released 2019. Armonk, NY: IBM Corp.).

Sample size

Convenience sampling was used and the study period was determined by the availability of patient registers. At least 100 participants would have been needed in each of the sentinel presentation groups to detect a 10% difference in the action completion rate (significance level 5%, power 90%).

Results

Study population

A total of 685 eligible patients presented within the stated timeframe and were triaged yellow, orange or red. We included 388 (56.6%) patients. Seventy-two participants had multiple concurrent sentinel presentations (Fig. 1).

Fig. 1

Flowchart of study population.

The median age of participants was 1.1 years (IQR 0.3–3.6) and varied from 0.4 years (IQR 0.2–1.2) in the respiratory distress group to 6.6 years (IQR 4.5–9.2) in the polytrauma group. (` 1) The number of participants aged <5 years was 311 (80.1%). Most participants had a high acuity (triaged Orange or Red, n = 271, 69.8%), and most presented during day and evening shifts (08 h00 to 23 h59, n = 321, 82.7%). Half of the participants were seen within the target time of <60 min (n = 196, 50.5%), 117 participants within 30 min (30.2%). A total of 118 participants (30.4%) were directly discharged home from the emergency centre, 123 (31.7%) admitted to the general paediatric wards of Khayelitsha Hospital, 111 (28.6%) to the general paediatric wards of a tertiary hospital, and 31 (8.0%) were admitted to the PICU of a tertiary hospital. Five participants (1.3%) died in the emergency centre. The median hospital stay was 4 days (IQR 2–8) and the median PICU stay was 4 days (IQR 2–7).

Table 1

Demographic data of paediatric participants presenting to the emergency centre with one or more of the six sentinel presentations.

	Seizure	Altered mental status	Diarrhoea	Fever	Respiratory distress	Polytrauma	Multiple diagnoses	Total
Total n (%)a	36 (9.3)	20 (5.2)	79 (20.4)	55 (14.2)	85 (21.9)	41 (10.6)	72 (18.6)	388 (100)
Gender: Female	23 (63.9)	8 (40.0)	36 (45.6)	28 (50.9)	36 (42.4)	16 (39.0)	34 (47.2)	181 (46.6)
Median age at presentation (Q1 – Q3)	3.9 years (1.7–8.7)	6.4 years (0.4–9.1)	221 days (94–457.5)	1.4 years (0.3–2.6)	142 days (83–456)	6.6 years (4.5–9.2)	269 days (61.5–951.8)	1.1 years (0.3–3.6)
Acuity of casesb
Urgent (yellow)	1 (2.8)	6 (30.0)	33 (41.8)	19 (34.5)	25 (29.4)	22 (53.7)	11 (15.3)	117 (30.2)
Very urgent (orange)	6 (16.7)	8 (40.0)	35 (44.3)	29 (52.7)	51 (60.0)	12 (29.3)	31 (43.1)	172 (44.3)
Emergent (red)	29 (80.6)	6 (30.0)	11 (13.9)	7 (12.7)	9 (10.6)	7 (17.1)	30 (41.7)	99 (25.5)
Disposition from emergency centre
Discharged home	8 (22.2)	2 (10.0)	21 (26.6)	30 (54.5)	27 (31.8)	18 (43.9)	12 (16.7)	118 (30.4)
Admitted paediatric department at Khayelitsha hospital	7 (19.4)	4 (20.0)	31 (39.2)	18 (32.7)	35 (41.2)	6 (14.6)	22 (30.6)	123 (31.7)
Admitted to general paediatric ward at referral hospital	18 (50.0)	6 (30.0)	25 (31.6)	5 (9.1)	22 (25.9)	16 (39.0)	19 (26.4)	111 (28.6)
Admitted to PICU at referral hospital	3 (8.3)	7 (35.0)	2 (2.5)	2 (3.6)	1 (1.2)	1 (2.4)	15 (20.8)	31 (8)
Death in emergency centre	0 (0.0)	1 (5.0)	0 (0.0)	0 (0.0)	0 (0.0)	0 (0.0)	4 (5.6)	5 (1.3)
Median in-hospital stay, daysc (Q1 – Q3)	2 (1.0–11.0)	4 (1.5–9.5)	4.5 (3.0–7.25)	4.5 (3.0–10.0)	5 (3.0–8.0)	3 (1.0–4.5)	5 (3.0–9.0)	4 (2.0–8.0)
Median PICU stay, daysc (Q1 – Q3)	5d	2 (1.0–3.0)	8 (0.5–16.3)	7.5 (0.5–28.0)	7d	2 (2.0–20)	5 (2.0–8.0)	4 (2.0–7.5)
In-hospital deaths	2.8 (1.0)	4 (20.0)	0 (0.0)	0 (0.0)	1 (1.2)	0 (0.0)	5 (6.9)	11 (2.8)

Abbreviations: PICU = Paediatric Intensive Care Unit, Q1 – Q3 = 25th – 75th percentile. Discharge data for 4 patients were missing, in 1 of those cases PICU discharge data was also missing.

Unless otherwise specified.

According to the South African Triage Scale.

First 24 h of admission are included as day 0.

Not enough cases to calculate quartiles.

Eleven participants (2.8%) died whilst in-hospital; the median age was 0.7 years (IQR 0.3–1.5), and only one participant was >5 years of age. Most of the deceased participants presented during daytime shifts (n = 8, 72.3%), whilst an altered mental status (n = 4, 36.4%) and multiple diagnoses (n = 5, 45.5%) were the most frequent presentations. The action completion rate in the deceased participants varied between 67% and 92%. (Appendix D).

Adherence to critical actions

The interrater agreement on adherence to critical actions per sentinel presentation between the reviewers was substantial (κ = 0.70, 95% CI, 0.63–0.77). (Appendix E).

The overall mean action completion rate was 82.4%. The rate varied substantially between groups; the respiratory distress group had the highest rate (90.3%) and the polytrauma group the lowest (62.2%) (Fig. 2). The mean action completion rate in the polytrauma group was >10% lower than in other groups (p < 0.001). Respiratory distressed participants also had a higher completion rate than participants with an altered mental status (p < 0.001). No other significant differences between groups were observed.

Fig. 2

Mean percentages of action completion rate per sentinel presentation for paediatric participants presenting to the emergency centre.

The action completion rate was non-normally distributed, documentation of adherence to individual critical actions varied between 2.1% and 100%. (Appendix F) Nine (39.1%) critical actions had low adherence (<75% critical actions completed) and included the primary survey, vital signs, physical exam and studies (investigations).

Potential confounders of action completion rates were age (p < 0.001), patient acuity (p < 0.001) and the specific sentinel presentation (p = 0.005) (Table 2). We found no effect modifiers. Participants with a high action completion rate (≥75%) had significantly more hospital admissions (adjusted OR 2.2, 95%CI: 1.19–4.1) and a longer hospital stay (≥4 days, adjusted OR 3.4, 95%CI: 1.5–7.9) than those with a low rate (<75%). No association was found between the action completion rate and admission to the PICU (adjusted OR 1.9, 95%CI: 0.5–6.9) or in-hospital mortality (adjusted OR 1.2, 95%CI: 0.1–11.9) (Table 3).

Table 2

Potential confounders of action completion rates in paediatric participants presenting to the emergency centre with one or more of the six sentinel presentations.

	Percentage of critical actions completed n (%)
	<75%	≥75%	All cases	p-Value
	n = 88 (22.7)	n = 300 (77.3)	n = 388
Median age at presentation, in years (Q1-Q3)	5.1 (1.3–8.2)	0.74 (0.2–2.2)	1.1 (0.3–3.5)	<0.001⁎
Time of presentation				0.062
00:00–07:59	10 (11.4%)	57 (19.0%)	67 (17.3%)
08:00–15:59	34 (38.6%)	120 (40.0%)	154 (39.7%)
16:00–23:59	44 (50.0%)	123 (41.0%)	167 (43.0%)
Patient acuitya				<0.001⁎
Urgent (yellow)	46 (52.3%)	71 (23.7%)	117 (30.2%)
Very urgent (orange)	30 (34.1%)	142 (47.3%)	172 (44.3%)
Emergent (red)	12 (13.6%)	87 (29.0%)	99 (25.5%)
Waiting time before seen by physician				0.314
<30 min	32 (36.4%)	98 (32.7%)	130 (33.5%)
30–60 min	15 (17.0%)	51 (17.0%)	66 (17.0%)
60–120 min	14 (15.9%)	34 (11.3%)	48 (12.4%)
>120 min	27 (30.7%)	117 (39.0%)	144 (37.1%)
Sentinel presentationb				0.005⁎

Abbreviations: Q1 – Q3 = 25th – 75th percentile.

According to the South African Triage Scale.

Outcome measured for all presentations.

Significant on a 95% CI level.

Table 3

Critical action completion rate per clinical outcome in paediatric participants presenting to the emergency centre with one or more of the six sentinel presentations.

	Percentage of critical actions completed n (%)
	<75%	≥75%	All cases
	n = 88 (22.7)	n = 300 (77.3)	n = 388
Disposition from Emergency Centre
Discharged home	43 (48.9)	75 (25.0)	118 (30.4)
Admitted to in-hospital paediatric department	15 (17.0)	108 (36.0)	123 (31.7)
Admitted to referral hospital general paediatric ward	25 (28.4)	86 (28.7)	111 (28.6)
Admitted to referral hospital PICU	4 (4.5)	27 (9.0)	31 (8.0)
Death in emergency centre	1 (1.1)	4 (1.3)	5 (1.3)
Length of hospital staya
<4 days	28 (63.6)	74 (33.5)	102 (38.5)
≥4 days	13 (29.5)	146 (66.1)	159 (60.0)
Discharge data missing	3 (6.8)	1 (0.4)	4 (1.5)
PICU staya
<4 days	4 (100)	13 (39.4)	17 (45.9)
≥4 days	0	20 (60.6)	20 (54.1)
In-hospital deaths	1 (1.1)	10 (3.3)	11 (2.8)

First 24 h of admission are included as day 0. Dichotomized values of length of hospital and PICU stay were based on a median split. Length of hospital stay was not calculated for patients who died in the emergency centre. Length of PICU stay was determined for all patients that at some point stayed in the PICU. Abbreviations: PICU = Paediatric Intensive Care Unit.

Discussion

This study assessed the association between the documented critical action completion rate and clinical outcomes in a district-level African emergency centre. The action completion rate varied substantially, and high completion rates were associated with young age, a high patient acuity, hospital admission, length of hospital stay ≥4 days, and the specific sentinel presentation. Participants presenting with respiratory distress had the highest action completion rate and the polytrauma group the lowest.

High action completion rates were mostly associated with sicker participants and could relate to the physicians' clinical gestalt. Subjective identification of critically ill children in the African setting is similar to clinical prediction scores [25]. It is also known that high patient acuity is associated with hospital admission and prolonged hospital stay [26,27], and although the practice-based tool was neither designed nor validated as a prediction tool, it seems to relate well with the patients' extended care needs. This reflects well on the selection of critical actions originally incorporated in the tool.

The low mortality rate was reassuring. Interestingly, four of the five children who died in the emergency centre (and 8 of 11 in-hospital deaths) presented during regular hours. This is similar to a UK study which indicated that patient-level differences at admission are associated with mortality rather than reduced hospital staffing or services [28]. A further explanation could relate to the socioeconomic status of the population served by the hospital. Many do not have their own transport and would thus have to wait for public transport services before being able to go to hospital. The high rate of violence in the area could also cause parents to wait till daylight before venturing outside.

Strengths and weaknesses

Strategies to improve accuracy and minimize inconsistencies were implemented. A pre-piloted standardized data collection form was used after it was adapted to local epidemiology (removing malaria testing). Variables were explicitly defined, and emergency centre personnel were unaware of the upcoming study. A random sample of the data was re-collected by a second reviewer, and the substantial interrater agreement reflects the reliability of the results. However, reviewers were not blinded to the study objectives. Another limitation includes the small sample size per sentinel presentation. Many eligible patients were excluded due to missing patient folders. The post hoc creation of the seventh group further reduced the participants' numbers per group. For future studies all paediatric patients presenting to the emergency centre within the given period should be counted, extracting information on their triage group and sentinel diagnoses to be able to compare different districts and countries with each other.

Lastly, the tool was designed for ‘real-time’ use by an observer and contrasts with the retrospective design of the study. Although the tool has not yet been validated, it has been specifically designed for resource limited settings. It has therefore allowed us to do a resource-specific systematic and reproducible assessment of the documentation of paediatric emergency care. The tool could also be used to re-assess documentation after quality improvement strategies were put in place. The tool however does not account for those with multiple diagnoses, whilst this is a more complex group with more severe clinical outcomes. We therefore advise future prospective studies to also include this seventh group. We acknowledge that the study essentially reflects record-keeping that could have underestimated the number of actions completed. However, the study provides critical information on paediatric emergency care that can be used as a stepping stone to future research in the area.

Interpretation in terms of mechanisms

Participants presenting with respiratory distress had the highest action completion rate and the polytrauma group the lowest. Children frequently present with respiratory-related problems to the emergency centre and are the most frequent presentation in children managed in triage and the resuscitation area of Khayelitsha Hospital [17]. One explanation could be that physicians document critical actions to allow them to assess response to treatment or that they expect these patients to end up in PICU which would require adequate assessment (and documentation) prior to referral. This could also explain why documentation was most thorough in patients who were admitted to the hospital (especially when they had a hospital stay of ≥4 days) and in the deceased patients.

On the other hand, paediatric polytrauma cases occur less frequently, although the hospital still manages a fair amount of paediatric trauma cases [17]. A possible explanation for the low adherence rate in the polytrauma group might relate to the challenges of adequate record-keeping during trauma resuscitations [24], especially since the polytrauma checklist requires precise documentation. However, the printed body picture on the admission form could easily be used to ensure good documentation. As well as that patients in severe pain are triaged with high acuity but might be, apart from the pain, in a good clinical condition. This mechanism of triage might also be the reason of the high amount of discharges in this patient group.

Allmost half of the patients (n = 192, 49.5%) was not seen by a clinician within 60 min, the largest amount was not seen within 120 min (n = 144, 37.1%). We also saw a large number of discharges (n = 118, 30.4%) from the emergency centre in general, potentially suggesting that these patients (triaged as urgent or higher) might not have been as severely ill and were over-triaged. A fine balance is needed between undertriage and over-triage. The initiation of acute care might be delayed if patients were inappropriately triaged to a less severe category; potentially worsening the clinical outcome. On the other hand, the negative impact of over-triaging would be minimal for the patient, but could indirectly affect other appropriately triaged patients if the healthcare sytem gets overburdened. It might well be that the best approach in children is to err on the side of caution, but more research is needed where the outcome of discharged patients are included.

Conclusion

This study has identified potential gaps in documentation of paediatric emergency care. The documented action completion rate varied substantially, however a completion rate of >80% was accomplished in five of the seven sentinel presentations. A high completion rate was associated with young age, high patient acuity, hospital admission, length of hospital stay ≥4 days, and the specific sentinel presentation. Future research should focus on the correlation between triage, documented care, delivered care and clinical outcomes. A minimal action completion rate that corresponds to good quality care should still be determined as well as the predictive value of the tool regarding clinical outcomes.

The study was not funded.

Dissemination of results

Results from this study were shared with staff members at the data collection site through an informal presentation. Results have contributed to improvement efforts made locally.

Author's contribution

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: EAB contributed 55%; EE 10%, NRV 15%, DJvH 15%, SNM and SL each 2.5%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Declaration of competing interest

The authors declared no conflicts of interest.