Drug dosing errors in simulated paediatric emergencies - Comprehensive dosing guides outperform length-based tapes with precalculated drug doses.

BACKGROUND: The accuracy of drug dosing calculations during medical emergencies in children has not been evaluated extensively. The objectives of this study were to evaluate the accuracy of drug dose calculations using the Broselow tape, the PAWPER XL tape plus its companion drug-dosing guide, a custom-designed mobile phone app and no drug-dosing aid (control group).
METHODS: This was a prospective study in which 32 emergency medicine volunteers participated in eight simulations of common paediatric emergency conditions, using children models. The participants used the three methods to estimate the children's weight and calculate drug doses. The accuracy of and time taken for the drug dose determinations were then evaluated for each of the methods.
RESULTS: The overall accuracy of drug dose determinations was extremely and potentially dangerously low in the control group in which no dosing guide was used as well as in the Broselow tape group (<20% of doses were correct). The accuracy was significantly higher with the PAWPER XL tape group and the mobile app group (47% and 31% respectively). The times taken to obtain the required information did not differ in a clinically meaningful magnitude.
CONCLUSIONS: Both an accurate weight estimation and a dosing guide with comprehensive information were necessary to produce an accurate prescription. The information on the Broselow tape was not sufficient for this purpose. The current guidelines recommending the use of tapes with limited information should be revised. The results from the comprehensive dosing guides were substantially better, but still had a lower proportion of accurate prescriptions than desirable. The role of training in every aspect of the emergency paediatric weight estimation and drug dosing procedure cannot be underestimated and should be routine in any environment where emergency care may be needed.

African relevance

Resuscitation guidelines with regard to drug dosing need to be reviewed - this would also impact in practice in Africa.

The Broselow tape is widely used in Africa, and may be harmful to be used in this environment.

Resource limited environments might be more at risk of poor outcomes because of limited awareness of these current issues.

Introduction

The purpose of weight estimation in paediatric emergencies is principally to enable the calculation of critically-needed drug doses [1]. While many studies have evaluated the accuracy of different emergency weight estimation systems in children, few studies have evaluated a more important end-point: the accuracy of the drug dose calculations. The effects of cognitive stress during emergencies, the complexity of the weight estimation system and the use of an appropriate drug-dosing aid may all influence the accuracy of drug dosing determination as much as the accuracy of the weight estimation itself [[2], [3], [4], [5]]. Resuscitation aids which contain comprehensive information on drug dilution, preparation and volume-to-administer (complete resuscitation aids) are more likely to result in accurate drug administration than those which contain less comprehensive information [6]. In addition, the effect of inadequate training of healthcare providers, who may use weight estimation and drug-dosing systems less accurately than those who are well-trained and well-practised, may also be significant [7]. The optimum resuscitation aid, at least in theory, is epitomised by a mobile phone application which could combine both weight estimation and drug dosing [8]. The use of mobile phone applications has not previously been evaluated in this context.

Current major international guidelines recommend the use of length-based tapes with pre-calculated drug doses during paediatric resuscitations [9]. There is substantial concern about the quality of evidence supporting these guidelines, however, which makes it imperative to obtain further research on the on the ability of devices such as the Broselow tape to reduce drug dose calculation errors [10].

The aim of this study was to evaluate how accurately four different weight estimation/drug dosing systems, with differing amounts of dosing information, would function during simulated paediatric resuscitation scenarios.

Methods

This was a prospective simulation study conducted in the Emergency Department of a tertiary, academic hospital in Johannesburg, South Africa. Emergency medicine registrars, emergency medicine consultants and senior advanced life support paramedics were invited to participate in the study. Permission to conduct the study was obtained from the Human Research Ethics Committee of the University of the Witwatersrand. All participants signed informed consent.

A sample size of 30 participants was required to detect a 10% difference in dosing accuracy between systems using the Fisher exact test and assuming a baseline accuracy of 60%, powered to 80% at a 0.05 significance level.

Study protocol

Eight simulation stations were used, each recreating an everyday emergency scenario with a child volunteer who simulated the medical condition specified for the scenario (see Table 1). Before commencing the simulations, each participant was fully trained in the use of the systems and materials and had sufficient time to practice with them. During the study, every participant conducted an abridged, simulated resuscitation at each station, as directed by a timekeeper. As part of the resuscitation, the participants were required to calculate two drug doses at every station and write a prescription in a format that would allow the drug to be diluted, prepared and administered by another person (e.g. hydrocortisone 100 mg/2 mL mix with 8 mL saline: give 6 mL IV). The participants used one of four weight-estimation/drug-dosing systems to estimate weight and calculate drug doses at each station, as shown in Fig. 1 and Table 1. The time taken to calculate the drug doses was recorded by the time-keeper. Upon completion of each six-minute simulation, the participants rotated to the next station until all eight were completed.

Table 1

The details of the children and scenarios used in each of the simulation stations. The method of weight estimation and drug dose guidelines are shown in the last column. EDD4C app = Emergency Drug Dosing 4 Children mobile application; EDDC book = EDDC Emergency Drug Dosing in Children book.

Station number	Characteristics of simulated patient	Clinical scenario	Drugs required	Methods
1	Age: 9 yearsWeight: 22.5 kgLength: 130 cmBMI-for-age centile: 1	Cardiac arrest	AdrenalineAmiodarone	Broselow tape
2	Age: 12 yearsWeight: 39.6 kgLength: 151 cmBMI-for-age centile: 42	Status epilepticus	LorazepamValproate	EDD4C app
3	Age: 11 yearsWeight: 34.2 kgLength: 145 cmBMI-for-age centile: 34	Respiratory distress - severe asthma	KetamineRocuronium	PAWPER XL tape and EDDC book
4	Age: 13 yearsWeight: 55.8 kgLength: 156 cmBMI-for-age centile: 90	Major trauma with abdominal injuries	Tranexamic acidMorphine	Mercy methodNone (control)
5	Age: 16 yearsWeight: 61.2 kgLength: 174 cmBMI-for-age centile: 46	Major trauma with head injury	MidazolamFentanyl	Mercy methodNone (control)
6	Age: 1 yearWeight: 11.7 kgLength: 80 cmBMI-for-age centile: 88	Severe gastroenteritis with hyperkalaemia	Calcium gluconateSoluble insulin	EDD4C app
7	Age: 8 yearsWeight: 31.2 kgLength: 132 cmBMI-for-age centile: 84	Unstable supraventricular tachycardia	PropofolFurosemide	PAWPER XL tape and EDDC book
8	Age: 7 yearsWeight: 26.4 kgLength: 131 cmBMI-for-age centile: 50	Severe pneumonia with hypoglycaemia	NaloxoneDextrose	Broselow tape

Fig. 1

A graphic representation of the study protocol. A – drug dose information from the Broselow tape is limited to the milligram dose to be administered without any instructions on preparation, dilution or administration. B – drug dose information from the Emergency Drug Dosing 4 Children app (EDD4C) is far more comprehensive, as shown in the panel. C – the drug dose information from the Emergency Drug Dosing in Children (EDDC) book which also contains comprehensive instructions on drug dilution and volume to administer. D – the control station provided no drug dosing information for the participants.

Resuscitation aid systems

The systems evaluated in this study were:

The Emergency Drug Dosing in Children (EDDC) book, which is the designated resuscitation aid of the PAWPER XL tape [11,12].

The Broselow tape 2011 edition A, with its own drug dosing information.

A custom developed mobile phone application (the EDD4C app) which provided both weight estimation as well as drug dosing information.

A control station with no drug dosing aid. The Mercy method was used to estimate weight (as a sham procedure), but the accuracy of the weight estimation was disregarded. Whatever weight was recorded by the participant for the drug dose calculations was regarded as the correct weight. The participants had to perform the drug dose calculations with no additional reference material.

Data analysis

The accuracy of weight estimation of each system was evaluated for overall accuracy. Accuracy was represented by the percentage of estimations falling within 10% of actual weight (PW10).

The accuracy of drug dose calculation was primarily assessed using the percentage error of the dose: dose errors of <10% were considered to be accurate, as previously described, while errors of >20% were considered to be critical errors [3]. The median percentage dose errors with interquartile ranges were also calculated. In order to eliminate the effect of weight estimation errors on drug dose calculations, dose errors were also calculated controlling for any weight estimation error (i.e. the result of the weight estimation obtained by each participant was assumed to be the correct weight for this aspect of the dose error analysis).

In addition, the format of the participants' written prescriptions was assessed to establish that all the instructions for preparation, dilution and delivery were present and complete.

Finally, the times taken to complete the drug dose calculations and prescription were analysed.

Outcome measures

The primary outcome measures were the accuracy of drug doses, the proportion of critical dosing errors and the correctness of the dosing prescriptions format. The secondary outcome measure was the time taken to complete drug dosing determinations. There are no commonly used criteria to establish a benchmark of acceptability with regards to these outcome measures. We therefore regarded an acceptable accuracy outcome as a system that achieved >70% accurate doses and <5% critical errors in doses.

Statistical analysis

Comparisons between accuracy outcomes were made using the Fisher-exact test, where appropriate. Comparisons between the times taken to complete the tasks were made using the Mann-Whitney test. Since multiple comparisons were made when comparing the groups for each outcome, an adjusted significance level was used, determined using a conservative model (Bonferroni correction). For this reason, a significance level of p < 0.01 was used throughout. Odds ratios were used to estimate the effect size of outcomes wherever possible.

Software

Microsoft excel (Microsoft Excel for Mac version 16.14.1) and GraphPad Prism (GraphPad Prism version 8.00 for Mac, GraphPad Software, La Jolla California USA, www.graphpad.com) were used for the data management and statistical analysis.

Results

Thirty-two participants were enrolled in the study: there were 21 emergency medicine registrars, five junior emergency medicine consultants and six senior paramedics. The participants were all reasonably experienced: the most junior participant had five years of post-qualification experience. The demographic characteristics of the participants are shown in Supplementary Table 1.

Weight estimation accuracy

The PAWPER XL tape estimated weight most accurately with a PW10 of 73.0%, followed by the EDD4C app with a PW10 of 68.1%, the Mercy method a PW10 of 57.3% (although for the drug dosing control all weights were considered to be correct) and the Broselow tape a PW10 of 47.7%.

Drug dose accuracy

The key outcomes in terms of drug dosing accuracy are shown in Fig. 2. The statistical test comparisons between the four groups are shown in Supplementary Table 2.

Fig. 2

The accuracy of drug dosing achieved with each of the three resuscitation aids and in the control group. The solid bars show the drug dosing accuracy achieved by each system when weight estimation errors were not excluded from the analysis. The partially transparent bars (also denoted by asterisk on the bar labels) represent the drug dosing accuracy achieved by each method when the effects of weight estimation errors were eliminated (the actual weight obtained was considered to be the correct weight). The green bar represents the percentage of drug doses within 10% of the correct dose. The red bars represent the percentage of drug doses with an error exceeding 20% of the correct dose.

When controlling drug dosing accuracy for weight estimation errors, there were no significant differences in the dosing accuracy or critical dose error rate in the PAWPER XL tape + EDDC book group (5.3% difference OR 0.8 (0.5, 1.4) p = 0.49 and 4.3% OR 0.4 (0.2, 1.3) p = 0.19 respectively). There was no significant difference in dosing accuracy in the Broselow tape group (9.7% difference OR 1.5 (0.9, 2.5) p = 0.16) but there was a significant difference in the critical error rate (12.6% difference OR 2.3 (1.2, 4.5) p = 0.02. The EDD4C application showed an absolute improvement in accuracy of 18.9% when controlling for weight estimation errors (OR 2.3 (1.3, 4.1) p = 0.003) and a 14.5% reduction in critical errors (OR 1.9 (1.1, 3.2) p = 0.02).

The other measures of the accuracy of the drug dose calculations are shown in Supplementary Table 3.

The results of the analysis of the formatting and completeness of the drug dose calculations are shown in Fig. 3.

Fig. 3

The accuracy of the dose and the completeness of the dosing information from each group. The correct dose was considered to be a dose within 10% of the correct dose. The correct format of the dosing information was considered to be present if the dilution, preparation and volume-to-administer had been included.

Time to obtain drug dose information

The times taken to complete the calculations to achieve a dosing answer were similar amongst the groups with the exception of the PAWPER XL tape + EDDC book group, which was significantly faster (see Table 2).

Table 2

The times taken to calculate drug doses for each of the systems. The medians with lower quartiles (LQ) and upper quartiles (UQ) are shown in addition to the proportions of calculations that took <30 s and >60 s. The outcomes of the Mann-Whitney tests (between times) and the Fisher exact tests (between proportions) are shown, with Odds ratios (OR).

Time to complete drug dose calculations
	Control	Broselow tape	EDD4C app	PAWPER XL tape + EDDC book
n	132	132	132	132
Time (s) – median (LQ, UQ)	20.5 (11.5, 30.3)a	22 (11.5, 41.5)b	19 (11.0, 32.0)c	10 (6.3, 18.8)a,b,c
Time > 60 s n (%)	24 (20.7)d	15 (11.8)	14 (11.2)	12 (10.2)d
Time < 30 s n (%)	87 (75.0)e	83 (65.4)f	88 (70.4)	104 (88.1)e, f

Mann-Whitney comparisons: ap < 0.0001; bp < 0.0001; cp < 0.0001;

Fisher exact test: dOR 2.5 (1.2, 5.2) p = 0.0187; eOR 10.8 (1.4, 85) p = 0.0101; fOR 13 (1.7, 103) p = 0.0027.

Discussion

There were four major findings of interest in this study: the differences in accuracy of weight estimation between the three systems evaluated; the relationship between the type of drug dosing guides and accuracy of the drug prescription; the observed relationship between weight estimation accuracy and the final drug dosing accuracy; and the potential pitfalls of each of the studied systems.

The PAWPER XL tape was the most accurate of the weight estimation systems in this study, as has been reported previously [8]. Both the Mercy method and the Broselow tape produced considerably lower accuracies of weight estimation. Although the Mercy method is also a dual length- and habitus-based system it was substantially less accurate than the PAWPER XL tape, a finding that has been consistent in studies performed independently of the developers of the Mercy system [13]. The accuracy of the Broselow tape was markedly lower than the PAWPER XL tape, again in keeping with recent findings, but is concerning because of the promotion of single-dimensional length-based tapes by international guideline organisations [9]. The EDD4C application was significantly more accurate than the Broselow tape, in terms of weight estimation, although the accuracy was a little lower than the 70 to 80% range that has been reported previously for dual length- and habitus-based weight estimation [11,14]. This was most likely as a result of user-difficulties with the app design, induced by the time-pressures of the simulation scenarios.

The type of dosing guide as a determinant of drug dosing accuracy

In the control group the weight was assumed to be correct for all further calculations. The dosing errors were thus independent of the weight estimation process. With no drug dosing reference material for the control group, both the accuracy of dose-determination as well as the preparation instructions were exceedingly poor, with just over 10% of doses accurate and virtually zero doses with correct preparation instructions. The critical error rate was extremely high: >40% of the doses were either more than double or less than half the correct dose. This finding provided convincing evidence that even experienced healthcare providers are unable to recall the doses of relatively commonly used emergency drugs and produce accurate prescriptions during the stress of emergency care. Previous studies have shown similar results in simulated emergencies [15]. The high frequency of critical dosing errors, in addition to very large errors, suggest that patient harm could easily arise from these significant errors. The unavoidable conclusion is that drug dose calculations during emergencies must not be performed without the use of appropriate reference material.

In this study conducted in realistic simulated paediatric emergencies, the weight estimation accuracy of the Broselow tape was poor, but on par with performances reported in previous studies (PW10 in the 40 to 60% range) [16,17]. There are two determinants of successful drug dose outcomes: the first is an accurate weight estimation. The second is a source of complete information on the drug preparation, dilution and volume to administer. The Broselow tape had a dosing accuracy of just over 50% which roughly corresponded to its weight estimation accuracy. But, with no instructions on the tape on how to prepare and administer medications, the total accuracy of prescription was <10%. Previous studies have shown the benefit of using comprehensive information sources but have not demonstrated the impact of the combined effects of weight-estimation and calculation errors that we found in this study [3,18,19]. The pre-calculated doses available on the tape were inaccurate for two reasons: the weight estimation accuracy of the tape was poor and the absence of comprehensive reference material to guide medication preparation left the participants to rely on their memory and arithmetic during the stressful resuscitation.

The calculation of drug dilutions and volumes for administration is known to be a significant source of errors and delays, which is why comprehensive resuscitation aids are considered by many experts to be essential [2,20]. The Broselow tape, and other similar tapes, should thus not be used without additional reference material [10,19]. Some international guidelines on paediatric advanced life support, which recommend the use of length-based tapes with pre-calculated drug doses if actual weight is not available, need to be reviewed in the light of these findings [9].

Although the newest version of the Broselow tape, the 2017 edition, has additional information on volumes to administer, there is still no information on preparation or dilution of medications. It is therefore still at risk of errors when drug dilutions are required.

When the EDD4C app was used the proportion of accurate drug doses was similar to that of the Broselow tape, but the prevalence of critical dose errors was substantially lower, and the proportion of correct prescriptions was much higher. The availability of comprehensive information from the application enabled this improvement in performance, but the results were not as good as might have been expected. We can speculate that user difficulties and insufficient training and practice with this system provided challenges to the users, especially with the time constraints and pressure of the simulation scenarios. Other data on the use of apps in resuscitation scenarios is limited, but in agreement with this finding [[21], [22], [23]]. This needs additional research as the use of apps is potentially one of the next big breakthroughs in enhancing patient safety in this environment, once the limitations are fully understood, as well as the training required for them to be optimally effective.

The PAWPER XL tape and the EDDC book combination was the best performer in terms of drug dosing and prescription accuracy, but it was very far from perfect. The data suggest that the system has the inherent ability to achieve a higher level of accuracy (the high individual scores for dosing and prescription accuracy), but that human factor errors were important in the suboptimum outcomes. As suggested above, it is likely that additional training and practice would be required to improve the consistency of performance of the system. There is little doubt that sufficient training is important to optimise outcomes in many aspects of paediatric resuscitation [24].

The association between weight estimation accuracy and drug dosing accuracy

It was clear that the accuracy of the final drug dose was dependent on more than just an accurate weight estimate. Although it was not possible to produce an accurate drug dose without an accurate weight estimation, an accurate weight estimation did not guarantee an accurate drug dose or final prescription. Both of these elements need to be researched further and – in a clinical setting – addressed in training, ideally under simulated resuscitation scenarios.

Times taken to complete drug dosing information

All the systems produced information that, despite differences, were clinically reasonable and acceptable. The Broselow tape, which required calculations to complete the prescriptions, was slower than the complete systems. The accuracy of dosing is a far more important consideration than speed, however. There is also no evidence about what constitutes a reasonable time in which to calculate and prepare drugs in an emergency situation. This needs further research to establish parameters to guide decision-makers and researchers in the use and development of resuscitation aids.

Potential pitfalls of the drug dosing systems

It was clear that having no dosing guide produced a catastrophic prevalence and magnitude of dosing errors. It is self-evident that appropriate dosing guides must be available for use in any prehospital or hospital-based environment where children might require resuscitation.

Although the Broselow tape does provide partial dosing information (and even more with the latest edition of the Broselow tape), it is still not complete and additional calculations are required. The lack of this information was evident in this study with the very high incidence of large dosing errors and incomplete or incorrect prescriptions.

While the PAWPER XL tape plus EDDC book and the EDD4C app were the best performers, it was evident that these systems, and apps in particular, can be difficult to use. They showed potential for delivering the best dosing assistance, but also require training so that they can be used optimally.

Individual vs team-based resuscitation implications

The findings in the present study, especially the time elements, apply primarily to a clinician working in isolation, or with a very inexperienced team. Nonetheless, although the time-pressures on a single individual in a team would be less, each of the steps performed in this study would still have to be performed by one or more team members. It seems reasonable to speculate that drug knowledge and drug dose calculation errors might not be substantially different in a team setting. Clearly if other forms of drug dosing aids (such as wall charts) were available then dosing errors might be reduced, which is another avenue for future research.

The use of other weight estimation systems

Some experts advocate the use of pre-calculated drug dosing regimes for emergency drugs based on best estimation weight calculations based on age [25]. Although age-based weight estimation might still be practised in some settings, many experts consider it to be dangerously inaccurate, vulnerable to calculation errors and no longer appropriate for clinical use, especially since other – and better – methods are available [[26], [27], [28], [29]]. These formulas are also not advocated by the leading international resuscitation organisations [9]. For that reason, the use of age-based formulas was not evaluated in this study.

Conclusions

In summary, it was clear that the most important determinant of base drug dose accuracy was an accurate weight estimation and the most significant determinant of correct drug preparation information was the availability of comprehensive information. The accuracy and critical error rate were disturbingly high in the Broselow tape and control groups, which raises significant concern for patient safety if these systems are used in clinical practice. Guidelines which advocate for the use of “length-based tapes with precalculated drug doses” need to be revised.

Weight estimation and comprehensive drug dose calculations should be practised during simulation exercises. An increased, directed training would improve the overall accuracy for all users, and could also be used to provide a degree of inoculation to resuscitation-induced stress.

Limitations

There were some technical problems with the EDD4C application with fluctuating Wi-Fi access within the hospital precincts on the day of the study. This slowed its functioning and may have skewed data against the app. This was a real-world problem, however, which provided some additional insight into the fact that electronic resources are not without problems.

During the study, the participants were offered teaching and training until they were satisfied that they knew the systems. It is clear that their perception of their abilities was not entirely accurate – this most likely resulted in a lower than potential performance of the PAWPER XL and app systems. It also emphasises the need for appropriate situation-specific training.

The following are the supplementary data related to this article.

Supplementary Table 1

Demographic information of the study participants. The medians with lower quartiles (LQ) and upper quartiles (UQ) are shown.

Supplementary Table 2

The outcomes of the Fisher exact tests between each of the dosing systems with regards to the proportions of doses with <10% error (clear section of top panel) and critical dosing errors (shaded section of top panel). The differences in accuracy of the format of the dose prescriptions (clear section of the bottom panel) and the combined accuracy of the dose and format (shaded section of the bottom panel) are shown.

Supplementary Table 3

Median drug dosing errors and number of large dosing errors for each group.

Dissemination of results

These findings have been shared with local units in South Africa.

Author contributions

Authors contributed as follows to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: MW 70% and LG 30%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Conflicts of interest

Profs Wells and Goldstein are editors of the African Journal of Emergency Medicine. Profs Wells and Goldstein were not involved in the editorial workflow for this manuscript. The African Journal of Emergency Medicine applies a double blinded process for all manuscript peer reviews. Prof Wells developed the PAWPER tape systems but derives no financial benefit from them. The authors declared no further conflict of interest.