A pre-postintervention study to evaluate the impact of dose calculators on the accuracy of gentamicin and vancomycin initial doses.

OBJECTIVES: Gentamicin and vancomycin are narrow-therapeutic-index antibiotics with potential for high toxicity requiring dose individualisation and continuous monitoring. Clinical decision support (CDS) tools have been effective in reducing gentamicin and vancomycin dosing errors. Online dose calculators for these drugs were implemented in a London National Health Service hospital. This study aimed to evaluate the impact of these calculators on the accuracy of gentamicin and vancomycin initial doses.
METHODS: The study used a pre-postintervention design. Data were collected using electronic patient records and paper notes. Random samples of gentamicin and vancomycin initial doses administered during the 8 months before implementation of the calculators were assessed retrospectively against hospital guidelines. Following implementation of the calculators, doses were assessed prospectively. Any gentamicin dose not within ± 10% and any vancomycin dose not within ± 20% of the guideline-recommended dose were considered incorrect.
RESULTS: The intranet calculator pages were visited 721 times (gentamicin=333; vancomycin=388) during the 2-month period following the calculators' implementation. Gentamicin dose errors fell from 61.5% (120/195) to 44.2% (95/215), p<0.001. Incorrect vancomycin loading doses fell from 58.1% (90/155) to 32.4% (46/142), p<0.001. Incorrect vancomycin first maintenance doses fell from 55.5% (86/155) to 33.1% (47/142), p<0.001. Loading and first maintenance vancomycin doses were both incorrect in 37.4% (58/155) of patients before and 13.4% (19/142) after calculator implementation, p<0.001.
CONCLUSIONS: This study suggests that gentamicin and vancomycin dose calculators significantly improved the prescribing of initial doses of these agents. Therefore, healthcare organisations should consider using such CDS tools to support the prescribing of these high-risk drugs. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This study suggests that accuracy of gentamicin and vancomycin initial doses can be improved using dose calculators linked to electronic prescribing.

Dose improvements were demonstrated in multiple specialities within a large acute teaching hospital.

Unlike previous studies, this study evaluated the accuracy of loading and first maintenance doses of vancomycin.

This study did not assess the clinical and/or pharmacodynamic outcomes of the calculators.

It was not possible to identify whether individual doctors who accessed the calculators used the tool for calculating drug doses prior to prescribing.

The use of different methods to identify patients precalculators and postcalculators may have led to differences in patient demographics.

The long-term impact of the calculators (eg, 12 months postimplementation) was not assessed in this study.

Background

Gentamicin and vancomycin are narrow-therapeutic-index antibiotics with potential for high toxicity and require dose individualisation with regular therapeutic drug monitoring. The main side effects of gentamicin are dose-related, and include nephrotoxicity and irreversible ototoxicity, while the main side effects of parenteral vancomycin, not necessarily dose-related, include nephrotoxicity, normally-reversible ototoxicity and blood disorders including neutropenia.1

Antibiotics were the most common medications associated with prescribing errors (39.7%, 276/696) in a US study. Overdosing and underdosing accounted for 58.3% of these errors (406/696).2 Gentamicin and vancomycin are often poorly prescribed, and dosing errors are a particular problem. Underdosing may lead to treatment failure while overdosing can cause toxicity. An Australian study showed that only 30.3% (40/132) of all gentamicin initial doses were in accordance with hospital guidelines.3 Another Australian study found that of 60 eligible gentamicin initial doses, only 46.7% (n=28) were consistent with local guidelines.4 A US study conducted in a tertiary-care hospital identified that only 50.6% (128/253) of vancomycin initial doses were appropriate according to national guidelines.5 Fuller et al6 found that only 22.1% (980/4441) of all vancomycin doses prescribed at a US emergency department (ED) were correct as per national guidelines.

Narrow-therapeutic-index medications are more likely to cause adverse drug events (ADEs) and may be more prone to medication errors. Clinical decision support (CDS) may therefore be of particular value for these high-risk medications.7 A number of CDS tools have been shown to be effective in reducing gentamicin/vancomycin dosing errors. One of these tools, ‘Pharmacist-to-Dose’, a computerised request sent by the prescriber to the pharmacist for dosing guidance on vancomycin and aminoglycosides, was evaluated by Vincent et al. This tool significantly reduced medication errors with these drugs (31.6%, 18/57 vs 7.0%, 5/71, p=0.002).7 Another CDS tool, GFR+, automatically calculates and updates doses of key drugs based on renal function. When evaluated by Roberts et al,8 this tool improved dosing conformity for both gentamicin (63%, 46/73 vs 87%, 33/38, p=0.01) and vancomycin (47%, 16/34 vs 77%, 13/17, p=0.07).

Other CDS tools were not effective in reducing gentamicin/vancomycin dosing errors. Some were even potentially harmful. One CDS tool was a computerised prescriber order entry (CPOE) system that displayed an initial default dose for gentamicin and tobramycin in the dose box on the electronic prescription. In a large proportion (58%, 227/392) of prescriptions, the suggested default dose had not been amended. This dose was used in 51.6% (132/256) of prescriptions for renal-insufficiency patients and 85.6% (113/132) of these resulted in potential ADEs compared with 53.2% (66/124) for the remaining prescriptions (p<0.0001). For patients without renal impairment, use of the CDS tool to determine doses was not significantly better than regular dosing. The dose was incorrect in 72.7% (165/227) of CDS prescriptions compared to 77% (127/165) in non-CDS prescriptions (p=0.4). Therefore, this tool was a source of potential ADEs especially in renal-insufficiency patients.9

Risks with gentamicin and vancomycin were identified locally in a number of ways. An analysis of antibiotic-associated medication incidents reported to the centralised incident reporting systems of two large UK teaching hospitals, including the study site, over 2 years, was undertaken.10 One-third of dose/frequency errors reported (32/96) at the study site were related to gentamicin (n=16) and vancomycin (n=16). A local Failure Modes and Effects Analysis for gentamicin identified that risks with dose calculation and prescribing, especially in patients with renal impairment or obesity, were greater than risks with preparation of infusions. The implementation of electronic prescribing across the hospital facilitated the development of computerised dose calculators for these drugs to improve prescribing, championed by the Lead Antimicrobial Pharmacist with the support of the Antibiotic Usage Steering Group.

The aim of this study was to evaluate the impact of these calculators on the accuracy of prescribing of gentamicin and vancomycin initial doses.

Methods

Study design

The study used a pre–postintervention design and was undertaken at a 950-bed acute National Health Service (NHS) teaching hospital in London. Prescriptions for gentamicin initial dose and vancomycin initiation regimen (loading dose and first maintenance dose) written during the 8-month period (1 January 2011–31 August 2011) before implementation of the dose calculators were retrospectively assessed for appropriateness. Following implementation of gentamicin and vancomycin dose calculators, and promotion throughout the hospital, data on the appropriateness of prescriptions for their initial doses were collected prospectively for a 2-month period (1 June 2013–2 August 2013). In addition to loading doses, first maintenance doses were assessed as part of vancomycin initiation regimens. This was considered important, as the first maintenance dose is essential in ensuring appropriate serum levels, and that is reflected in that the new calculator provides both loading and first maintenance doses.

The study was categorised by the hospital Research and Development Department as a clinical audit/service evaluation. Thus, ethical approval was not required. However, the study was registered with the Clinical Effectiveness and Audit Department.

Inclusion/exclusion criteria

All adults without severe renal impairment (creatinine clearance (CrCl) ≥20 mL/min and not receiving haemo/peritoneal dialysis) who received parenteral gentamicin or vancomycin (loading and at least one maintenance dose) were included in this study. Patients who received gentamicin as antibiotic prophylaxis prior to urinary catheterisation, and patients receiving gentamicin as part of a two times or three times/day endocarditis treatment regimen were excluded. Patients who received vancomycin as an oral dose, once-only intravenous dose, or continuous infusion were also excluded. Patients without recorded height and weight were excluded, as these criteria are essential for calculating ideal body weight (BW) required for dose calculation; also excluded were those with a height less than 152.4 cm (5 ft).

Definitions

Incorrect doses of gentamicin and vancomycin have been described in the literature as doses outside the ranges of 10%9 and 33.3%.8 For the purposes of this study, an incorrect dose of gentamicin was defined as ‘any dose more than 10% higher or lower than the recommended dose as specified in the hospital guidelines’. An incorrect dose of vancomycin was defined as ‘any dose more than 20% higher or lower than the recommended dose as specified in the hospital guidelines’. These margins were chosen taking into account that gentamicin and vancomycin have a narrow therapeutic index and that many patients receiving these drugs have some element of renal-insufficiency. They also consider that doses are usually rounded based on the available dosage forms and the practicalities of dose administration.

Sample size calculation

In order to perform power and sample size calculations, data were collected retrospectively for 49 patients who received (both prescribed and administered) gentamicin and for 36 patients who received vancomycin. A power analysis conducted using assumptions of 95% power identified that at each stage preintervention and postintervention, a minimum sample of 192 patients who received gentamicin and a minimum sample of 141 patients who received vancomycin were required to detect a 30% reduction in error incidence with each drug.

Study intervention

Calculators for gentamicin and vancomycin doses were implemented in the hospital as an Excel application on the hospital intranet. This is accessed by users from a menu within the Electronic Prescribing and Medicines Administration system (EPMA) of the Electronic Patient Records (EPR) iSOFT Clinical Manager, which gives access to patient demographics, hospital visit histories, clinical notes, laboratory results and drug prescribing and administration records. Prescribers are directed to the calculators through a written note within the electronic prescription for each drug. The calculators are not automatically populated from demographic or laboratory data entered elsewhere within EPR. The calculator uses patient information manually entered by the prescriber, including age, actual BW, height and serum creatinine (SrCr) to provide the appropriate dose and frequency for each patient based on their weight and CrCl. These calculators can be used with either metric or imperial units, and are gender-specific. Once required data are entered, the calculator will automatically determine the appropriate weight to be used (actual, ideal or adjusted), CrCl, recommended dose (loading and first maintenance for vancomycin), recommended dosing interval and duration of vancomycin infusion (to avoid possible Red man syndrome). The dose given by the calculator is the dose recommended by the hospital antimicrobial guidelines.11

The hospital antimicrobial guidelines include instructions for dosing gentamicin and vancomycin. Ideal BW should be used for dosing and CrCl calculation, unless the patient is underweight (below ideal BW) when actual BW is used or overweight (actual BW >20% over ideal BW) when adjusted BW is used. Estimated BW can only be used when other options are not available. The initial gentamicin dose is calculated depending on patient's appropriate weight and CrCl, which is calculated using the Cockcroft-Gault equation.12 Although there are more accurate methods (eg, insulin clearance13), it is the method recommended by the hospital guidelines for routine measurement of renal function. The loading dose of vancomycin is calculated based on actual BW. If the patient is ‘unfit’ to be weighed, ideal BW is used, unless the patient looks underweight, in which case estimated BW is used. Vancomycin first maintenance dose is calculated based on the patient's appropriate weight and CrCl. Details of equations and calculations used in the calculators are in table 1.

Table 1

Equations/calculations used in gentamicin and vancomycin dose calculators

Variable	Underlying equation/calculation
Ideal body weight (kg)14	▸ With height in feet/inches=[males 50 kg, females 45.5 kg]+2.3 kg for every inch in height over 5 ft▸ With height in centimetres=[male 50 kg, female 45.5 kg]+[(2.3×height in cm above 152.4)/2.54]
Adjusted body weight (kg)15*	= ideal body weight (kg)+0.4 [(actual body weight (kg)−ideal body weight (kg)]
Creatinine clearance (mL/min)12
	CrCl (mL/min)	Dose and dosing interval
Gentamicin initial dose11	>80	5.0 mg/kg every 24 h
	60–80	4.0 mg/kg every 24 h
	40–60	3.5 mg/kg every 24 h
	30–40	2.5 mg/kg every 24 h
	20–30	4.0 mg/kg every 48 h
	Actual body weight (kg)	<60	60–90	>90
Vancomycin loading dose11 16	Loading dose	1000 mg	1500 mg	2000 mg
	Fluid (NaCl or glucose)	250 mL	500 mL	500 mL
	Infusion period	120 min	180 min	240 min
	Pump rate	125 mL/h	167 mL/h	125 mL/h
	CrCl (mL/min)	Dose (mg)	Dosing interval (h)	Fluid (mL)	Infusion period (min)	Pump rate (mL/h)
Vancomycin maintenance dose11 16	>110	1500	12	500	180	167
	90–110	1250	12	250	150	100
	75–89	1000	12	250	120	125
	55–74	750	12	250	90	167
	40–54	500	12	250	60	250
	30–39	750	24	250	90	167
	20–29	500	24	250	60	250

*Also called dose-determining weight.

Calculator implementation

On 1 September 2011, the calculators were first available on the hospital intranet. The calculators were promoted locally by pharmacists in specific clinical areas, particularly in haematology/oncology, where gentamicin is widely prescribed for the treatment of neutropenic sepsis. In February 2013, a link was created between EPMA and the relevant page on the intranet, enabling prescribers to access the required calculator on the same screen as the EPR. Junior doctors joining the hospital in February 2013 were informed about the calculators during their induction programmes for safe prescribing.

In May 2013, a non-mandatory instruction to use the calculator was added to gentamicin and vancomycin orders on EPMA, and the availability of the calculators was advertised on the intranet news page and via an email to all doctors. Moreover, information about the calculators and how to use them, as well as the precalculator results, were presented at one of the hospital Grand Rounds to encourage their use.

Data collection and processing

Data were collected using EPR (iSOFT Clinical Manager 1.4). Paper notes were also reviewed, if necessary, to collect data not available within EPR (eg, missing weight or height). A data collection form was developed, piloted and optimised before formal data collection started. Demographic data collected for each patient included age, gender, weight, height, SrCr, clinical specialty and ward. Patients’ body mass index (BMI) was also calculated. Details collected about antibiotic therapy included the first dose for gentamicin, and the loading and first maintenance doses for vancomycin. Dose frequency was also documented. Doses were compared with the hospital antibiotic guidelines valid at the time of prescribing. If patient height was not recorded on electronic or paper notes, patients were asked about their height. If patients were able to self-report their height, this height was used in the study. Estimated weights and heights were used if they were recorded in patient notes (eg, for critical-care patients).

In the precalculator phase of the study, patients who received gentamicin or vancomycin were identified retrospectively from the Microbiology database of patients for whom a gentamicin or vancomycin serum level had been requested. A random sample of gentamicin initial doses and a random sample of vancomycin initiation regimens that were prescribed during the 8 months prior to implementing the calculators were assessed against the guidelines for accuracy. In the postcalculator phase, patients were identified prospectively through an electronic filter of all active antimicrobial prescriptions. This filter was checked daily, and any patient who was prescribed gentamicin and/or vancomycin was reviewed on EPMA (and paper notes if necessary) for eligibility. During the 2 months following promotion of the calculators, the accuracy of gentamicin initial doses and vancomycin initiation regimens were assessed for all eligible patients (figure 1).

Figure 1

Study participant flow chart.

Data were transferred into SPSS, which was used for analysis. Quality-assurance procedures were undertaken to assure the quality and accuracy of the data transferred. The correct dose was calculated for each patient based on hospital guidelines. Then, the difference between the prescribed dose and the calculated dose was determined. Based on its deviation from the guideline-recommended dose, each dose was categorised as an underdose (>10% under for gentamicin and >20% under for vancomycin), overdose (>10% over for gentamicin and >20% over for vancomycin), or correct dose (≤10%± for gentamicin and ≤20%± for vancomycin). Data on number of times the intranet calculator pages were visited were provided by the hospital senior Web Developer.

Data analysis

Statistical data analysis was performed using SPSS V.21. Binary logistic regression was used to assess the significance of the difference between the number of correct doses, overdoses and underdoses, before and after implementation of the calculators. It was also used to produce ORs for these results. χ2 Analysis and Fisher's exact test were used to assess the significance of the difference between the patients’ gender, ethnicity and prescriber specialty, before and after the calculators’ implementation. The same tests were also used to measure the significance of the difference between the number of correct doses among different patient groups before and after the calculators. Two-sample t test was used to evaluate the difference in the age, BMI and CrCl of patients, before and after the calculators. The level of significance was chosen as 5%.

Results

In total, 707 patients were included in the study; 410 received gentamicin (195 before and 215 after the calculator implementation) and 297 received vancomycin (155 before and 142 after the calculator implementation). While Haematology/Oncology were overall the most frequent prescribers of gentamicin (37.6%, n=154), Medicine was the specialty in which vancomycin was most frequently prescribed (30.3%, n=90). Further details in table 2.

Table 2

Demographic data for included patients

Category	Gentamicin before (%)n=195	Gentamicin after (%)n=215	p Value	Vancomycin before (%)n=155	Vancomycin after (%)n=142	p Value
Gender
Male	117 (60.0)	97 (45.1)	0.003	87 (56.1)	81 (57.0)	0.907
Female	78 (40.0)	118 (54.9)		68 (43.9)	61 (43.0)
Age±SD (years)	55.4±17.3	57.6±18.5	0.211	60.5±17.3	57.7±16.6	0.159
BMI±SD (kg/m2)	25.8±6.0	26.2±6.0	0.546	26.4±45.9	27.9±6.5	0.030
CrCl±SD (mL/min)	97.3±51.7	90.6±48.2	0.176	84.6±42.4	100.7±43.0	0.001
Ethnicity
White	133 (68.2)	144 (67.0)	0.833	100 (64.5)	106 (74.6)	0.060
Black	37 (19.0)	49 (22.8)	0.396	30 (19.4)	28 (19.7)	1.000
Asian	10 (5.1)	13 (6.0)	0.831	6 (3.9)	5 (3.5)	1.000
Other	7 (3.6)	4 (1.9)	0.364	8 (5.2)	2 (1.4)	0.107
Not specified	8 (4.1)	5 (2.3)	0.400	11 (7.1)	1 (0.7)	0.006
Specialty
Haematology/oncology	99 (50.8)	55 (25.6)	<0.001	39 (25.2)	16 (11.3)	0.003
Critical care	45 (23.1)	24 (11.2)	0.001	10 (6.5)	3 (2.1)	0.089
Medicine	16 (8.2)	65 (30.2)	<0.001	44 (28.4)	46 (32.4)	0.528
Surgery	14 (7.2)	27 (12.6)	0.073	22 (14.2)	27 (19.0)	0.277
Liver	8 (4.1)	7 (3.3)	0.794	15 (9.7)	13 (9.2)	1.000
Women/child	6 (3.1)	18 (8.4)	0.033	1 (0.6)	2 (1.4)	0.608
Renal	3 (1.5)	1 (0.5)	0.350	0 (0)	0 (0)	–
Neuroscience	3 (1.5)	13 (6)	0.021	14 (9.0)	25 (17.6)	0.038
Cardiology	1 (0.5)	5 (2.3)	0.219	10 (6.5)	10 (7.0)	1.000

BMI, body mass index; CrCl, creatinine clearance.

The gentamicin calculator page on the hospital intranet was visited 333 times during the 2-month period following implementation of the calculator. Correct gentamicin doses increased from 38.5% (75/195) before to 55.8% (120/215) after the calculator implementation, OR=2.02, p<0.001 (table 3).

Table 3

Analysis of the accuracy of gentamicin initial doses

Category	Before (%)n=195	After (%)n=215	OR	95% CI	p Value
Correct dose	75 (38.5)	120 (55.8)	2.02	1.36 to 3.00	<0.001
Overdose	83 (42.6)	62 (28.8)	0.55	0.36 to 0.82	0.004
Underdose	37 (19.0)	33 (15.3)	0.77	0.46 to 1.30	0.331

The vancomycin calculator page on the hospital intranet was visited 388 times in the post calculator data collection period. Correct vancomycin loading doses increased from 41.9% (65/155) before to 67.6% (96/142) after the calculator implementation, OR=2.89, p<0.001. Correct vancomycin first maintenance doses increased from 44.5% (69/155) to 66.9% (95/142), OR=2.52, p<0.001. The whole vancomycin initiation regimen was correct (both loading and first maintenance doses were correct) in 23.9% (37/155) of patients before and 47.9% (68/142) after the calculator implementation, OR=2.93, p<0.001 (table 4).

Table 4

Analysis of the accuracy of vancomycin initial doses

Category	Before (%)n=155	After (%)n=142	OR	95% CI	p Value
Loading dose
Correct dose	65 (41.9)	96 (67.6)	2.89	1.80 to 4.65	<0.001
Overdose	5 (3.2)	5 (3.5)	1.10	0.31 to 3.86	0.888
Underdose	85 (54.8)	41 (28.9)	0.33	0.21 to 0.54	<0.001
First maintenance dose
Correct dose	69 (44.5)	95 (66.9)	2.52	1.57 to 4.04	<0.001
Overdose	56 (36.1)	32 (22.5)	0.51	0.31 to 0.86	0.010
Underdose	30 (19.4)	15 (10.6)	0.49	0.25 to 0.96	0.037
Initiation regimen (both loading and first maintenance doses)
Correct	37 (23.9)	68 (47.9)	2.93	1.79 to 4.81	<0.001
Incorrect	118 (76.1)	74 (52.1)	0.34	0.21 to 0.56	<0.001

The vancomycin initiation regimen was considered incorrect in three scenarios: loading dose is correct and first maintenance dose is incorrect: 18.1% before (28/155) and 19.7% after (28/142) implementation of the calculator; loading dose is incorrect and first maintenance dose is correct: 20.6% before (32/155) and 19% after (27/142) calculator implementation; or loading dose and first maintenance dose are both incorrect: 37.4% before (58/155) and 13.4% after (19/142) calculator implementation (table 5).

Table 5

Subanalysis of the accuracy of vancomycin initiation regimens

Category	Before (%)n=155	After (%)n=142	OR	95% CI	p Value
LD correct and MD correct	37 (23.9)	68 (47.9)	2.93	1.79 to 4.81	<0.001
LD correct and MD incorrect	28 (18.1)	28 (19.7)	1.11	0.62 to 1.99	0.716
LD incorrect and MD correct	32 (20.6)	27 (19.0)	0.90	0.51 to 1.60	0.725
LD incorrect and MD incorrect	58 (37.4)	19 (13.4)	0.26	0.14 to 0.46	<0.001

LD, loading dose; MD, first maintenance dose.

A subanalysis was undertaken for patients at higher risk including elderly (≥65 years), renally-impaired (CrCl<60 mL/min) and obese (BMI≥30 kg/m2). No significant differences in the rate of gentamicin correct doses before and after the calculator implementation were found between elderly (14, 31.1% and 31, 68.9%) and non-elderly (61, 40.7% and 89, 59.3%) patients (p=0.248), patients with (13, 35.1% and 24, 64.9%) and without (62, 39.2% and 96, 60.8%) renal impairment (p=0.644), or obese (17, 43.6% and 22, 56.4%) and non-obese (58, 37.2% and 98, 62.8%) patients (p=0.462). There were also no significant differences in the rate of vancomycin correct regimens before and after the calculator between elderly (9, 38.9% and 24, 61.1%) and non-elderly (28, 27.3% and 44, 72.7%) patients (p=0.247), patients with (6, 46.2% and 7, 53.8%) and without (31, 33.7% and 61, 66.3%) renal impairment (p=0.379), or obese (5, 20% and 20, 80%) and non-obese (32, 40% and 48, 60%) patients (p=0.068).

Discussion

The initial dose accuracy for both gentamicin and vancomycin significantly improved after implementation of new calculators providing CDS to prescribers at an NHS teaching hospital. Incorrect initial doses of gentamicin were reduced from 61.5% to 44.2% (p<0.001). Incorrect loading doses of vancomycin were reduced from 58.1% to 32.4% (p<0.001), while incorrect first maintenance doses were reduced from 55.5% to 33.1% (p<0.001). Incorrect vancomycin initiation regimens were reduced from 76.1% to 52.1% (p<0.001). The calculator pages on the hospital intranet were frequently accessed in the postcalculator phase. The activities undertaken to promote the calculators and make them compatible with the electronic prescribing system are anticipated to have a role in such access.

Different CDS tools have shown improvements in gentamicin and vancomycin dosing. A previous UK study found that introducing an online gentamicin dose calculator led to a 300% improvement in dosing accuracy (30%, 15/50 precalculator to 92%, 46/50 postcalculator) at an NHS hospital. The calculator in this study required the same data as the current study (ie, gender, age, weight, height, SrCr) to provide the correct dose. Prophylactic doses were also excluded from the study.17 However, it was conducted only on surgical wards, had a relatively small sample size (only 50 patients in each phase) and no statistical analysis was conducted. Another study conducted in Taiwan investigated the impact of an online gentamicin dose calculator incorporated into the hospital CPOE system, using a pharmacokinetic formula to provide the correct dose. The main outcome measure was the number of gentamicin orders that resulted in undesirable serum levels. The calculator reduced undesirable levels from 32.7% (152/465) to 13.5% (40/297).18 Nevertheless, the study was conducted only in intensive care units and used a compound pharmacokinetic equation, which might be more suitable only for specific patient groups. In addition, no statistical analysis was performed.

A US study by Frankel et al19 showed an improvement in initial vancomycin dosing at the ED after the introduction of CPOE with vancomycin weight-based orders. The CPOE doses showed 51.1% (120/235) compliance to the recommended initial dosing guideline compared with 34.9% in the pre-CPOE doses (82/235, p<0.001). Compared to the vancomycin calculator in the current study, this CPOE was only valid for loading doses since it did not adjust doses for height or CrCl, and it was examined only in the ED. The ‘Pharmacist-to-Dose’ intervention in the Vincent et al7 study showed a significant reduction in aminoglycoside and vancomycin dosing errors. However, the study had a relatively small sample size (49 patients preintervention and 48 postintervention) and lacked details on how many correct and incorrect doses were associated with each drug (only combined data were provided). The intervention by Roberts et al8 (GFR+) also reduced dosing errors with gentamicin and vancomycin. However, the reduction in vancomycin errors was not statistically significant (p=0.07). The number of patients in each group was relatively small (73 pre-GFR+ and 38 post-GFR+ for gentamicin and 34 pre-GFR+ and 17 post-GFR+ for vancomycin). Moreover, the study used a modified version of Cockcroft-Gault equation that caps SrCr to a minimum of 60 μmol/L and also a complex gentamicin dosing model that was derived from local population kinetics, which limits the generalisability of its findings.

The vast majority of incorrect vancomycin loading doses in the current study were underdoses (94.4%, 85/90). Similarly, 90.8% (3143/3461) of incorrect vancomycin doses in the Fuller et al6 study were underdoses. Although actual BW is the ideal method for calculating initial vancomycin doses, a fixed dose of 1000 mg is commonly used in the ED.20 Most patients in a US study conducted in the ED (87.5%, 210/240) were administered an initial vancomycin dose of 1000 mg.20 In the current study, 1000 mg dose was used in 40.4% of all loading doses (120/297), 75.8% (91/120) of them were underdoses, which shows that this practice is not common only at the ED. The use of 1000 mg loading doses was significantly reduced from 52.3% (81/155) before to 27.5% (39/142) after implementation of the calculator (p<0.001). However, the extent of underdosing here was relatively unchanged from 79% before to 69.2% after the intervention. Likewise, the study by Frankel et al19 identified that initial doses of 1000 mg vancomycin were prescribed in 206/235 (87.7%) of cases and 72.3% of these were underdoses. Following implementation of CPOE, that rate of prescribing of 1000 mg doses fell to 128/235 (54.5%), of which 59.4% were underdoses. The current study was the first to assess first maintenance dose as part of the initial dose of vancomycin in addition to loading dose, so no comparison with past studies is possible.

The introduction of CDS tools into healthcare is a complex interaction between people, technology and organisational workflow. The automatic provision of CDS tools as part of the clinician workflow is seen as the most important element for successful CDS implementation.8 In this study, efforts were made to incorporate the use of calculators into the prescriber workflow by adding a direct link to them from EPMA and setting an instruction to use them on the electronic prescription forms. However, further work is required to fully integrate these calculators into the electronic prescribing system (eg, automatic population of appropriate patient demographic data and laboratory results). The reported advantages of CDS tools in optimising drug dosing may cause physicians to over-rely on their suggestions. However, prescribers should always be careful when using CDS tools, particularly for prescribing high-risk and narrow-therapeutic-index medications, as studies have highlighted that these tools may lead to unintended negative results.9 21 22

One limitation of this study is that it was not possible, due to technical barriers, to identify whether visitors to the calculator pages actually used the tools to aid prescribing. Therefore, the improvements in gentamicin and vancomycin initial-dose accuracy cannot be definitively linked to the use of the calculators, and so further work should be undertaken to link visits to these calculators to actual patient dosing. However, the significant improvements in dose accuracy after implementation of the calculators provide an indication that they contributed to this improvement, especially as no other proactive initiatives to improve gentamicin and vancomycin dosing took place during the same period. In addition, gentamicin and vancomycin dosing guidance was the same preintervention and postintervention. Prospective data collection would have been the ideal method to collect data in the precalculator and postcalculator phases. As this was not feasible, the use of a retrospective method to collect data and a different source to identify patients (ie, Microbiology database) in the precalculator phase was considered a limitation of this study. Selection of the precalculator study population differed from that associated with the postcalculator intervention group. The population in the precalculator phase was retrospectively identified from records of patients who had been prescribed gentamicin or vancomycin during an 8-month period and for whom a serum level had been requested. However, the postcalculator group was identified prospectively over a 2-month period using an antimicrobial filter within EPR to identify all active prescriptions for gentamicin and vancomycin, regardless of whether a serum level had been requested. The differences in these methodologies may have resulted in different patient and prescriber groups, which may have had an impact on the findings of this study. However, this impact is likely minimal. The guidelines for gentamicin and vancomycin prescribing applied across all patient groups included in the study. Where local specialty-specific guidelines for gentamicin and vancomycin were in place (eg, continuous vancomycin infusions in critical care), these prescriptions and patients were excluded from the study. The precalculator data had to be collected retrospectively due to the unavailability of EPMA on all hospital wards at that time. Moreover, the calculators were already available online, although not promoted or linked with EPMA, when the decision to conduct the study was taken. Thus, the retrospective-review dates (1 January 2011–31 August 2011) were chosen before the date calculators were first available online. The Microbiology database was the only source available to retrospectively identify patients prescribed gentamicin and vancomycin.

There were some differences in demographics precalculators and postcalculators, including patient gender in the gentamicin group, BMI and CrCl in the vancomycin group, and prescriber specialty in both groups. These differences are probably due to the different data collection methods used, but it is not anticipated that they affected the overall results. The comparison between high-risk and non-high-risk patients showed that age, weight and renal function did not affect the dosing accuracy. In addition, the differences in specialty rates are unlikely to have affected the overall outcomes because the number of correct doses in the specialty with the highest number of patients (Haematology/Oncology in the gentamicin group and Medicine in the Vancomycin group) was higher postcalculator, although the overall proportion of doses was higher or similar precalculator.

Data from the literature show that giving the correct dose of gentamicin or vancomycin improves clinical outcomes. A Scottish study showed that cure rate was higher in patients for whom the gentamicin dose was given according to protocol (95.7%, 22/23) compared to those for whom it was not (75%, 24/32), p=0.06. Less toxicity was observed in patients for whom the doses were adjusted according to protocol (4.3%, 1/23 vs 28.1%, 9/32, p<0.05).23 Fuller et al6 demonstrated that patients who received a vancomycin overdose (>20 mg/kg) stayed in hospital longer (p=0.005), were more likely to spend ≥3 days in hospital (OR=1.49, p=0.006) and more likely to die (OR=1.88, p=0.004). They also showed that correct doses were associated with significantly higher numbers of therapeutic serum levels (21.6% vs 14.3%, p=0.004). However, this study did not assess the independency of the association between these outcomes and vancomycin overdoses. With the exception of one study, which showed a reduction in undesirable serum levels with the intervention,18 none of the intervention studies discussed above have assessed patient clinical outcomes. Since this study focused on identifying the accuracy of prescribing gentamicin and vancomycin initial doses according to an evidence-based guideline, it did not assess the difference in patient clinical outcomes between precalculator and postcalculator doses (eg, correct serum level, treatment-success rate). The aim of treatment with these drugs is to administer safe initial doses to ensure prompt, effective treatment of potentially life-threatening infections while minimising the risk of toxicity. Ensuring first doses are accurately calculated is the first stage of safe, effective treatment. The assessment of serum levels after subsequent doses to exclude toxicity, and resolution of infection, are subject to many patient and process variables and were beyond the scope of this study. However, further work would be needed to assess the direct impact of this intervention on patient clinical outcomes. As the postcalculator phase was conducted directly after completing the calculators’ implementation, the dosing accuracy and calculators’ usage should be reassessed after 12–24 months to evaluate the long-term impact of the calculators.

Conclusion

This study suggests that gentamicin and vancomycin dose calculators significantly improved the prescribing of initial doses of these agents. Healthcare organisations implementing electronic prescribing systems should consider including such CDS tools in their programmes to support the prescribing of these high-risk drugs. However, this study did not assess the long-term impact of the calculators and their clinical outcomes. Therefore, further work is needed to evaluate the prolonged effect of these calculators, and to determine the association between their use and improvement of clinical outcomes.