Missed medication doses in hospitalised patients: a descriptive account of quality improvement measures and time series analysis.

OBJECTIVE: To investigate the changes in overdue doses rates over a 4-year period in an National Health Service (NHS) teaching hospital, following the implementation of interventions associated with an electronic prescribing system used within the hospital.
DESIGN: Retrospective time-series analysis of weekly dose administration data.
SETTING: University teaching hospital using a locally developed electronic prescribing and administration system (Prescribing, Information and Communication System or PICS) with an audit database containing details on every drug prescription and dose administration. PARTICIPANTS: Prescription data extracted from the PICS database. INTERVENTION(S): Four interventions were implemented in the Trust: (i) the ability for doctors to pause medication doses; (ii) clinical dashboards; (iii) visual indicators for overdue doses and (iv) overdue doses Root Cause ANALYSIS: (RCA) meetings and a National Patient Safety Agency (NPSA) Rapid Response Alert. Main outcome measure(s) The percentage of missed medication doses.
RESULTS: Rates of both missed antibiotic and non-antibiotic doses decreased significantly upon the introduction of clinical dashboards (reductions of 0.60 and 0.41 percentage points, respectively), as well as following the instigation of executive-led overdue doses RCA meetings (reductions of 0.83 and 0.97 percentage points, respectively) and the publication of an associated NPSA Rapid Response Alert. Implementing a visual indicator for overdue doses was not associated with significant decreases in the rates of missed antibiotic or non-antibiotic doses.
CONCLUSIONS: Electronic prescribing systems can facilitate data collection relating to missed medication doses.
INTERVENTIONS: providing hospital staff with information about overdue doses at a ward level can help promote reductions in overdue doses rates.

Introduction

The prescribing of medicines is one of the most common healthcare acts in acute hospital care. When a medicine is prescribed there is usually the understanding that the patient will have the medicine administered according to the prescription schedule. Failure to do so can lead to, or has potential to lead to, patient harm, which constitutes a medication error [1]. Overdue doses, that is, medication doses that are prescribed but not administered, are one form of medication error. Attention was raised around the issue of medication errors due to overdue doses through a Rapid Response alert by the UK National Patient Safety Agency (NPSA) [2]. One of the drivers to produce this guidance was the fact that missed and delayed medicines was the second largest cause of medication incidents reported to the UK National Reporting and Learning System in the year 2007 [3]. In a recent review it was found that 15.6% of all 5 437 999 medication incident reports across the National Health Service (NHS) from 2005 to 2010 were due to missed or delayed medication [4]. This is a figure that the NHS is aiming to reduce, particularly following the introduction of the Commissioning for Quality and Innovation agenda in 2009 [5], which has included medicine management issues in many local and regional targets.

Missed dose errors have been associated with adverse events in hospitalized patients for many decades [6] and may take several forms: omission of medicines on admission or discharge, omission of details about the drug (e.g. formulation or dose frequency) or omission of treatment when indicated—all of which can be considered prescribing errors. However, here we are concerned with medication administration errors (MAEs). Specifically, the failure to physically administer the drugs when scheduled for patients, which has previously received less attention in the literature compared with other MAEs, despite evidence that up to 50% of medical incidents are made during the medication administration stage [4]. A variety of reasons are associated with overdue doses administrations [7], but clinically inappropriate omissions are a key concern.

Varying rates of MAEs have been reported in studies. Ridge et al. [8] examined the nature and rate of drug administration errors on six wards in an NHS hospital. Among 3312 drug administrations observed, the average rate of drug administration errors across six wards was 3.5% (95% confidence interval (CI) = 2.9%, 4.1%) of which 68% were classed as overdue doses errors, an overall rate of 2.4%. In contrast, Rodriguez-Gonzalez et al. [9] analysed 2314 medication administrations. Of 509 errors (an error rate of 22.0%) 441 (86.6%) of the errors were due to dose omission (giving an overall 19.1% overdue doses rate). Other studies have shown overdue doses rates between these two extremes [10-12] and have examined overdue doses in specific drug classes or in specific circumstances, with differing rates of dose omission at the administration stage [13-16].

One reason that MAEs have not received as much attention as other adverse drug events is that counting them accurately is difficult and time consuming [17]. Some previous studies have used the observation technique to detect medication errors [13, 14], in which the total opportunity for errors (TOEs) is calculated by combining the sum of doses omitted and doses administered. However, limitations of this technique include observer bias and the potential for the presence of the observer to affect practitioner behaviour [17]. The era of electronic prescribing and medication administration has made counting drug administrations much easier and removed the aforementioned limitations associated with the TOE technique, although it is realized that computerized provider order entry systems may not prevent administration errors or timing discrepancies [18, 19]. The implementation of such systems may be related to improvements in the quality of care [20], which may include decreased rates of overdue doses. Thus, our objectives are to describe the quality improvement measures introduced in a teaching hospital and dose omission rates over time to evaluate the impact of such interventions.

Methods

Setting and study population

This work was carried out in a large NHS Foundation Trust. The Trust has a locally developed electronic prescribing and administration system known as PICS (Prescribing, Information and Communication System), which is in use throughout all (∼1200) inpatient beds and for all prescribing, except for some chemotherapy regimens. All inpatient admissions are entered into the clinical information system (PICS) for the purpose of prescribing. Prescribers add all drugs to the system which appear on electronic patient records for nurses to administer. The administration stage is also electronically recorded and it is compulsory for all scheduled doses to be charted on PICS as administered or not. The system was first installed in the renal unit in 1998 [20], and now covers general and specialist medical and surgical specialities apart from obstetrics, paediatrics and mental health. A key feature of the system, for the purposes of our study, is that on a weekly basis all information about prescriptions and dose administrations are exported to a comprehensive audit database for subsequent investigation and analyses. In particular, the informatics department is able to generate regular, as well as ad hoc, reports to provide information on specific issues for managers and clinical staff within the Trust.

Interventions

The Trust has prioritized its work to reduce errors over the last few years, and reducing inappropriate dose omissions was the first priority for quality improvement in the financial year 2008–2009. Overdue doses continued to be a key target for the Trust in 2010–11, as specified in the priorities within the Trust's quality accounts. One key driver for this focus has been the ability to report on the details of all drugs prescribed and administered within the PICS system. These and subsequent improvement measures formed the key interventions considered in this investigation, which were introduced as follows: (i) pause function for electronic prescriptions; (ii) clinical dashboards; (iii) visual indicators for overdue doses and (iv) overdue doses RCA meetings/NPSA Rapid Response Alert. Table 1 details the interventions.

Table 1

Key to interventions

	Date	Intervention	Description
A	15 April 2009	Pausing electronic prescriptions	During the first quarter of 2009, an intervention allowing doctors to pause medication within the PICS system was introduced. ‘Paused’ medications are temporarily unavailable for administration until the prescription is subsequently ‘reactivated’. Prior to this, prescriptions not required for a period of time (i.e. those that will now be ‘paused’) were annotated or communicated in other ways, leading to numerous overdue doses being recorded. This new function therefore allowed overdue doses to be acknowledged and audited whilst removing the impact of doses not given for clinically valid reasons
B	4 August 2009	Clinical dashboard	Over the first two financial quarters of 2008–2009 the informatics department interrogated the drug administration records within the data warehouse to construct a clinical dashboard and produced automated reporting tools relating to dose omissions. Targets were set for the ‘acceptable’ rates of overdue doses for three key drug categories: antibiotics, non-antibiotics and dietary supplements (see below). Individual ward performance levels were presented on clinical dashboards available to view by all clinical and managerial staff. In addition, weekly emails based on directorate-level information were sent to divisional directors and managers, with an escalation to executive level if unacceptable thresholds were reached. This system was implemented on the 4 August 2009 and the provision of this information continued throughout the investigation period with the intention-to-raise awareness and motivate clinicians to reduce overdue doses
C	15 December 2009	Visual indicator for overdue doses	Later in 2009, a visual indicator was introduced into the electronic prescribing system interface to show overdue doses in the patient list view. This function indicates where past administrations have not been charted and aims to alert staff to unintentional dose omissions in a timely manner. By alerting to such missing information, doses that were given but not charted can retrospectively be charted, and where clinically viable any actual overdue doses may still be given rather than being completely omitted
D	24 February 2010 and 30 March 2010	NPSA rapid response and overdue doses RCA meetings	In view of the Trust's quality priority to reduce medication errors, monthly executive team meetings were initiated in March 2010, with specific focus on inappropriate overdue doses. Clinical cases were selected via interrogation of electronic records and an RCA was presented to the executive team in meetings chaired by the hospital Chief Executive. Shortly prior to this, in February 2010, a NPSA Rapid Response Alert was distributed regarding overdue doses, requesting NHS organizations to take action in a 12-month plan. These interventions occurred within 6 weeks of each other and thus were combined within our analysis. The Trust maintained its emphasis on drug omissions and undertook the executive meetings throughout the investigation period, with the intention of frequently assessing overdue doses, reviewing targets and maintaining a greater awareness of reducing dose omissions throughout the hospital

Data capture and analysis

Data were abstracted from the PICS audit database on all overdue doses to be administered to adult inpatients between 1 January 2008 and 31 July 2012. Doses were grouped into antibiotics (including antibacterials, antivirals and antifungal drugs for treating infectious conditions), non-antibiotics and nutritional supplements/dietary products. The appropriate administration of antibiotics has been an institutional focus, communicated widely via clinical dashboards, quality accounts [21] and RCA meetings. Thus, the data for non-antibiotics and antibiotics were extracted and analysed separately. Overdue doses were counted if there was a charted ‘non-administration’ (i.e. an active acknowledgment of the omitted dose) but not if there was no charting of a dose (i.e. no record of either administration or omission), which occurred in <5% of cases. All charted omitted doses were included even if a potentially valid reason (such as patient vomiting or a patient refusal) was documented. Due to the large data set (over 3.5 million omitted doses), of which 30% would require manual coding [7], it was not feasible to examine the reasons for omission in charted ‘non-administrations’. We did not consider delays in administration as omissions. Permission to perform this evaluation was obtained from the Clinical Governance Support Unit of the University Hospitals Birmingham NHS Foundation Trust.

Data were analysed using segmented regression analysis of interrupted time series. This method used linear regression to model the change in the rate of overdue doses over time. Variables were included in the model to define the points at which each intervention under consideration occurred. The coefficients of these terms in the model were then used to estimate the magnitude of any change that occurred after each intervention and to test whether these changes were significant. Further details of this methodology are reproduced in the Supplementary data.

Results

During the 239-week study period, a total of 2 121 765 antibiotic and 25 668 583 non-antibiotic doses were prescribed through PICS (a mean average of 8878 and 107 400 per week, respectively). Of these 154 412 (7.3%) antibiotic and 3 293 467 (12.8%) non-antibiotic doses were missed, equivalent to an average of 646 and 13 780 per week, respectively. Omission rates were reduced from 10.3 to 4.4% for antibiotics and from 16.4 to 8.2% for non-antibiotics across the intervention period, indicating total error reductions over the period of 57% for antibiotics and 50% for non-antibiotic drugs.

The time-series plots for percentage of missed antibiotics and non-antibiotics, along with the parsimonious models, are shown in Figs 1 and 2, respectively. The R2 values for the full and parsimonious models were very similar, implying that variability in overdue doses rates explained by the interventions excluded from the latter models was negligible. Hence, the results of the parsimonious models are reported. Further details of these models can be found in Tables 2 and 3.

Figure 1

Observed rates of missed antibiotics, with a regression model.

Figure 2

Observed rates of missed non-antibiotics, with a regression model.

Table 2

Regression coefficients for antibiotic modela

	P	Coefficientb (95% CI)
Antibiotics: full model (R2 = 0.951)
Constant	<0.001*	7.44 (6.14, 8.75)
Lagged dependent variable	<0.001*	0.25 (0.13, 0.38)
Initial gradientc	0.222	0.23 (−0.14, 0.59)
15 April 2009: step change	0.038*	−0.67 (−1.30, −0.04)
15 April 2009: gradient changec	0.971	−0.06 (−3.13, 3.02)
4 August 2009: step Change	0.140	−0.55 (−1.29, 0.018)
4 August 2009: gradient changec	0.515	−1.27 (−5.12, 2.57)
15 December 2009: step change	0.766	0.13 (−0.71, 0.96)
15 December 2009: gradient changec	0.576	−1.78 (−8.06, 4.50)
30 March 2010: step change	0.468	−0.42 (−1.55, 0.71)
30 March 2010: gradient changec	0.501	2.00 (−3.85, 7.85)
Antibiotics: parsimonious model (R2 = 0.950)
Constant	<0.001*	7.43 (6.15, 8.71)
Lagged dependent variable	<0.001*	0.27 (0.15, 0.39)
15 April 2009: step change	<0.001*	−0.49 (−0.80, −0.18)
4 August 2009: Step Change	0.001*	−0.60 (−0.95, 0.26)
4 August 2009: gradient changec	<0.001*	−0.87 (−1.08, −0.67)
30 March 2010: step change	<0.001*	−0.83 (−1.17, −0.50)

aThis table shows the P-values and coefficients of the variables in both the ‘full model’, which considers all of the interventions and the ‘parsimonious model’, which uses a stepwise technique to incrementally remove non-significant variables, giving more statistical power to detect the effects of the remainder. The interventions in the model are represented by ‘step change’ and ‘gradient change’ variables. The coefficient of the former indicates the percentage point change in overdue doses that occurred directly after the intervention. For example, a coefficient of −1 means that, directly after the intervention, the rate of overdue doses fell by 1 percentage point. The gradient change variables have coefficients stating the progressive reduction in overdue doses that occur after an intervention, in terms of percentage points per year. For example, if the rate of overdue doses was 10%, and an intervention had a coefficient of −1, then 1 year after the intervention, the rate of overdue doses would be expected to be 9% and 2 years after the intervention it would be 8%. The coefficient of ‘constant’ term gives the rate of overdue doses at the start of the study period, and the ‘initial gradient’ is analogous to gradient change, but in the period before the first intervention was introduced. The ‘lagged dependent variable’ term is included to adjust for the autocorrelation at lag 1. The significance of this term is indicative of the level of correlation between week x and week [x− 1].

bRepresented as a percentage point change.

cGradient stated in percentage points per year.

*Significant at P< 0.05.

Table 3

Regression coefficients for non-antibiotic modelsa

	P	Coefficientb (95% CI)
Non-antibiotics: full model (R2 = 0.984)
Constant	<0.001*	6.68 (4.89, 8.46)
Lagged dependent variable	<0.001*	0.61 (0.50, 0.71)
Initial gradientc	0.192	−0.19 (−0.48, 0.10)
15 April 2009: step change	0.455	−0.19 (−0.69, 0.31)
15 April 2009: gradient changec	0.826	0.28 (−2.19, 2.74)
4 August 2009: step change	0.166	−0.41 (−1.00, 0.17)
4 August 2009: gradient changec	0.994	−0.01 (−3.09, 3.07)
15 December 2009: step change	0.393	−0.31 (−1.01, 0.40)
15 December 2009: gradient changec	0.928	−0.24 (−5.53, 5.04)
30 March 2010: step change	0.077	−0.88 (−1.85, 0.10)
30 March 2010: gradient changec	0.835	−0.52 (−5.47, 4.42)
Non-antibiotics: parsimonious model (R2 = 0.984)
Constant	<0.001*	6.50 (4.76, 8.23)
Lagged dependent variable	<0.001*	0.62 (0.52, 0.72)
Initial gradientc	0.010*	−0.28 (−0.50, −0.07)
4 August 2009: step change	0.007*	−0.41 (−0.70, −0.11)
30 March 2010: step change	<0.001*	−0.97 (−1.32, −0.61)
30 March 2010: gradient changec	0.003*	−0.38 (−0.64, −0.13)

aThis table shows the P-values and coefficients of the variables in both the ‘full model’, which considers all of the interventions and the ‘parsimonious model’, which uses a stepwise technique to incrementally remove non-significant variables, giving more statistical power to detect the effects of the remainder. The interventions in the model are represented by ‘step change’ and ‘gradient change’ variables. The coefficient of the former indicates the percentage point change in overdue doses that occurred directly after the intervention. For example, a coefficient of −1 means that, directly after the intervention, the rate of overdue doses fell by 1 percentage point. The gradient change variables have coefficients stating the progressive reduction in overdue doses that occur after an intervention, in terms of percentage points per year. For example, if the rate of overdue doses was 10%, and an intervention had a coefficient of −1, then 1 year after the intervention, the rate of overdue doses would be expected to be 9and 2 years after the intervention it would be 8%. The coefficient of ‘constant’ term gives the rate of overdue doses at the start of the study period, and the ‘initial gradient’ is analogous to gradient change, but in the period before the first intervention was introduced. The ‘lagged dependent variable’ term is included to adjust for the autocorrelation at lag 1. The significance of this term is indicative of the level of correlation between week x and week [x− 1].

bRepresented as a percentage point change.

cGradient stated in percentage points per year.

*Significant at P < 0.05.

Inspection of the segmented regression models reveals that, for each of the drug types, both the constant and the lagged dependent variable are significant (P≤ 0.001). This indicates that, in both cases, the initial proportion of overdue doses per week is significantly greater than 0, and that there is significant autocorrelation at Lag 1. This autocorrelation implies that the rate of overdue doses in a week is connected to the rate in the previous week. The remainder of the variables in the parsimonious models are explored below.

Missed antibiotics

The lack of the Week Number term in the parsimonious model indicates there is no evidence that the missed antibiotic dose rate was improving or declining prior to the first quality improvement measure introduced in the Trust—the ability to pause prescriptions. Following this first quality improvement intervention on the 15 April 2009, there was a significant step-change reduction in missed antibiotic doses of 0.49 (95% CI = 0.18, 0.80) percentage points (P < 0.001).

The reporting of overdue doses on clinical dashboards commencing on the 4th of August 2009 also resulted in a step-change reduction in missed antibiotic doses of 0.60 (95% CI = 0.26, 0.95) percentage points (P = 0.001). The intervention also coincided with a significant gradient change of −0.87 (95% CI = −1.08, −0.67) percentage points per year (P < 0.001).

The third intervention (visual indicators for overdue doses) did not coincide with a significant step-change reduction in the number of overdue doses.

A further significant change coincided with the commencement of overdue doses RCA meetings. This took the form of a step-change reduction in the rate of missed antibiotic doses of 0.83 (95% CI = 0.50, 1.17) percentage points after the 30th of March 2010 (P < 0.001). There was no significant gradient change after this intervention.

Missed non-antibiotics

The inclusion of the Week Number term in the parsimonious model indicates that the percentage of missed non-antibiotic doses was already in significant decline prior to the first quality improvement measure introduced in the Trust. The magnitude of this initial gradient is −0.28 (95% CI = −0.50, −0.07) percentage points per year (P = 0.010).

The reporting of overdue doses on clinical dashboards commencing on the 4th of August 2009 coincided with a significant step-change reduction in the rate of missed non-antibiotic doses of 0.41 percentage points (95% CI = 0.11, 0.70; P = 0.007).

Similarly to the missed antibiotics, the third intervention (visual indicators for overdue doses) did not coincide with a significant step-change reduction in the number of overdue doses.

A second significant step change was observed upon the introduction of overdue doses RCA Meetings on the 30th of March 2010, at which the rate of missed non-antibiotic doses fell by 0.97 percentage points (95% CI = 0.61, 1.32; P < 0.001). In addition, a significant change in the gradient of −0.38 (95% CI = −0.64, −0.13; P = 0.003) was also detected at this time.

Discussion

The analysis of the time-series data may reflect sustained improvement in the reduction of overdue doses following a number of targeted quality improvement interventions leading to significant changes in behaviour within a healthcare institution. At the end of the intervention period, the overdue doses rates in this study were reduced to 4.4 and 8.2% for antibiotic and non-antibiotic drugs, respectively, indicating lower overdue doses rates at this hospital compared with other research [9, 10, 11, 13–15]. However, electronic prescribing in isolation cannot result in reductions in overdue doses. Instead, reducing medication omissions is a multifaceted task with various influential factors. When utilized in a proactive way, data from the electronic prescribing system coupled with other features (e.g. pausing prescriptions, clinical dashboard) can be used to help reduce overdue doses rates. In particular, board-level involvement with a specific strategic goal for quality improvement together with tools allowing critical oversight of the issue has improved institutional performance in this area.

Following implementation of the pause function for doctors, a significant decline of 0.49 percentage points was observed for antibiotic drugs. Taking into account the levels of prescription activity, which were on average 8878 antibiotics and 107 400 non-antibiotics per week, this value is equivalent to 43 fewer missed antibiotic doses per week. No significant change was observed in non-antibiotic drugs. This may be because intravenous drugs requiring therapeutic drug monitoring could now be paused until the drug concentration is available to the physician, or due to physicians pausing antibiotic doses for perioperative patients. Thus, the lack of administration of a particular antibiotic dose is no longer recorded as a overdue doses.

The implementation of clinical dashboards alongside regular feedback was followed by an immediate reduction in overdue doses of 0.60 percentage points in antibiotic prescribing and of 0.41 in non-antibiotic prescribing, representing reductions in the frequency of weekly dose omissions of 53 and 436, respectively. In addition, the significant gradient change for missed antibiotic doses of −0.87 percentage points per year implies that the step-change reduction of 53 overdue doses per week increased by 1 during each subsequent week over the evaluation period.

Despite the previous two interventions appearing successful, implementing visual indicators for overdue doses did not appear to result in significant decreases in overdue doses rates. This intervention only involved one small change to the system, and therefore may not have been substantial enough to produce significant changes to drug administrators' behaviour. However, it is also possible that the significance of this intervention in reducing overdue doses was effectively overshadowed by the previous interventions. As such, it is not possible to conclude whether in isolation the implementation of visual indicators would help to reduce missed medication doses.

Targeted improvement measures are often more successful compared with provision of information alone. Results of the analysis support this, with larger reductions in dose omissions observed after implementing targeted improvement measures. For example, the RCA meetings (a targeted intervention) were associated with step-change reductions of 0.83 and 0.97 percentage points, equivalent to weekly reductions in dose omissions of 74 and 1038 for antibiotics and non-antibiotics, respectively. In comparison, the implementation of clinical dashboards (a non-targeted intervention) were associated with smaller step-change reductions of 0.60 and 0.41 percentage points, equivalent to weekly reductions in dose omissions of 53 and 436 for antibiotics and non-antibiotics, respectively. An additional significant gradient change for missed non-antibiotics of −0.38 percentage points per year, equivalent to reductions of 8 dose omissions each week, also coincided with the targeted RCA intervention.

Owing to the close temporal association of the NPSA alert and the introduction of executive RCA meetings, it is impossible to attribute either one to these step changes in omission rates. The publication of the NPSA alert supported the credibility of the executive team's already planned RCA meetings in 2010, and it is the authors' opinion that the RCA meetings provided greater impetus for change within the organization. This would support other healthcare management literature demonstrating the positive effects of executive level oversight of quality and safety measures within organizations [22].

There are many other strategies to reduce overdue doses in hospitalized patients that we have not systematically implemented within our institution, including interventions such as unit dosing systems (as commonly used in the USA), where drug doses are individually dispensed, often from electronic carts [23], increased reliance on patient self-administration where appropriate [24] and separate nursing bedside reminder systems [25].

Electronic prescribing systems with embedded electronic medication charts allow easy auditing of overdue doses but when used in isolation are unlikely to demonstrate reduced omission rates. Rather, previous studies have demonstrated that dose omissions may actually increase with the introduction of electronic prescribing systems, partly because nurses' drug delivery expectations increase as orders are made immediately available to them [26]. The significant declines in overdue doses rates upon the implementation of interventions suggests that if audit data gathered by electronic prescribing systems are made readily available to hospital staff (e.g., through the use of clinical dashboards) and monitored regularly (e.g., by board-level staff) then using such a system can stimulate decreased rates of overdue doses.

What is of particular interest is the potential burden of omitted doses in this study across the groups of drugs. Whilst single dose omissions may be considered to have relatively low risk of harm they are costly in terms of efficiency of staff time searching for doses and may in some circumstances (such as first doses of antibacterial drugs) contribute to serious adverse events. Within the same hospital being investigated in the current study, Rosser et al. [27] found that using the electronic prescribing system to assist with overt efforts to reduce overdue doses rates was associated with a decrease in mortality rates by 16.2%, although clearly causality cannot be proven. The reduction of inappropriate overdue doses rates therefore seems self-evident. However, clinical judgement would tend to indicate that a 0% overdue doses rate is neither achievable nor desirable. There are likely to be situations where nurse judgement to omit medicines despite them being due on the prescription is valid given new or evolving circumstances to which the prescriber is not aware. Examples may include adverse effects of a recent dose or a change in situation that would suddenly contraindicate the use of a drug. Furthermore, Coleman et al. [7] discovered that patient refusal is the most common reason provided for dose omissions, accounting for 45% of all overdue doses over four randomly selected 7-day periods in 2010. Patient rights to refuse medication must remain, and refusal will likely continue at similar rates over time. Therefore, although reducing overdue doses is generally believed to be ideal, it is currently unknown what an acceptable rate of overdue doses would be in a hospitalized inpatient population. However, if no valid reason can be provided for missing a dose, it might be argued that the rate of overdue doses should be 0.

Limitations

Whilst segmented regression analysis is considered a strong quasi-experimental design, it still suffers from various limitations. The key one is the assumption that, apart from the interventions considered, all other factors remain constant throughout the whole intervention period. Any variability caused by factors other than the interventions considered could have had a confounding effect on any observed changes in the rates of overdue doses. For example, it is possible that the patient case mix changed over the study period. However, given the large tertiary care hospital setting from which the data are extracted we believe this is unlikely.

A further limitation is that we only considered doses where there were charted administrations or non-administrations, which therefore may lead to the over- or under-estimation of the actual number of overdue doses. However, an active decision was taken to exclude these cases as it is not possible to determine whether these doses have actually been omitted or not. In addition, due to the large number of cases it was not practical to investigate reasons for dose omission in this study. However, in future research it would be ideal to consider performing a sensitivity analysis to investigate whether the relative proportions of these reasons change over time. Also, there is an assumption in the analysis that the omission of doses is an independent event. It is possible that there is some non-independence, for example patients repeatedly refusing medications. However, we believe that the effect of this on the final analysis is minimal. Finally, the temporal proximity of the interventions may limit the ability to determine the precise effects of each individual intervention.

The ability to pause drugs coincided with fewer missed antibiotics. It could be argued that the inability to electronically ‘cross-out’ doses prior to the first (pausing function) intervention may provide reason for the initially high omission rates. However, as our initial rates of overdue doses were not dissimilar to previous research studies, and due to temporal trends coinciding with the other interventions, we believe that institution-wide improvement, rather than simply fixing system problems, is demonstrable by this evaluation.

Within this analysis we did not investigate the different rates of change for overdue doses in antibiotic and non-antibiotic drugs prior to the first intervention, or whether the medication administration method may affect overdue doses rates. For example, differences may occur between orally and intravenously administered medications. Furthermore, despite coded and free-text reasons being given for all documented omitted doses, we did not consider reasoning for, or outcomes of drug omissions, as previous studies have done [7, 12]. However, this was not feasible due to the sheer numbers of administrations being analysed in our study. We also did not consider drug refusal, or other legitimate reasons for omission in this evaluation; however, we have previously described how 45% of omissions may fall into this category [7]. Assuming the proportion of valid reasons for omissions remained stable over the study period, we still believe the intervention to have produced real reductions in omission errors. Reasons behind overdue doses in different drug classes involving different administration methods deserve further consideration in future research to complement the current findings.

Conclusion

Computerized physician order entry systems allow a level of data collection for measuring medicines administration errors on a different scale to direct observational review. Our study has evaluated over 23 million charted doses in hospitalized patients to detect overdue doses over time. The use of clinical dashboards allowing performance indicators to be communicated to staff and board-level involvement in specific quality improvement targets appear to demonstrate some effectiveness in reducing overdue doses. Omitted drug therapy remains an important safety issue that the Trust is committed to reducing. Reduced rates of overdue doses are one indicator of increased quality of care within the hospital. A defined, clinically acceptable level of omissions is not clear from this data analysis; however it is apparent that continued observation of overdue doses rates and targeted interventions within hospital settings will be required to reach such a consensus in the future.

Supplementary material

Supplementary material is available at .

Authors’ contribution

All authors had full access to all of the data (including statistical reports and tables) in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. All authors contributed to the writing of the manuscript, the interpretation of data and approved the final version.

Funding

This work was supported by the National Institute for Health Research (NIHR) through the Collaborations for Leadership in Applied Health Research and Care for Birmingham and Black Country (CLAHRC-BBC) programme. The views expressed in this publication are not necessarily those of the NIHR, the Department of Health, University Hospitals Birmingham NHS Foundation Trust, University of Birmingham or the CLAHRC-BBC Theme 9 Management/Steering Group. Funding to pay the Open Access publication charges for this article was provided by Collaboration for Leadership in Applied Health Research and Care.

Conflict of interest

(i) Financial support for the submitted work was obtained from the National Institute of Health Research CLAHRC for the West Midlands and Black Country. (ii) J.J.C., D.R., H.L.B. and J.H. work within the University Hospitals Birmingham NHS Foundation Trust which is collaborating with CSE Healthcare Systems to commercialize the PICS system in the UK. All other authors report no financial relationships with commercial entities that might have an interest in the submitted work. (iii) No spouses, partners or children of the authors have relationships with commercial entities that might have an interest in the submitted work. (iv) None of the authors have non-financial interests that may be relevant to the submitted work.

Data sharing

Additional information about the data analysis and results is available from the corresponding author on request.