Cost-effectiveness analysis of a hospital electronic medication management system.

OBJECTIVE: To conduct a cost-effectiveness analysis of a hospital electronic medication management system (eMMS).
METHODS: We compared costs and benefits of paper-based prescribing with a commercial eMMS (CSC MedChart) on one cardiology ward in a major 326-bed teaching hospital, assuming a 15-year time horizon and a health system perspective. The eMMS implementation and operating costs were obtained from the study site. We used data on eMMS effectiveness in reducing potential adverse drug events (ADEs), and potential ADEs intercepted, based on review of 1 202 patient charts before (n = 801) and after (n = 401) eMMS. These were combined with published estimates of actual ADEs and their costs.
RESULTS: The rate of potential ADEs following eMMS fell from 0.17 per admission to 0.05; a reduction of 71%. The annualized eMMS implementation, maintenance, and operating costs for the cardiology ward were A$61 741 (US$55 296). The estimated reduction in ADEs post eMMS was approximately 80 actual ADEs per year. The reduced costs associated with these ADEs were more than sufficient to offset the costs of the eMMS. Estimated savings resulting from eMMS implementation were A$63-66 (US$56-59) per admission (A$97 740-$102 000 per annum for this ward). Sensitivity analyses demonstrated results were robust when both eMMS effectiveness and costs of actual ADEs were varied substantially.
CONCLUSION: The eMMS within this setting was more effective and less expensive than paper-based prescribing. Comparison with the few previous full economic evaluations available suggests a marked improvement in the cost-effectiveness of eMMS, largely driven by increased effectiveness of contemporary eMMs in reducing medication errors.

Introduction

Evidence of the cost-effectiveness of most clinical information systems remains scant. Medication-related health information technologies (HITs), which include computerized order entry (CPOE) systems with electronic medication ordering and administration functions have been shown to significantly reduce medication errors, particularly prescribing errors among hospital patients. A 2012 systematic review of economic evaluations of medication-related HIT, however, identified only five full economic evaluations that combined data on the incremental costs and effects, and calculated either an incremental cost-effectiveness ratio (ICER) or net benefit of the electronic medication management system (eMMS). The systematic review identified 26 partial economic evaluations in the form of cost analyses and broader financial implications of CPOE and/or clinical decision-support systems in hospitals and primary care settings. The authors concluded that the quality of the reviewed studies was generally poor, and, although some studies found cost advantages of HIT, the estimated financial gains and other benefits were associated with considerable uncertainties that prevented a definitive conclusion as to whether the additional costs of CPOE and clinical decision-support systems represent value for money.

Australian hospitals are in the early stages of the implementation of eMMS. The majority has selected to implement commercial eMMS, many of which are from US-based companies. Australian State and Territory governments have e–health strategic plans with eMMS identified as the biggest driver for achieving significant clinical benefits expected to deliver reduced health care costs. However, these strategic plans are accompanied by very modest procedures for assessing or quantifying expected benefits. There has been criticism that due attention is not being placed on the evaluation of systems, particularly given the substantial financial investments being made. As recent government reports, have highlighted, there are growing concerns about this situation: “Unfortunately, there has been limited assessment to date of the benefits and outcomes of the various clinical IT systems put in place … Until this work is done, it will be difficult to convince taxpayers that public funds have been well spent on these systems and that any further investment on clinical ICT systems is justified, or will improve clinical and patient outcomes,” In contrast, the “Meaningful Use” incentive program in the United States has motivated a substantial acceleration in HIT evaluation studies, yet HIT cost-effectiveness studies are still rare. The aim of this study was to contribute to the limited evidence-base regarding the costs and benefits of eMMS by undertaking a cost-effectiveness analysis of eMMS versus paper-based prescribing in reducing medication errors and preventing adverse drug events (ADEs) in hospital.

Methods

The intervention evaluated was a commercial eMMS (CSC MedChart) implemented in a 326 bed academic teaching hospital in Sydney, Australia. A modeled economic evaluation was conducted from the perspective of the State’s health system. The time horizon for the economic evaluation was 15 years starting from 2005, the year the eMMS was first implemented in a ward at the hospital.

Prior to the implementation of eMMS, all wards used paper medication charts onto which doctors directly wrote medication orders. These charts were then used by nursing staff to administer medications. In 2005 the hospital commenced implementation of a commercial eMMS – CSC MedChart – that interfaced with the hospital’s existing CPOE. The eMMS allows doctors to prescribe medication electronically, but contains no bar coding or automated dispensing of medications. The system included a number of alerts for drug allergy checking, pregnancy warnings, therapeutic duplication, some dose-range checking, and a number of local decision-support rules. Drug–drug interaction alerts were not operational at the time post-eMMS medication error data were collected. Details of the functionalities of MedChart are published elsewhere.,

Outcomes

Prescribing error data collected pre-eMMS and post-eMMS implementation allowed identification of potential ADEs occurring on the 30 bed cardiology ward at the study hospital. Data for the 30-bed cardiology ward were collected over 16 weeks before eMMS (from November 2007 to March 2008) and over 10 weeks after implementation, which occurred in August 2009 (from December 2009 to February 2010).

Three independent clinical pharmacists reviewed each patient’s medication chart to identify prescribing errors. Inter-rater reliability of reviewers was assessed. Potential severity of all identified prescribing errors was rated on a five-point scale (Severity Assessment Code) by the pharmacists. Reliability of severity scoring was performed by a multidisciplinary committee conducting an independent classification of subsets of errors. The severity scale classified prescribing errors as: “insignificant” (incident is likely to have little or no effect on the patient); “minor” (incident is likely to lead to an increase in level of care, eg, review, investigations, or referral to another clinician); “moderate” (incident is likely to lead to permanent reduction in bodily functioning, surgical intervention); “major” (incident is likely to lead to a major permanent loss of function); and “serious” (incident is likely to lead to death). ADEs are generally defined as “an injury, large or small caused by the use (including nonuse) of a drug.” Prescribing errors that may cause harm to patients included those from the moderate, major, and serious severity categories, and were categorized as potential ADEs.

During chart review, any evidence that medication errors had been detected and intercepted by staff, such as a correction or note made by a doctor, nurse, or pharmacist on the medication chart, or progress notes, or the existence of a pharmacist’s intervention report (eg, indicating that the doctor had been contacted to review an error in the order), was recorded. These data were used to determine the proportion of potential ADEs intercepted by staff.

Costs

Data on resource use associated with the implementation and operation of the eMMS were obtained from the IT vendor, examination of hospital financial records, and other relevant documentation (eg, schedules of regular clinical personnel training, work diaries). The accuracy of the data was confirmed during interviews with hospital pharmacists and from clinical information by IT and hospital managers who were involved in eMMS implementation and maintenance. The cost of resources was estimated in 2012–2013 prices. In addition to discounting, the annualized value of initial investments and recurrent costs were calculated using constant 2012–2013 Australian dollars.

Costs were categorized as initial costs (eg, investments in equipment and software, infrastructure establishment and upgrade, staff time spent on the initial system configuration and implementation) and operating costs (eg, annual licences and subscriptions, annual clinical personnel training, and system updates). MedChart requires annual license fees to be paid by the hospital on a per bed basis.

The proportion of initial costs that occurred at the hospital level was allocated to the study ward in proportion to the number of beds and combined with initial costs that occurred at the ward level. The service life of IT and other equipment varied from between two to seven years depending on replacement practices, while infrastructure and accumulated staff knowledge were assumed to last over the time horizon. The present value of each cost component was obtained using an annual 5% discount rate and the relevant useful life. This represents a social discount rate rather than hospital accounting practices.

Cost calculations were conducted using an “incremental approach,” where only eMMS-specific costs were included. The costs of IT equipment and infrastructure in place prior and irrespective of eMMS introduction were not counted. Annual salaries of eMMS staff with 20% on-costs (eg, annual leave, superannuation) were obtained from the relevant award categories. As work hours spent on eMMS-related activities varied between staff, costs were estimated in proportion to eMMS-related duties and then allocated to the cardiology ward.

Although the study unit was a cardiology ward, where the eMMS was introduced in 2009, the implementation was part of a hospital-wide rollout that took place over 2005–2011 as described elsewhere. A proportion of staff time incurred at the start of eMMS implementation was allocated to the subsequent stages of the rollout to recognize the value of significant “knowledge capital” accumulated during the first year by the core project staff. In addition to discounting, the annualized value of initial investments and recurrent costs were calculated. Annualization converts the entire stream of discounted costs into a series of equal annual payments. These cost data were used to produce an incremental cost per additional ADE avoided in a typical admission to the cardiology ward.

Decision-analytic model

A simple decision tree (figure 1) was designed to estimate ICER, where costs include the initial and operating costs of eMMS, as well as any cost-offsets resulting from the downstream consequences of implementation. Effectiveness was assessed in terms of reduction in actual ADEs per admission post eMMS. The structure of the model takes into consideration that a proportion of prescribing errors are intercepted before reaching patients,, therefore, the probability of actual harm (ADE) applied only to nonintercepted errors. The values of parameters that could not be obtained from our original eMMS study were obtained from the literature. Uncertainty of results was assessed by one-way and two-way sensitivity analyses (varying the parameters that most affected the results). The software TreeAge Pro was used to run the model. A file with our model is available by contacting the authors.

Figure 1:

The structure of a decision tree model.

Populating the decision analytic model

The following parameter estimates were obtained from the literature. The probabilities of a nonintercepted ADE resulting in an actual ADE were obtained from two papers by Bates et al. with the corresponding estimates of 0.385 and 0.591. The estimate of 0.591 was used in the base–case analysis as this represented the rate of nonintercepted prescribing errors resulting in actual ADEs. In the sensitivity analysis we used Kaushal et al.’s considerably lower rate of 0.096 of nonintercepted ADEs resulting in actual ADEs. The distribution of actual ADEs by degree of seriousness was also obtained from the studies by Bates et al.,. The weighted average proportions from these two studies were: 20% severe, ie, fatal/life threatening; 41% serious, resulting in temporary harm to the patient and requiring hospitalization; 39% significant, resulting in temporary harm to the patient and requiring intervention.

Utilization of health care resources for each category of ADE was reported in 2009 British pounds by Karnon et al. These figures were converted to Australian dollars using the 2009 purchasing power parity of A$2.18 to UK£1 (http://stats.oecd.org/Index.aspx?datasetcode) and uprated to 2013 values using the health inflation index. As these cost estimates do not reflect Australian health care costs and practices, we also undertook an alternative estimation of cost savings resulting from the reduction in ADEs and the subsequent reduction in length of stay (LOS) for each ADE category,, which is multiplied by the cost of a bed day (A$472.5) obtained from the hospital accounting records. Table 1 lists variables and assumptions used in the modeled cost-effectiveness analysis.

Table 1:

Assumptions and parameter estimates used in the modelled cost-effectiveness analysis

Variable	Base–case parameter estimates (range)	Source
Modeling assumptions
Useful life of the eMMS	15 (10–20) years	Expert advice
Number of admissions	1541 (1494–1584)	Hospital admissions database
Effectiveness
Probability of potential ADE pre-eMMS	0.17	Data collected during the MedChart effectiveness study8
Probability of potential ADE occurring post- eMMS implementation	0.05	Data collected during the MedChart effectiveness study8
Probability of intercepting a potential ADE pre-eMMS	0.161	Data collected during the MedChart effectiveness study8
Probability of intercepting a potential ADE post-eMMS	0.074	Data collected during the MedChart effectiveness study8
Probability of nonintercepted potential ADE resulting in actual ADE	0.591	Bates et al.16
Probability of actual ADE by severity level
Severe	0.20	Campbell et al.21
Serious	0.41
Significant	0.39
Cost
Annual cost of the eMMS attributable to the cardiology ward	A$61 741	Tables 2–4
Cost per ADEa by severity
Severe	A$3679 (A$2490–A$4866)	Karnon et al.24
Serious	A$2522 (A$1637– A$3406)
Significant	A$247 (A$149–A$344)
Additional length of stay associated with ADEs of differing severity
Severe	7.5 (4.6–10)	Karnon et al.22
Serious	4.3 (4–4.6)	Hug et al.25
Significant	1.5 (0–3)	Bates et al.26
Cost of a bed day	472.5 (423–522)	Hospital accounting records

aConverted into 2012 Australian dollars.

Results

The eMMS effectiveness

There were 801 patient admissions to the cardiology ward in the pre-eMMS study period and 401 in the postimplementation period. There were no statistically significant differences in mean age, gender profile, or LOS in the cardiology ward patients pre-eMMS and post-eMMS (table 2).

Table 2:

Demographic and clinical characteristics of cardiology patients

Demographics/LOS	Preintervention N = 801	Postintervention N = 401	Results of statistical tests
Mean age (SD)	64.3 (17.2)	63.5 (17.8)	t = 0.75, P = 0.45
Gender (% female)	34.2	36.2	χ2 = 0.45, df = 1, P = 0.5
Mean length of stay in days (SD)	7.3 (9.1)	8.1 (11.6)	t = 1.16, P = 0.24

Reduction in the proportion of prescribing errors not classified as potential ADEs (ie, insignificant or minor severity) post-eMMS was 38% (from 0.45 to 0.28 per admission; χ2 = 32, df = 1, P < 0.0001). The proportion of potential ADEs fell from 0.17 per admission to 0.05 (χ2 = 36, df = 1, P < 0.0001); a reduction of 71%. The effectiveness gain of preventing 0.12 potential ADEs per admission, and the associated reduction in resource use, were assumed to remain constant over the time horizon of the economic evaluation. Post-eMMS, the proportion of intercepted potential ADEs changed from 0.16 to 0.07 but was not statistically significant (χ2 = 1.4, df = 1, Fisher’s Exact test P = 0.4); however, the total number of these ADEs in the postimplementation period was small.

Cost of the eMMS

Table 3 shows hospital-level costs of installing and maintaining the eMMS over the time horizon of 15 years and estimates of the annualized costs attributable to the cardiology ward. Initial hospital-wide investments associated with eMMS implementation, together with regular replacement of equipment and rental opportunity costs over the period of 15 years, were A$317 786 (US$284 766). The annualized cost of hardware, database, and infrastructure attributable to the cardiology ward was estimated at A$2371.

Table 3:

Hospital-level costs associated with the implementation and maintenance of the eMMS incurred over the estimated time horizon of 15 years and annualized cost for the study ward

Cost category	Amount of resources	Cost per unit of resource ($A)	Total cost over 15 years in 2012–2013 prices (A$)	Annualized cost attributed to the study ward (A$)
Hardware and peripherals
Serversa	2	6000	60 000	532d
Databaseb	1	10 000	21 428	190d
Training room refurbishment
Rental opportunity cost (market price per year × 15 years)	20 m2	545/m2	163 500	1003d
Furniture (whiteboard, etc.)b	1	2500	5357	47d
Equipment (PCs)c	9	1000	67 500	598d
Subtotal cost (nonpersonnel investment)			317 786	2371
MedChart Configuration and associated tasks (“knowledge capital” at the start of the project in 2005)		Full time equivalent	Knowledge capital
One IT specialist	3 months	180 000	45 000
One clinical Information System	6 months	110 876	55 438	N/Ae
Manager (Health Service Manager, Level 2)
One Pharmacist Grade 1, final year	12 months	85 135	85 135
Subtotal cost (personnel investment)			185 573

aUseful life assumed to be 3 years; buseful life is 7 years; cuseful life is 2 years; dallocated in proportion to the number of beds in the ward (30/326); eincluded in Table 3

In addition, staff time invested at the start of the project accounted for more than half the total personnel cost at hospital-level (A$185 573). Initial installation, configuration, testing of the eMMS, as well as updating hospital protocols and guidelines required the equivalent of one full-time pharmacist and clinical information system manager working half-time. Together they generated significant “knowledge capital,” which reduced staff time in the subsequent stages of eMMS rollout between 2007 and 2011. To reflect the long-lasting effect of staff knowledge, the annualized amount of “knowledge capital” (A$32 070) was allocated in proportion to the 2 months it took to implement the eMMS in the cardiology ward in 2009 (A$5345, table 4). The equivalent annual cost of this initial investment of “knowledge capital” at the ward level was A$692 (table 4).

Table 4:

Ward-specific costs of implementation and maintenance of the eMMS

Cost category	Amount of resources	Cost per unit of resource (A$)	Total cost over 10 years in 2012–2013 prices, (A$)	Annualized cost (A$)
Share of “knowledge capital” attributable to the eMMS rollout in the cardiology ward in 2009	N/A	N/A	5345	692
Hardware and peripherals
Customized trolleys for laptopsa	6	1200	30 000	3885
PC laptopsb	6	1000	14 400	1865
Direct staff time spent on rollout of the eMMS in the cardiology ward, 2009 (Personnel investment)		Full time equivalent
One IT specialist	19 h	180 000	2025	262
One Clinical Information System	39 h	110 876	2495	323
Manager (Health Service Manager, Level 2)
One Pharmacist Grade 2, second year	39 h	94 613	2129	276
One Training coordinator (Health Service Manager, Level 1)	82 h	90 482	2036	264
Facilities upgrades, including ward preparation and renovationc	N/A	N/A	2,000	259
Wireless access installation, equipmentc	6	180	1080	490
six access points and six network ports	6	450	2700
Security provision
Emergency PCb	1	1000
Uninterruptible Power Supply (UPS)b	1	100	6000	777
Printerb	1	100
Total initial cardiology ward-specific costs			70 210	–
Total cardiology ward-specific annual costs			–	9093

aUseful life is 5 years; buseful life is 2 years; cuseful life is 10 years

Table 4 shows that the initial cost of implementation and ongoing maintenance of the eMMS in the study ward was A$70 210 in 2012–2013 prices, which corresponds to an annualized cost of $9093. Table 5 shows the eMMS operating annual costs attributed to the cardiology ward. The single largest cost component was the MedChart software licence fee of A$25 680 for the 30 bed ward.

Table 5:

Annual operating costs associated with the eMMS incurred at the ward level

Cost category	Amount of resources	Cost per unit of resource (A$)	Annual cost (A$)
MedChart software licence fees and upgrades (annual cost per bed)	30 beds	856	25 680
Subscription to online reference texts for the integrated clinical decision-support system
Therapeutic Guidelines	1	3500	322a
MIMS	1	11 000	1012a
Australian Medicines Handbook	1	4300	396a
Database maintenance and training (salaries of pharmacists, clinical information manager and eMMS trainer)		Full time equivalent
One IT specialist	0.25 FTE	180 000	3409b
One Clinical Information System Manager (Health Service Manager, Level 2)	0.5 FTE	110 876	4200b
One Senior Pharmacist Grade 3, second year	1 FTE	110 924	6855b
One Clinical information trainer (Health Service Manager, Level 1)	1 FTE	90 482	8403b
Total operating ward-specific costs	50 277

aAllocated in proportion to the number of beds in the ward relative to the total number of beds at the hospital (30/326); ballocated in proportion to “weights” reflecting resources required for the complexity of management of MedChart in different ward environments

The total annual cost of the eMMS in the cardiology ward was A$61 741 ($2371 + $9093 + $50 277). To estimate the economic impact of the eMMS on ADEs avoided, we undertook a modelled economic evaluation using the data from our previous study at the hospital, which measured prescribing errors pre-eMMS and post-eMMS implementation. We also populated the model with published data on the proportion of nonintercepted potential ADEs likely to result in actual ADEs. Modelled economic evaluation includes the eMMS-related costs, as well as cost-savings associated with the reduction in resource use (therapy and hospital stay) resulting from a reduction in actual ADEs.

Results of the modelled economic evaluation

Table 6 shows results of the base–case analyses of the incremental cost per additional ADE avoided in a typical admission following eMMS implementation on the cardiology ward. Two modelled economic evaluations were performed. One utilized published estimates of costs per ADE (Model I), and the second used a combination of the published estimates of additional LOS with the actual cost per bed day in the cardiology ward (Model II).

Table 6:

Results of the cost-effectiveness analyses of the eMMS in preventing ADEs per admission

Intervention/Comparator	Total cost per admission (eMMS + cost of ADEs)	Number of ADEs per admission	Incremental cost (A$)	Incremental number of ADEs	ICERa (ΔA$/ΔADE)
Model I: Published estimates of costs per ADE by level of severity
No eMMS	157.31	0.084	–	0.057	eMMS dominates
eMMS	91.12	0.027	−66.17	–
Model II: Published estimates of additional LOS per ADE combined with average cost per bed day in the cardiology ward
No eMMS	153.26	0.084	–	0.057	eMMS dominates
eMMS	89.80	0.027	−63.43

Δ = incremental; ICER = incremental cost-effectiveness ratio.

aAn intervention that is both more effective and less expensive is said to dominate the comparator.

The base–case analysis shows that implementation of eMMS prevented 0.057 actual ADEs per admission (from 0.031 at baseline to 0.0098 post eMMS). Across the expected useful life of the eMMS, the total number of actual ADEs avoided in the cardiology ward, discounted at 5%, would be equal to 818.

Model I

Results of the base–case analysis with published costs of ADEs show the reduction in resources used to treat ADEs post eMMS implementation was more than sufficient to offset the costs of eMMS implementation, maintenance, and operation. An intervention that is both more effective and less expensive is said to dominate the comparator (in this case a paper-based medication system). The amount of savings resulting from eMMS implementation is estimated at A$66.17 per admission or about A$102 000 per annum on this ward. When the lower published estimates of costs resulting from ADEs of different severity levels were used in the model, eMMS remained the dominant intervention, but savings per admission decreased to A$29.8. When the upper estimates of cost per ADE by severity were used in the model savings, post eMMS increased to A$103 per admission (Table 7).

Table 7:

Results of the sensitivity analyses

Parameter	Value	ICER (ΔA$/ΔADE)
Number of admissions (doubled)	1500*2 = 3000	eMMS dominates, saving A$86 per admission
Number of admissions (reduced by half)	1500/2 = 750	eMMS dominates, saving A$24 per admission
Probability of potential ADE pre-eMMS implementation (doubled)	0.17*2 = 0.34	eMMS dominates, saving A$224 per admission
Probability of potential ADE pre-eMMS implementation (reduced by half)	0.17/2 = 0.085	ICER = $A844 per additional ADE avoided
Probability of potential ADE after eMMS implementation (doubled)	0.05*2 = 0.1	eMMS dominates, saving A$15 per admission
Probability of potential ADE after eMMS implementation (reduced by half)	0.05/2 = 0.025	eMMS dominates, saving A$92 per admission
Probability of intercepting an error (assumed unchanged from pre-eMMS period)	0.161	eMMS dominates, saving A$71 per admission
Probability of intercepting an error as reported in a review by Kaushal et al.20	0.68	eMMS dominates, saving A$2.3 per admission
Probability of harm from nonintercepted error reported by Kaushal et al.20	0.096	ICER = $2466 per additional ADE avoided
Probabilities of intercepting an error and resultant harm from nonintercepted error as reported by Kaushal et al.20	0.68 0.096	ICER = $A9002 per additional ADE avoidedb
Probability of harm from nonintercepted ADE reported from a study by Bates et al.16	0.385	eMMS dominates, saving A$29 per admission
Distribution of ADE by degree of severity
Significant	0.30	eMMS dominates, saving A$91 per admission
Serious	0.30
Severe (doubled)	0.40
Distribution of ADE by degree of severity
Significant	0.45	eMMS dominates, saving A$52 per admission
Serious	0.45
Severe (reduced by half)	0.10
Cost per potential ADE (lower boundaryc)
Severe	A$2,490	eMMS dominates, saving A$29.8 per admission
Serious	A$1,637
Significant	A$149
Cost per potential ADE (upper boundaryc)
Severe	A$4,866	eMMS dominates, saving A$102 per admission
Wherever the range of parameter serious	A$3,406
Significant	A$344

aWherever the range of parameter estimates was not available, the values were doubled or reduced by half; btwo-way sensitivity analysis; cKarnon et al. as shown in Table 5

ICER = incremental cost-effectiveness ratio.

Model II

In this second model the costs per ADE of differing severity were calculated by multiplying the cost per bed day in the cardiology ward by the published estimates of the corresponding extended LOS per ADE. This showed savings were A$63.43 per admission. Applying the lower or the upper boundary of the estimated LOS varied the corresponding savings from A$28.8 to A$95.9. In the base–case, savings were estimated at A$97 740 per annum.

A threshold analysis was undertaken to examine the extent to which the costs of the eMMS could increase before savings were eliminated. This analysis demonstrated that the annualized cost of the eMMS could increase by up to 2.65 times (ie, A$163 650) before reaching the breakeven point. Alternatively, we examined the extent to which the effectiveness of the eMMS could be reduced before savings were eliminated. We calculated that the rate of potential ADEs post eMMs could increase and be as high as 0.115 or 57% higher than the rate of 0.05 observed at the study site. Table 7 (online supplement) shows a number of one-way and two-way sensitivity analyses undertaken to investigate the robustness of results to individual parameter variations.

Results were robust with respect to the main conclusion that the eMMS presents good value for money by saving costs and preventing ADEs. The amount of savings was sensitive to the effectiveness of the eMMS in reducing potential ADEs, costs associated with ADEs, variations in the number of admissions, and the baseline probability of potential ADEs. Results were sensitive to large variations in the probabilities of intercepting an error, and of nonintercepted errors resulting in actual ADEs. Results were less sensitive to the distribution of ADEs by degree of severity. Results changed very little when the time horizon varied from 15 years in the base–case to 10 and 20 years.

Discussion

Results showed that investment in the eMMS, in an academic Australian public hospital, was associated with savings estimated at A$63–66 (US$56–59) per admission, which translates to estimated savings for the cardiology ward alone of between A$97 740 and $102 000 per annum. Across the hospital with 39 900 annual admissions this equates to savings of ∼A$2.5 M. The sensitivity analyses demonstrated results were robust to large variations in eMMS effectiveness (rate of ADEs prevented) and costs of treating an actual ADE.

This study is one of only a few full economic evaluations, which relate costs of implementation and maintenance of eMMS to incremental benefits in terms of reduced ADEs and their associated costs. We modelled the entire medication error process from prescription to the occurrence of errors and harm to patients (figure 1). Importantly, we were able to populate our model primarily with data about the costs and effectiveness of the eMMS from our hospital site. This has been a limitation of some previous cost-effectiveness studies, which have had to heavily rely upon either cost and/or effectiveness data from the literature to support their analyses.,,

A Canadian cost-effectiveness study of an eMMS published in 2007 estimated the incremental cost per ADE prevented at US$12 700. The study, across a University Health Network of three hospitals in Toronto, involved an eMMS that comprised electronic prescribing and administration systems integrated to an existing CPOE, with no bar coding or automated dispensing, as was the situation at our study site. However two central factors limit the generalizability of the results. Firstly, their eMMS was developed “in house,” and, secondly, the study relied entirely on eMMS effectiveness data collected in the 1990s from a combination of US studies, conducted ∼10 years prior and pertinent to a different health care system.

More recently, a study in two hospitals in the Netherlands investigated the cost-effectiveness of eMMS (with limited decision-support), drawing upon statistical analysis of medication error and ADE data collected pre-eMMS and post-eMMS implementation. They reported the incremental cost of avoiding a medication error and avoiding a preventable ADE at €3.54 (US$4.8) and €322.70 (US$439), respectively. These ICERs were significantly lower than reported in the Canadian study by Wu et al. and most likely reflect improved eMMS performance as technology has advanced. Consistent with the recent Dutch results, we also reported a significantly greater effectiveness of eMMS to reduce ADEs than early reported eMMS studies. Our previous findings from a second hospital that implemented a different commercial eMMS (Cerner) at the same time as our MedChart study site, showed a very similar level of potential ADE reduction, strengthening the case that eMMS effectiveness has generally improved in the last 10 to 15 years.

Differences between the results of our study, which showed the eMMS paid for itself due to its ability to reduce ADEs and associated costs, and those from the Dutch study, may relate to a range of factors such as differences in the medication processes in the two countries, research methods (statistical analysis vs decision analytic modeling) and approach to cost calculation. In Australian hospitals, unlike in the Dutch hospitals, transcription of medications was minimal both pre-eMMS and post-eMMS and would not constitute a cost component if a similar costing method were used. Unlike the Dutch study, we employed an “incremental” principle to costing by including only “additional” staff time associated with the eMMS implementation and management (eg, cost of staff employed to train doctors and nurses in using the eMMS). Since no staff reduction occurred following the eMMS, it was assumed that other hospital staff costs would remain the same, although the time allocation across different tasks might have changed. The Dutch study found that medication processing time of doctors and pharmacy technicians decreased after the eMMS, whereas nurses and hospital pharmacists increased their time inputs. Estimation of the possible effect of eMMS on time allocation and change in productivity was outside the scope of our study. However, a previous Australian study of the impact of eMMS on changes in task time allocation found doctors and nurses experienced no significant change in proportion of time spent on medication-related tasks post-eMMS compared to staff on control wards with no eMMS. For simplicity, we assumed that staff time spent in attending eMMS training sessions was incorporated into their existing workloads as no new staff were employed to cover their time.

The baseline rates of potential ADEs reported by us (17%) and by the Dutch group (15.5%) were similar, yet we found a greater reduction in potential ADEs, which corresponded to post-eMMS ADE rates of (5% versus 7.3% in the Dutch study).,, Further, whereas Vermeulen et al. relied on statistical data analysis, we used decision analytic modeling of the entire medication error process, which necessitated assumptions about the proportion of intercepted errors and probability of nonintercepted errors resulting in actual ADEs. Despite our inability to make direct, detailed comparisons, what is clear is that both studies indicate that the cost-effectiveness of contemporary eMMS in different hospitals and in different health care systems represents good value for money. Both studies had methodological strengths in terms of drawing upon cost and effectiveness data from the study sites, which was not the case with past studies.

The limited decision-support embedded in the eMMS at the time of our study also suggests that further improvements in the effectiveness of the eMMS to reduce ADE rates can be expected as decision-support is added to the system, if it is well-designed and targeted. Beyond reducing ADEs, eMMS with decision-support can also be effective in driving more appropriate drug therapy, such as improvements in the rates of venous thrombosis prophylaxis and appropriate antibiotic prescribing. Such effects should improve patient outcomes and long-term costs of care. However, monitoring and maintaining a safe and effective decision-support system is also likely to demand more hospital resources, consequently increasing the operating costs of an eMMS.

While our study had several strengths in comparison to the few previous cost-effectiveness analyses, there were also limitations. Our model proved to be sensitive to large variations in the parameters of the probability of intercepting an error and the probability of error causing harm, for which the range of estimates in the literature, is particularly wide. Presently, there are no good quality observational studies that have established, on an individual patient level, the relationship between prescribing errors and actual ADEs categorized by degree of severity, while also accounting for the proportion of intercepted errors. The paucity of such data limits reliability in cost-effectiveness estimates and highlights the need for intensive research in the area.

This is the first Australian cost-effectiveness study of an eMMS and one of very few full economic evaluations of eMMS available internationally. The results provide some confidence to policy-makers, consumers, and clinicians that the benefits, in terms of ADEs prevented, from these systems provide a sound return on the investments made. Commercial eMMS, such as the one studied, are increasingly being marketed and implemented on a global scale, thus, providing new opportunities for more meaningful cost-effectiveness comparisons across organizations and countries in the future.

Contributors

J.I.W. conceived of the study. J.I.W., L.L., and R.O.D. designed and conducted the eMMS effectiveness study, which provided the data for the C-E model. L.L. conducted medication error data analyses to support the C-E study. E.G., J.I.W., and L.L. developed the decision-tree model. E.G. conducted the economic analyses and modeling. N.G. provided further expert advice on health economic evaluation. K.R., D.R., and M.H. provided costing data and expert advice regarding the interpretation of these data. All authors contributed to the interpretation of findings. J.I.W. and E.G. jointly drafted the manuscript, and all authors contributed to the review of the manuscript and approved the final version. J.I.W. takes overall responsibility for the integrity of the study.

Funding

This work was supported by funding from the National Health and Medical Research Council Program grant 1054146.

Conflicting interests

None.

Click here for additional data file.

Click here for additional data file.