A cost-effectiveness analysis of two different antimicrobial stewardship programs.

There is a lack of formal economic analysis to assess the efficiency of antimicrobial stewardship programs. Herein, we conducted a cost-effectiveness study to assess two different strategies of Antimicrobial Stewardship Programs. A 30-day Markov model was developed to analyze how cost-effective was a Bundled Antimicrobial Stewardship implemented in a university hospital in Brazil. Clinical data derived from a historical cohort that compared two different strategies of antimicrobial stewardship programs and had 30-day mortality as main outcome. Selected costs included: workload, cost of defined daily doses, length of stay, laboratory and imaging resources used to diagnose infections. Data were analyzed by deterministic and probabilistic sensitivity analysis to assess model's robustness, tornado diagram and Cost-Effectiveness Acceptability Curve. Bundled Strategy was more expensive (Cost difference US$ 2119.70), however, it was more efficient (US$ 27,549.15 vs 29,011.46). Deterministic and probabilistic sensitivity analysis suggested that critical variables did not alter final Incremental Cost-Effectiveness Ratio. Bundled Strategy had higher probabilities of being cost-effective, which was endorsed by cost-effectiveness acceptability curve. As health systems claim for efficient technologies, this study conclude that Bundled Antimicrobial Stewardship Program was more cost-effective, which means that stewardship strategies with such characteristics would be of special interest in a societal and clinical perspective.

Introduction

Antimicrobial stewardship programs (ASP) promote adequate use of antimicrobial drug therapy (ADT) to infected patients. They are meant to reduce undesired events due to inappropriate use of antibiotics, which is known to lead to worse clinical and economic outcomes, such as development of resistant bacteria, hospitalizations and mortality, in addition to increased ADT-related expenditures.1, 2, 3, 4

Nowadays, due to increasing health care costs, cost-effectiveness analysis (CEA) are the watershed of many health systems, as they provide better planning, financial and human resources allocation.

ASP economic outcomes have been analyzed with cost-reduction studies and positive results due to decreased antibiotics consumption and lower hospital length of stay were observed. However, these results are controversial7, 8 and the final outcome may be subject to many confounders. Therefore, the incorporation of clinical data and sensitivity analysis are necessary to assess whether one intervention could lead to robust economic outcomes.

As a matter of fact, cost-reduction studies are not formal economic analysis and, to our knowledge, there is only one publication that investigated whether ASP are cost-effective. Furthermore, this single cost-effectiveness publication was not free of bias, as there were many theoretical assumptions when assigning clinical probabilities, leading to results that could be unclear to clinicians due to complex reporting and modeling methods. Moreover, the aforementioned CEA employed diverse research outcomes (i.e. risk of death), while costs were estimated from other health institutions and specific wards, such as critical care units.

Since 2001, some authors have been advocating that well designed investigations with economic outcomes are needed, especially on ASP.6, 10 In addition, one recent publication has suggested that different ASP strategies could lead to different clinical outcomes. In that study, an ASP with proactive characteristics lead to improved 30-day mortality results.

In this context, considering that international literature lacks direct comparison among ASP strategies, our hypothesis was that different ASP strategies could also lead to different economic outcomes. The objective of this research was to assess whether two different ASP strategies could lead to different performance results.

Methods

Ethics and reporting

The present study complies with Helsinki's Declaration and Local Bioethics Committee approved it. We followed the suggestions of a Panel of Experts to conduct adequate reporting.

Definitions of two different strategies: conventional and bundled ASP

This CEA compared two different modalities of ASP. We used a previous cohort study that evaluated how Conventional or Bundled ASP differed in terms of mortality and antibiotic doses consumption.

Conventional Strategy was defined as a simplified stewardship program, which included a clinical pharmacist screening for antimicrobial drug-related problem (ADRP), case discussions with infectious disease physicians (ID-MD) and telephone-based interventions.

On the other hand, the Bundled ASP had a more active design, which included: prospective auditing and local education/feedback about antimicrobial therapy prescription; microbiological data discussion with laboratory personnel to guide empirical or preemptive therapy; and face-to-face interventions to improve antimicrobial drug therapy.

Study perspective and other nation-related issues

The perspective of this study was a Southern Brazilian University Hospital, which is a 550-bed public and clinical reference institution with an average of 55–65% occupation rate. The Brazilian Health-System, namely Sistema Único de Saúde (SUS), is a primary care-centered system with universal access to all Brazilian citizens. More information regarding the aforementioned health system should be consulted in excellent reviews published elsewhere.

Costs and definitions

All costs were collected and analyzed as local currency (R$, Brazilian Reais) and converted to United States Dollars (US$). Exchange values were collected at and they were expressed as mean value from February to September 2013, so R$ 1.00 was equivalent to US$ 0.47.

There were four relevant costs included in this CEA, namely: (I) hospital length of stay/patient-day, (II) cost of defined daily doses (DDD)/patient, (III) resources to provide microbiological and imaging diagnosis of infections, and (IV) human resources workload per day. These variables were collected through institutional databases, such as medication purchasing receipts and data from hospital's Human Resources and Planning Department. Table 1 summarizes all costs accounted for in this study and supplementary material provides detailed information.

Table 1

Summary of all included costs.

Average per patient	Bundled ASP expenditures	Conventional ASP expenditures
	Cost (US$)	Cost (US$)
Length-of-stay per daya	1457.35	1457.35
Antimicrobial consumption	1875.77 ± 107.05	4614.77 ± 174.15
Imaging or lab resourcesb	89.04	91.98
Cost of workload per dayc	92.79	45.33

DDD, defined daily doses.

The aforementioned table just illustrates the average value of length of stay per patient and mathematical modeling included individual calculations,

Values considered in mathematical model comprised individual calculations of image or lab resources and the aforementioned values are only average costs per patient,

Workload was expressed in this table as total ASP hours spend by day.

Notes: See Supplementary Material for more details about cost calculation.

Hospital length of stay was defined as the average cost per patient/day admitted to intensive care units or general wards, which included water consumption, human resources, medical material costs, and other relevant costs, except cost-related to ADT (see supplementary material).

DDD is a validated tool to standardize the number of doses consumed from each medication, allowing comparison of drug consumption between different health settings. Therefore, DDD was collected according to the original method developed by the World Health Organization. DDD was calculated based on pharmacy dispensation registries. Each unit of DDD was multiplied by the cost of drug, so antimicrobial therapy was expressed as “cost-DDD per patient”.

Regarding the costs related to bacterial infections diagnosis, we defined all diagnostic criteria according to international guidelines.14, 15, 16, 17, 18, 19, 20, 21 Prevalence of infections and their respective topographies were collected from the same previous cohort study, while costs of antibiograms, cultures, and other laboratory and imaging methods were calculated by means of microcosting bottom-up method.

At last, cost of human workload per day was calculated by estimating the amount of time spent by each health care staff involved with ASP, whereby Bundled ASP accounted for a full-dedication clinical pharmacist resident and two partially dedicated ID physicians (one preceptor and one third-year post-graduate MD).

Clinical outcomes (effectiveness)

Transition probabilities among health states were obtained through effectiveness data. For both strategies, we compared 30-day mortality, which was expressed as either time-to-event data or point estimate value: Absolute Risk Rate (AR), Risk Difference (RD), and Number Needed to Treat (NNT) (Table 2).

Table 2

The base case: outcomes, costs per patient, CER, and ICER.

	Absolute Risk	Direct costs (average value)	CER	ICER
Conventional ASP	0.6209	US$ 18,013.22	US$ 29,011.46	US$ 19,287.54
Bundled ASP	0.7308	US$ 20,132.92	US$ 27,549.15
Conventional ASPa	0.6202 ± 0.08	US$ 18,021.21 ± 5.72	US$ 29,057.10	US$ 19,317.58
Bundled ASPa	0.7328 ± 0.11	US$ 20,196.37 ± 6.33	US$ 27,560.55

ASP, antimicrobial stewardship program; AR, Absolute Risk; CER, Cost-Effectiveness Rate; ICER, Incremental Cost-Effectiveness Ratio.

After 10,000 iterations.

Notes: CER represents the cost per patient that survives 30 days. ICER represents the cost per incremental patient that survives 30 days.

Survival probabilities through each day (out of 30-day mortality) were also obtained from the same previous cohort.

Data analysis

Data were analyzed by comparing both strategies in the base case, by reporting crude costs, clinical outcomes, and efficiency indicators such as: Cost-Effectiveness Rate (CER) and Incremental Cost-Effectiveness Ratio (ICER).

Thereafter, costs and effectiveness variables were modeled by using TreeAge Pro 2009 Suite Software (TreeAge Software Williamstown, MA, USA). A health-transition model was developed to simulate the probability of patients to make transitions between two health states (alive or death). We developed a Markov process in order to make the model sensitive to changes that might occur from one day to another. The aforementioned model consisted of 30 cycles aligned to 30-day mortality outcome. Each cycle-length corresponded to one day. Fig. 1 illustrates the model adopted in this study.

Fig. 1

Markov process.

A deterministic one-way sensitivity analysis was conducted to identify whether variables range could critically influence the final ICER. Because binary variables cannot be assessed within this method, this sensitivity analysis did not include clinical outcomes, but only costs.

Multivariate analysis was performed by 2nd order Monte Carlo technique that simulated 10,000 probabilistic iterations. This analysis tested whether large ranges and their 95% confidence intervals could disturb the final ICER.

All cost variables were analyzed as Gama distributions and hyperparameters were calibrated to a 95% probability interval. Beta distributions were applied for effectiveness probabilities.

Finally, a tornado diagram illustrated one-way analysis, while Cost-Effectiveness Scatterplot, Acceptability Curve of Cost-Effectiveness and ICE Scatterplot showed the results from probabilistic sensitivity analysis. Discounting was not applied in the model since the horizon corresponded to one month; however, half-cycle correction was applied for all probabilistic iterations.

Results

The base case

In a preliminary analysis, the Bundled ASP was more expensive than a conventional ASP (Table 2) and the difference of crude costs was US$ 2119.70/patient. Considering point estimate values, which are expressed as Absolute Risk, Bundled ASP was more effective and patients are more likely to survive (73.1% vs 62.1%, p < 0.05).

Nevertheless, by observing health economic indicators (CER) the base case suggests that Bundled Strategy was more cost-effective, which was also seen after 10,000 virtual simulations. Finally, ICER suggests that each averted death in 30 days costed US$ 19,287.54.

Deterministic sensitivity analysis

ICER did not change when confronting the aforementioned base case results against critical variables. Therefore, according to one-way deterministic analysis, previous ICER was robust (Fig. 2). We observed that length of stay cost, human resources, and antimicrobial consumption were the critical variables that had the greatest ICER range.

Fig. 2

Tornado diagram with critical variables. Notes: Vertical line represents final ICER. Critical variables are represented as horizontal bars. All variables crossed final ICER, which means that none of them were able to disturb the final result from base case. c_obs-otim_intern, length of stay cost; c_otim_equipe, Bundled Strategy human resources cost; c_obs_medct, antimicrobial cost in Conventional Strategy; c_ c_otim_medct antimicrobial cost in Bundled Strategy; c_obs_dx_inf, bacterial infections diagnostic cost in Conventional Strategy; c_otim_dx_inf, bacterial infections diagnostic cost in Bundled Strategy.

Probabilistic sensitivity analysis

Probabilistic sensitivity analysis showed that ICER was robust. Scatter Plot graph shows that Bundled Strategy stayed clinically superior and at similar costs compared to the other group (Fig. 3).

Fig. 3

Scatterplot with 10,000 virtual cohorts. Note: Each dot represents one virtual cohort, thus, Bundled Strategy is likely to be more effective (y axis) and at similar costs (x axis).

Finally, Cost-Effectiveness Acceptability Curve (CEAC) (Fig. 4) was used to assess whether Bundled ASP would be affordable in the perspective of the studied health institution.

Fig. 4

Cost-Effectiveness Acceptability Curve (CEAC) between Bundled and Conventional ASP.

Quadrants are read as “I” (upper right, the most effective and expensive), “II” (upper left, the less effective and the most expensive), “III” (lower left, the less effective and less expensive strategy) and “IV (lower right, the best scenario with higher probabilities of effectiveness at lower costs).

Diagonal line represents Willingness to Pay Function (WTP) and all virtual cohorts under this curve were considered affordable by the studied health institution.

“Component” represents a group of representative dots in CEAC. C1 and C2 represent that Bundled strategy is, respectively, dominant and under WTP function. C4 represents the probability (40%) that Bundled Strategy is more expensive and above WTP function. C3, C5 and C6 (less than 10% of all iterations) are less representative probabilities and account for less effective and more expensive percentages.

We observed that Bundled ASP has higher probability of being the most effective (C1 + C2 + C4 = 92.7%) and the most cost-effective strategy (C1 + C2 vs C4 = 52.6% vs 40.1%).

When ICER was set as Willingness-To-Pay function (diagonal line represents stakeholders’ cutoff), Acceptability Curve demonstrated higher probabilities of Bundled Strategy stay under stakeholders’ willingness-to-pay. These results corroborated with previous base case and deterministic analysis.

Discussion

CEA analysis is an interesting tool to improve health systems’ performance, so decision-makers can assess and incorporate one technology that is the most cost-effective. In our study, we found that Bundled ASP Strategy was effective, efficient and at affordable costs, even after 10,000 probabilistic iterations.

Relevant costs and effectiveness

Due to our close relation to Planning Department, it was possible to collect specific data and assemble them in our economic modeling. Microcosting bottom-up method required time and may not be feasible to other institutions, but still is a reliable way to collect costs.

Previous CEA that assessed the efficiency of one ASP used similar modeling and data analysis. The difference between this and past study is that previous publication included multiple setting clinical probabilities and costs. We also observed that some data derived from 1999, but manuscript was published in 2009.

Costs from other settings and inclusion of clinical outcomes from different institutions constitute the main drawback from previous publications. To avoid such bias, we believe that the present study has overcome these problems by considering expenditures and clinical data from a single institution.

Different ASP interventions were never compared regarding their efficiency, and international literature lacks direct comparison between ASP strategies.

One could notice that Bundled ASP resembles other published and practiced stewardships with multiple types of interventions and, therefore, at this viewpoint, external validity is one of the positive aspects of our clinical effectiveness probabilities.

Markov model and accuracy of this study

Instead of choosing a Decision Model Tree, which is commonly used to acute care conditions, Markov Model provides more accuracy for each of the 30-day probabilities. Markov Model could precisely estimate the probability of one patient to die or to survive each day: costs quantification is dependent on accumulative probabilities of surviving or dying. In other words, if one patient survives, there would be an accumulative cost in the next day; likewise, if one patient dies, it would probabilistically reduce cost-effectiveness ratio. Previous publications included a decision tree analysis, and due to the aforementioned reasons, they were poorly planned and could have methodological bias.

With respect to internal validity, patients should be clinically compared and primary site of infection may be insufficient to determine if groups are homogeneous. In a previous study, multivariate analysis determined that Charlson Comorbidity Index could independently predict mortality in patients assisted by ASP. The known association between Charlson Comorbidity Index and Mortality impacts previous study's list of assumptions, where it was stated that “patients who never received an active antimicrobial drug therapy would have twice as poor outcomes (die twice) as those receiving an active therapy”. In other words, when defining an economic model, we should attempt to choose evidence-based probabilities of mortality to accurate predict real cost-effectiveness of ASP.

Other relevant interpretations

CEAC analysis was never performed before in other ASP research. It is a comprehensive way to help those who are not familiar with economic assessments to understand CEA. Nevertheless, to conduct such analysis, we set how much stakeholders would need to pay to have one Bundled ASP, by considering its structure, clinical and economic outcomes. Herein, we suggested ICER as the Willingness to Pay value (US 19,000 dollars per patient who survived 30-day mortality outcome).

ICER was determined as cut off value, because direct or indirect ways of defining “Willingness to Pay” are not validated and are subject to criticism. In addition, clinicians and stakeholders may interpret it more properly if NNT is also incorporated.

Therefore, by considering ICER value and that Bundled ASP has a NNT = 9, we suggest the following interpretations:

for every nine patients treated, one will benefit (or achieve 30-day outcome);

for every US19,000 dollars invested, one will experience the benefit;

as NNT and ICER conceptions are similar, every nine patients treated will cost US19,000 dollars and one will have positive clinical outcome.

Limitations

Our study is not free of limitations. The present study used effectiveness probabilities from a historical cohort, which means that retrospective data collection should always be cautiously interpreted due to incomplete registries, censoring, and research blinding problems.

Moreover, as two different ASP strategies were rarely compared in literature, more studies should be performed to assess if previous epidemiological findings (base case) are reproducible. When interpreting our results, one should consider the population included in the previous study, which was based on general wards and intensive care units inpatients. Other health settings should adapt this information before extrapolating our results to different practice scenarios.

Although sensitivity analysis corroborated with base case, cost-effectiveness should always consider one perspective. Readers should be aware of our study's external validity, especially because it was conducted in a Latin American health service, and differences are notorious between health systems and clinical settings.

Finally, although we believe that relevant costs were included in our economic modeling, ICER and Willingness-to Pay are not interchangeable concepts. In the present study, we did so because base case was a retrospective study, so all expenditures were already paid by hospital's financial department. Therefore, we considered that all values under Willingness-to Pay function would be affordable.

Conclusion

Bundled ASP was more cost-effective and also associated with higher probabilities of clinical success and at reasonable implementation costs, while conventional ASP was a cheaper strategy but less efficient. Therefore, (I) stakeholders should be aware to our critical variables (length of stay and human resources costs), before implementing an ASP; (II) patients would receive all advantages from a healthcare program that promotes better outcomes; (III) economic assessments are still sparse in literature, so we wish our study could motivate other researchers to conduct other ASP formal economic analyses.

Finally, our results should be assessed in other settings and countries. Future stewardship researches should made efforts toward coupling well conducted clinical and epidemiological studies, as backbones of economic analysis, once there is a special interest on incorporating high performance technologies to health settings and systems.

Funding

The corresponding author was given a monthly scholarship from the Brazilian Ministry of Education (Residência Multiprofissional em Atenção Hospitalar, Ministério da Educação e Cultura).

Conflicts of interest

The authors declare no conflicts of interest.

Component	Quadrant	Incremental effectiveness	Incremental cost	ICER	# of dots	%
C1	IV	IE > 0	IC < 0	Superior	3414	34.14
C2	I	IE > 0	IC > 0	<ICER	1851	18.51
C3	III	IE < 0	IC < 0	>ICER	351	3.51
C4	I	IE > 0	IC > 0	>ICER	4013	40.13
C5	III	IE < 0	IC < 0	<ICER	222	2.22
C6	II	IE < 0	IC > 0	Inferior	149	1.49

Quadrants are read as “I” (upper right, the most effective and expensive), “II” (upper left, the less effective and the most expensive), “III” (lower left, the less effective and less expensive strategy) and “IV (lower right, the best scenario with higher probabilities of effectiveness at lower costs).

Diagonal line represents Willingness to Pay Function (WTP) and all virtual cohorts under this curve were considered affordable by the studied health institution.

“Component” represents a group of representative dots in CEAC. C1 and C2 represent that Bundled strategy is, respectively, dominant and under WTP function. C4 represents the probability (40%) that Bundled Strategy is more expensive and above WTP function. C3, C5 and C6 (less than 10% of all iterations) are less representative probabilities and account for less effective and more expensive percentages.