Potential harm caused by physicians' a-priori beliefs in the clinical effectiveness of hydroxychloroquine and its impact on clinical and economic outcome - A simulation approach.

BACKGROUND: Despite growing controversies around Hydroxychloroquine's effectiveness, the drug is still widely prescribed by clinicians to treat COVID19 patients. Therapeutic judgment under uncertainty and imperfect information may be influenced by personal preference, whereby individuals, to confirm a-priori beliefs, may propose drugs without knowing the clinical benefit. To estimate this disconnect between available evidence and prescribing behavior, we created a Bayesian model analyzing a-priori optimistic belief of physicians in Hydroxychloroquine's effectiveness.
METHODOLOGY: We created a Bayesian model to simulate the impact of different a-priori beliefs related to Hydroxychloroquine's effectiveness on clinical and economic outcome.
RESULTS: Our hypothetical results indicate no significant difference in treatment effect (combined survival benefit and harm) up to a presumed drug's effectiveness level of 20%, with younger individuals being negatively affected by the treatment (RR 0.82, 0.55-1.2; (0.95 (1.1) % expected adverse events versus 0.05 (0.98) % expected death prevented). Simulated cost data indicate overall hospital cost (medicine, hospital stay, complication) of 18.361,41€ per hospitalized patient receiving Hydroxychloroquine treatment.
CONCLUSION: Off-label use of Hydroxychloroquine needs a rational, objective and datadriven evaluation, as personal preferences may be flawed and cause harm to patients and to society.

Introduction

The use of drugs that have not received regulatory approval, known as off-label prescribing, has been widely practiced before, [1] and continued to be applied during the COVID-19 crisis [2]. In the absence of proven medical therapies, the standard of care to treat viral pneumonia in patients with COVID-19, consists of supportive treatment aimed at allowing the body to rest and focus its energy on fighting the disease itself [3]. Yet, when the standard of care (SOC) does not instantly translate into survival benefits, it is psychologically understandable that medical professionals want to trial new medical technology and treatments to help these vulnerable patients [2,4]. This impulse to hope that the adjunct use of readily available medicines may translate into clinical benefits has led to boost several existing drugs to be used beyond their original indication [5,6]. Such unsupported use of off-label drugs has raised major concerns about safety and effectiveness [1,4]. In fact, previous data showed that the majority of off-label drug use is limited or had no scientific support (73%); and was mainly based on personal preferences [7,8].

One of the drugs, which has been proposed off-label to treat COVID-19 patients, is Hydroxychloroquine (HQC). Hydroxychloroquine, an anti-malaria drug, has known benefits for other infectious and autoimmune indications, but also has a worrying side effect profile ranging from retinopathy to life-threatening cardiac arrhythmia, prevalent in the general population including children and young patients [[9], [10], [11]]. At the beginning of the COVID-19 crisis, we witnessed a surge in non-randomized trials demonstrating some clinical benefits related to HQC for severe acute respiratory syndrome coronavirus 2 (SARS-CoV) infected patients. However, the majority of these studies had severe limitations in the methodology, producing mainly controversial findings, with one large study even being retracted after publication due to data inconsistency [[12], [13], [14]]. Over the course of the following month several large randomized studies showed no benefit for patients exposed to HQC treatment [15,16]. Despite rising evidence against its use, HQC is still being prescribed by clinicians and promoted by politicians, leading to its continued, yet unsupported off-label use [17,18]. In the absence of an evidence-based, risk-benefit analysis conducted by relevant regulatory and scientific bodies, we need to be aware that a drug may benefit some patients, but can also expose them to unknown clinical harm, or be ineffective and hence cause waste in an already burdened healthcare system [2,19].

As such, the off-label use of a drug without evidence of its effectiveness has an enormous economic impact, and, may waste valuable resources urgently needed in other areas [20,21]. Such “allocative inefficacy” may distract resources away from more important priorities, for example, promising clinical research studies, increasing ICU bed capacity, augmenting ventilation support technology, among many other initiatives.

To quantify the gap between available evidence and personal preferences that may influence decisions, we estimated the impact of a-priori beliefs related to the effectiveness of Hydroxychloroquine on clinical and economic outcomes. We modeled different clinical scenarios, taking into account different stages of the disease manifestation as well as stratifying the patients according to different age and risk groups.

Methodology

We designed a Bayesian hierarchical model to simulate the clinical effect and the overall costs of HCQ, when assuming different a-priori beliefs about the effectiveness of the drug. We chose the Bayesian approach, as it allowed for the incorporation of the uncertainty related to the available knowledge through the specifications of prior distributions for all unknown parameters in the simulation model [22,23].

Model Structure and Input Parameter

Our simulation pathway started with the entire population being potentially exposed to SARS-CoV ( Fig. 1 ). After a positive COVID-19 test, the symptomatic patient received a course of HCQ treatment (10 days, 300-600 mg twice a day). Based on the effectiveness of the drug, which we defined as a reduced probability of transiting from a mild to a severe clinical state (requiring intensive care admission), the subjects recovered or transited with predefined probability distributions of progression to the next node, conditional to the former node. In our example, we used effectiveness levels of 5%, 10%, 20%, 50% decrease with the probability of transiting into a severe state. To account for different risk profiles, we stratified the cohort according to age (I 0–9; II 10–19; III 20–29; IV 30–39; V 40–49; VI 50–59; VII 60–69, VIII 70–79 IX 80–89; X 90+). Prior specification on parameter distributions can be found in Table 1 (clinical probabilities) and Table 2 (cost input parameter). For example, if an elderly patient (70 years) has a 7% baseline risk of being SARS-CoV infected, once infected there is a 30% probability of transiting into a severe state with a 12% risk of death thereafter. In Table 2, it can also be observed that the patient will occur additive costs along the pathway (hospitalization, complication, treatment costs, etc.), until death or discharge.

Fig. 1

Simulation Model.

The model simulates the pathway of a cohort exposed to the SARS COV-2, receiving a therapeutic course of Hydroxychloroquine once exposed to COVID-19.

Table 1

Input parameter: Transition probabilities (expressed as a natural central parameter, in absolute numbers) stratified by age group.

Probabilities (by age, in life years)	Distribution	0–9	10–19	20–29	30–39	40–49	50–59	60–69	70–79	80–89	>90
SARS COV Infection	Beta	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07	0.07
Mild presentation	Beta	0.01	0.01	0.04	0.06	0.10	0.15	0.12	0.13	0.12	0.04
Severe presentation	Beta	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30
Hospitalizations	Log Normal	0.24	0.24	0.24	0.25	0.27	0.31	0.51	0.70	0.70	0.70
Severe in-hospital disease progression	Log Normal	0.22	0.22	0.22	0.23	0.25	0.29	0.49	0.70	0.70	0.70
ICU after severe manifestation	Log Normal	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30	0.30
Death	Log Normal	0.00	0.00	0.00	0.00	0.00	0.01	0.05	0.12	0.16	0.13
Death after severe manifestation	Log Normal	0.00	0.00	0.00	0.01	0.03	0.07	0.27	0.70	0.93	0.78
Death after ICU	Log Normal	0.02	0.02	0.14	0.14	0.22	0.30	0.60	0.80	0.80	0.80
Adverse event	Log Normal	0.01	0.01	0.01	0.01	0.01	0.05	0.10	0.10	0.10	0.10

Table 2

Cost parameters. Cost data apply per event (e.g. predicted costs (€) per complication), with each cost block being accounted for (and added) in the individual pathway.

	Distribution	Costs (per event)
Major complications (€)	Log Normal	200.7
Hospitalizations (€)	Log Normal	987.0
Ward length of stay (days)	Log Normal	10
Severe disease (€)	Log Normal	987.0
Increase length of stay (days)	Log Normal	5
ICU after sever disease progression (€)	Log Normal	21,300.0
ICU length of stay (days)	Log Normal	5
Drug (€)	Log Normal	40.0
Workdays lost (€)	Log Normal	46.3
Absence from works (days)	Log Normal	2.9
Cost to receive intervention (€)	Log Normal	15.8

Transition probabilities, clinical outcomes, length of stay for the COVID-19 cohort were retrieved from the publicly available health database on COVID-19 [24]. This was complemented and cross-checked with a literature search and peer-reviewed published studies on COVID-19 cases [25,26]. Cost data were derived from literature research, and included direct costs such as treatment, hospital and ICU costs, human resources, minor and mayor complications, and indirect costs, such as loss of productivity [[27], [28], [29]]. For statistical reasoning, we followed international guidelines on conducting and reporting Bayesian statistics [30].

Uncertainty

Because of the novelty of this illness and the many unknown factors related to its diffusion and mortality, we had to express a certain degree of uncertainty in our parameters. We used a Gibbs sampling, a Markov Chain Monte Carlo algorithm, to generate a sequence of samples from our set of input variables.

Results

We simulated a hypothetical population of 10^ {6} people, having the same age distribution as that of the Italian population ( Fig. 2 ). A snapshot of the relative risk of death in severe ICU cases (posterior outcomes) of the estimations are presented in Fig. 3 . These data illustrate that, if we prospectively predict that the drug has no or little effect on disease progression, this would cause harm to the population. Only if there is an a-priori believe that the drug can improve clinical progression by more than 20%, we may see a potential clinical benefit (Relative Risk (RR) 0.81, Confidence Interval (95% CI) 0.76–0.87) of the drug on survival, but only in the elderly population (60 years and above). We further compared the expected side effects with the survival and determined the cut-off point, (the balance between severe side effects related of the drug and increased patient survival rate) to be at an effectiveness level slightly below 20%. At a presumed effectiveness level of 20%, we saw 266 patients surviving (ICU and hospital), while 336 patients experiencing severe side effects ( Table 3 ).

Fig. 2

Population Pyramid.

Population pyramid of the Italian sample population. Age distribution in both standard treatment and intervention [source ISTAT 2015].

Fig. 3

Distribution of the relative risk effect across age groups.

Posterior distribution of the relative risk effect across a) age groups, b) general population stratified by a-priori beliefs.

Table 3

Side effects and hospital outcomes related to different a-priori beliefs on effectiveness (throughout all age groups, in absolute and relative numbers).

	Believed Effectiveness 5%				Believed Effectiveness 10%				Believed Effectiveness 20%				Believed Effectiveness 50%
	HQC	SD	SOC	SD	HQC	SD	SOC	SD	HQC	SD	SOC	SD	HQC	SD	SOC	SD
Adverse effect	340 (5.6%)	127 (1.1%)			337 (5.6%)	121 (1.1%)			336 (5.6%)	123 (1.1%)			333 (5.6%)	122 (1.1%)
Death in ICU	167 (2.7%)	66 (0.6%)	175 (2.9%)	68 (0.6%)	159 (2.6%)	61(0.6%)	175 (2.9%)	67 (0.6%)	140 (2.3%)	54 (0.5%)	173 (2.8%)	65 (0.6%)	90 (1.5%)	36 (0.4%)	172 (2.9%)	67 (0.6%)
ICU	393 (6.5%)	119 (0.7%)	413 (6.9%)	124 (0.7%)	372 (6.3%)	109 (0.7%)	411 (6.9%)	119 (0.7%)	33 (5.6%)4	98 (0.6%)	411 (6.9%)	120 (0.7%)	214 (3.6%)	65 (0.4%)	405 (6.9%)	120 (0.7%)
Death in Hospital (excl. ICU)	1.181 (22.2%)	523 (6.1%)	1.239 (23.3%)	548 (6.4%)	1.096 (21%)	469 (5.9%)	1.210 (23.2%)	517 (6.5%)	989 (18.7%)	445 (5.5%)	1.222 (23.1%)	547 (6.8%)	642 (12.3%)	284 (3.6%)	1.218 (23.4%)	534 (6.6%)
Major complication in Hospital	2.494 (41.3%)	811 (5%)	2.617 (43.4%)	847 (5.2%)	2.335 (39.2%)	711 (4.8%)	2.580 (43.3%)	783 (5.3%)	2.100 (35%)	667 (4.5%)	2.591 (43.2%)	820 (5.5%)	1.355 (22.9	433 (3%)	2.567 (43.4%)	812 (5.3%)
Hospitalization	5.985 (100%)	1.638	5.987 (100%)	1.637	5.909 (100%)	1.493	5.911 (100%)	1.493	5.955 (100%)	1.574	5.958 (100%)	1.579	5.870 (100%)	1.568	5.870 (100%)	1.564

HQC Hydroxychloroquine; SD Standard Deviation; SOC Standard of Care;

Direct outcomes – Disease progression and ICU admission

In the general population, there is a severe disease progression of 41% (SD 5.1%) at a presumed drug effectiveness (pEff of 5%), 39% (SD 4.8%) (pEff 10%), 35% (SD 4.5%) (pEff 20%) and 23% (SD 3.0%), compared to 43% (SD 5.5%) when no drug was given, with avoided ICU admission, ranging from 0.33% (SD 1.1%) (pEff 5%), 0.63% (SD 1%) (pEff 10%), 1% (SD 1.2%) (pEff 20%) to 3% (SD 0.92%) (pEff 50%).

Indirect effect – Survival

Fig. 4 shows the expected risk reduction at an effectiveness level of 20%. When patients are stratified by age, we see an improvement in combined clinical outcomes only for a presumed effectiveness level above 20%. RR 0.81, 0.76–0.87 for a population older than 60 years.

Fig. 4

Outcome parameter at a presumed effectiveness of 20%.

Results of 1500 simulations based on a starting population of 1 million.

A) Box plot of absolute change in hospital death following intervention, stratified by age; each box represent the distribution of expected excess deaths prevented by the intervention in that age class; B) Box plot of overall in hospital deaths as percentage of total hospitalized; C) Box plot of absolute adverse event due to intervention, stratified by age; each box represent the distribution of expected adverse events due to intervention in that age class; D) Box plot of adverse events due to intervention, percentage of total hospitalized.

Looking at lower presumed effectiveness levels (< 10%), we see no benefits in prescribing Hydroxychloroquine on ICU admissions and survival, while still causing severe side effects (SAE 5.7% SD 1,1% cases / vs 1.1% SD 1.4% death prevented or treated population). Specifically for the younger generation (< 20 years old) we see an unfavorable risk versus benefit profile (0.95 (1.1) % expected adverse events versus 0.05 (0.98) % expected death prevented).

Costs impact

Our cost data indicate drug treatment related costs of €37.83 per patient receiving a course of 10 days HCQ (300-600 mg, twice a day), and incremental hospital costs of €18.361,41 per hospitalized patient (including hospital and ICU stay (LOS) and minor and major complication; treatment costs are included in the LOS and complication costs). Up to April 20, 2020 we knew of nearly 2.000 infected and hospitalized subjects [31] which translates into overall hospital expenditures of €457.000.000 related to confirmed and admitted COVID-19 patients in Italian hospitals, and €1100.000 directly related to the drug use and related complication.

Discussion

The safety and effectiveness profile for the off-label use of Hydroxychloroquine in COVID-19 patients is controversial [[32], [33], [34]]. Despite all the uncertainty related to the effectiveness of the drug and potential harm [18], we witnessed a disproportionate belief in the drugs, evidenced in the fact that Hydroxychloroquine was still being used after six month of evidence accumulated on its lack of effectiveness for COVID-19 patients [17,35].

Applying a Bayesian approach, we prospectively quantified that up to an effectiveness threshold of 20%, the drug had no benefit to the general population, and may cause unnecessary waste in the system. In particular, in young patients where case fatality rate is low, as in patient <20 years, we estimated harm caused by Hydroxychloroquine at 0.95 (1.1) % expected side effects versus 0.05 (0.98) % expected death prevented. By simulating different predictive clinical scenarios, taking into account personal preferences or cognitive bias, these results indicate that the belief in the effectiveness of the off-label use for Hydroxychloroquine was vastly overrated. While our data were predictive on the basis of hypothetical data, over the course of the COVID-19 pandemic, we saw emerging data confirm the ambiguity of the effectiveness of the drug, with some patient populations being exposed to ineffective treatment and even worse to potential harm [32,36,37]. One of the first studies, Tang et al. reported no superiority in conversion rates of SARS-CoV-2 (viral clearance) related to the treatment with Hydroxychloroquine, while side effects were apparent in up to 30% of the study population [38]. This was further evidenced by a recent large randomized study (UK Recovery) confirming the signal towards higher mortality in the Hydroxychloroquine arm, and this study arm was stopped preliminary due to safety and ethics concern [15]. Further study results from other randomized trials such as REMACAP or SOLIDARITY [16,39] are awaited. While such scientifically validated data can inform and update clinicians more accurately and provide evidence-based recommendations on the use of Hydroxychloroquine, our predictive model may aid decision making in the absence of clinical trial data.

Furthermore, our data demonstrated the actual and potential enormous economic impact related to the off-label use of the drug. In our example, we calculated hospital costs of € 18.361,41 and societal costs related to the COVID-19 hospitalization of € 45.000.000 for the Italian healthcare system. Such costs were mainly attributed to the prolonged length of stay (LOS), and the increased rate of complications in the treatment pathways related to the drug's side effects within all age groups. Considering an environment where we act with limited resources (human and financial), there is an increasing need for scientific assessments (costs and effects) prior to adopting new or widely prescribing repurposed drugs. Hospitals should be encouraged to conduct prospective analysis, combining clinical and economic costs, to define and constantly update if resources are effectively allocated to the most promising treatments.

Our data may be used to inform on the impact of our subjective beliefs on prescribing potentially harmful drugs. Governmental and scientific bodies can use such data to design educational pathways to update on statistic literacy and increase awareness on the gap between beliefs and evidence, in order to improve the quality of drug prescribing. This may lead to more informed decision making and positively impact research priorities, lower ICU admissions, and lower waste in an already overloaded healthcare system.

The results warrant further research and we are initiating follow-up studies to explore if increased awareness on our internal bias will directly translate into better prescribing in our clinical routine.

Limitations

Simulation models are always simplifications of the real systems being analyzed. Furthermore, we cannot forecast the future with precision, but only evaluate the situation at a point in time. The COVID-19 crisis has dynamics that alter our model input parameter continuously. Hence, this model can be used as a guidance, but our results need further confirmation within ongoing randomized trials.

We did not quantify or include data on indirect costs, such as costs of demoralizing staff due to proposing unreliable research activities, opportunity costs of non-conducting controlled research activities (defined as the lost opportunity to allocate scarce resources to activities which yield a better outcome in terms of effect and costs), or loss related to taking away the freedom of physicians to explore and potentially develop new solutions. However, we stressed the importance that solid evidence is needed before advising on the use of any drug.

Finally, we stratified our risk groups according to age, and did not include comorbidities in our model. At the time of modeling, no sufficient data were available on comorbidities and to avoid including more uncertainty we focused on age as a risk parameter.

Conclusion

Off-label use of Hydroxychloroquine needs a rational, objective and data-driven evaluation, as personal preferences may be flawed and cause harm to patients. Our Bayesian simulation highlights the vulnerability of a-priori beliefs of physicians prescribing off-label drugs, and its negative impact on clinical and economic outcomes. These data may be used to create awareness around biased preferences and may inform educational programs on statistical literacy for prescribing clinicians.

Statements and Declarations

Transparency declaration – the lead author Prof. M Cecconi affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Contribution: All authors have contributed to the manuscript. CE, MC and AM developed the initial idea for the article, CE performed the literature research and wrote the article, FC conducted the modeling and the statistical analysis; SE revised content and contributed to the ethics debate within the article, AM was substantially involved in designing the idea, and reviewing the script.

Competing interest: All authors declare no support from any organization for the submitted work; no financial relationships with any organizations that might have an interest in the submitted work in the previous three years, no other relationships or activities that could appear to have influenced the submitted work.

Details of funding: No funding was requested.

Data sharing statement: All authors agree to share data.

Checklist: The SQUIRD Checklist for Quality improvement reporting was used for this analysis.

Author contribution

All authors have contributed to the manuscript.