Exploring the misalignment on the value of further research between payers and manufacturers. A case study on a novel total artificial heart.

Payers and manufacturers can disagree on the appropriate level of evidence that is required for new medical devices, resulting in high societal costs due to decisions taken with sub-optimal information. A cost-effectiveness model of a hypothetical total artificial heart was built using data from the literature and the (simulated) results of a pivotal study. The expected value of perfect information (EVPI) was calculated from both the payer and manufacturer perspectives, using net monetary benefit and the company's return on investment respectively. A function was also defined, linking effectiveness to market shares. Additional constraints such as a minimum clinical difference or maximum budget impact were introduced into the company's decisions to simulate additional barriers to adoption. The difference in the EVPI between manufacturers and payers varied greatly depending on the underlying decision rules and constraints. The manufacturer's EVPI depends on the probability of being reimbursed, the uncertainty on the (cost-)effectiveness of the technology, as well as other parameters relating to initial investments, operating costs and market dynamics. The use of Value of information for both perspectives can outline potential misalignments and can be particularly useful to inform early dialogs between manufacturers and payers, or negotiations on conditional reimbursement schemes.

INTRODUCTION

Uncertainty on the true performance of medical devices (MDs) often exists in their early stages of diffusion (Girling et al., 2010; Polisena et al., 2018; Rothery et al., 2017). The underlying reasons for such uncertainty are various and include less stringent regulatory requirements for market access (Sedrakyan et al., 2016; Sorenson & Drummond, 2014); difficulties with generating solid evidence from RCTs (Neugebauer et al., 2017), and the fact that specific characteristics of MDs, such as their organizational impact, the existence of a learning effect and iterative product modifications make the results of available studies poorly generalizable in other settings or the real‐world (Sedrakyan et al., 2016; Tarricone et al., 2020). Consequently, healthcare payers' decisions on whether an MD should be adopted and reimbursed with public funds are often hampered by partial and not fit‐for‐purpose evidence. The consequences of uncertainty over decision‐making may carry substantial costs. Indeed, improper, or insufficient evidence at the time of making adoption and reimbursement decisions may result in unnecessary delays in providing access to valuable technologies or may even cause direct harm to patients should the technology fail to confirm its value after early adoption (Chalkidou et al., 2008; Fenwick et al., 2020). Therefore, before deciding on the reimbursement of a technology, payers and Health Technology Assessment (HTA) bodies are called to weigh the benefits of immediate, unconditional adoption against the potential benefits of reducing the existing uncertainty through conducting, or requiring further research (Fenwick et al., 2008; McKenna & Claxton, 2011).

On their side, manufacturers face the uncertainty that, even after obtaining market access, decision‐makers, such as payers or healthcare providers may hamper or even prevent adoption and uptake of their technology if the evidence around it is not considered robust enough. In addition, at different decisional levels, both the assessment criteria and the decision rules may vary. Therefore, what constitutes sufficient evidence for some, may be less relevant for others. For example, payers and HTA bodies may rely more on the cost‐effectiveness of the technology, using the wider perspective of the healthcare system and considering long‐term costs and consequences of adopting it. Conversely, providers may be more interested in knowing whether a technology reaches a minimum clinical difference (MCD) and whether the short time budget impact of adopting it remains under a certain threshold, at least in the short term. Therefore, soon after market approval, manufacturers usually start building their value dossier, planning what evidence should be generated to maximize the company's objectives. While deciding their post‐market plan for evidence generation, manufacturers need to weigh the cost of conducting further studies with their expected returns. In doing so, they need to estimate the expected payoffs with and without further evidence according to a specified utility function. Generally speaking, the value of conducting further research for manufacturers will be affected both by the behaviors of healthcare decision‐makers following a reduction in the uncertainty, and other aspects of their utility function such as the company operating margin, or the market dynamics, like for example, the time before manufacturers expect that their technology will become obsolete.

Compared to payers, manufacturers pursue different objectives and may have different perceptions of what evidence constitutes good value for money. Therefore, it may happen that the evidence they plan to generate does not align with the evidence required by payers and HTA bodies. To reduce existing misalignments in what evidence should be generated, early dialogs (EDs) between manufacturers and payers have been increasingly encouraged and used for pharmaceuticals (Ciani & Jommi, 2014; Tafuri et al., 2018). For MDs, their use is still relatively limited, but previous research has shown an interest in this policy tool from all parties involved in evidence generation (Blankart et al., 2021; Schnell‐Inderst et al., 2015). Also, in Europe, different countries provide early scientific advice to manufacturers, and the European Network of Health Technology Assessment has piloted several rounds of EDs specifically designed for MDs (EunetHTA, 2021). Briefly, in the EunetHTA EDs manufacturers can ask scientific advice to multiple HTA agencies in Europe about their intended evidence generation plan to ensure that this is relevant for allowing informed adoption and reimbursement decisions in each jurisdiction.

In addition, interactions between payers and manufacturers on evidence generation can occur also later in the product lifecycle. This is the case of conditional reimbursement schemes that are available as policy options to payers, such as coverage with evidence development (CED) schemes. Coverage with evidence development schemes imply a program of data collection for devices, which have been judged to be potentially valuable for the patients, but for which important evidence gaps exist to prove their true clinical benefit or cost‐effectiveness (Federici et al., 2021). In these schemes, a widely recognized challenge for implementation is the agreement among stakeholders (Garrison et al., 2013; Reckers‐Droog et al., 2020). In addition, in cases where manufacturers are responsible for data collection, the existence of misalignments in the perceived value of research may represent a disincentive for manufacturers to collect the evidence that is required by public decision‐makers (Federici et al., 2021).

Both EDs and CED schemes require that all parties involved have a clear understanding of the potential returns of further investments in research. Value of information (VOI) analysis has been proposed as a formal framework to estimate the value of reducing all or part of the existing uncertainty over a health technology (Fenwick et al., 2020; Rothery et al., 2017). A convenient aspect of VOI is that it converts the reduction in uncertainty that is achievable by conducting new studies directly into gains expressed in terms of the decision‐maker utility function. For example, when applied to the probabilistic output of a cost‐effectiveness analysis, VOI analysis can estimate what would be the effect of further research on the incremental net monetary benefit (INMB) of a technology compared to existing alternatives, thus providing a consistent framework to simultaneously address adoption and research decisions (McKenna & Claxton, 2011). To date, most of the theoretical and applied work on VOI has adopted the perspective of healthcare payers and used population net health benefit as a utility function over which VOI was calculated. Nonetheless, the same perspective and normative framework were also maintained when VOI was used to inform manufacturers' decisions (Ijzerman et al., 2017; Markiewicz et al., 2014; Miquel‐cases et al., 2016; Retèl et al., 2013; Vallejo‐Torres et al., 2008) However, this approach may lead to sub‐optimal decisions for technology developers given the inherent differences in the objectives between these two groups. To allow optimal decisions, the utility function used to estimate VOI for developers should reflect their objectives rather than the healthcare objectives of payers based on net population health gains. For example, Willan (2008) estimates an optimal sample size for a trial from a developer's perspective by relating the VOI to the increase in expected profits through changes in the likelihood of regulatory approval. Similarly, Breeze and Brennan (Breeze & Brennan, 2015) linked the VOI to the negotiated price during a reimbursement decision and the developers' expected financial return on investments, assuming that pricing would be determined using a value‐based pricing criterion (Sussex et al., 2013). This study contributes to the existing literature in two ways. First, we extended the use of VOI to the manufacturers' decision problem, using the net present value (NPV) of the operating income as a utility function and incorporating the effects on the company's decision processes of the possible reactions of healthcare decision‐makers following a reduction in uncertainty. Second, we investigated what factors could create a misalignment between payers and manufacturers on the value of further research all of which may lead to inefficient evidence generation plans, and ultimately delayed access to patients of valuable technologies.

In this study, the value of research under the manufacturer and the healthcare payer perspectives has been estimated within the limited standpoint of a single jurisdiction, or a single market. Nonetheless, clinical and economic evidence has the non‐rival characteristic of a public good (Eckermann & Willan, 2009) and is likely to have spill over effects also in other jurisdictions, depending on its degree of generalizability. The aspect of the potential spill over effects of evidence on other markets/jurisdictions and their consequences on the global VOI is addressed in more detail in the discussion. The following sections are organized as follows. First, the decision problem for manufacturers is presented. Second, a case study on a total artificial heart is used to estimate the VOI for manufacturers and to explore the existence of misalignments between payers and manufacturers concerning the value of further research. Lastly, the implications of such misalignments and the use of VOI for manufacturers and payers are explored in the discussion.

METHODS

Manufacturer decision problem and value of information

Companies plan their marketing strategy globally. However, since adoption and reimbursement decision are taken at the national or even regional level, it is natural to consider companies' strategy as a sequence of decisions to be taken in individual markets/jurisdictions.

In each jurisdiction, while payers' decisions on reimbursement normally assume perfect implementation, manufacturers' decisions to enter a market need to consider all aspects that can potentially affect adoption in clinical practice, market shares, and eventually their capability to generate profits. For example, in countries where coverage and reimbursement of technology are decided at a national level through a formal HTA assessment, companies will first have to estimate what is the likelihood that the actual level of evidence for their device is enough to attain a positive reimbursement decision and the extent to which such decision may affect their market size (e.g., if the use of the technology is restricted to a specific sub‐population, or approved only in the context of research). This is the case in many European countries like France or the Netherlands, where a positive national appraisal is required for the device to be reimbursed within the healthcare system, and conditional reimbursement schemes are also possible in addition to adopt/reject decisions. However, even if a device obtains a positive reimbursement decision, it is not certain that it will be adopted into clinical practice, as uptake may depend on other decision‐makers such as practitioners. As argued by Fenwick et al. (2008), the reasons for non or partial adoption following a positive reimbursement decision by the national authority are various and include practitioners facing different incentives or possessing different perspectives on cost‐effectiveness, the existence of imperfect clinical governance, and an asymmetry of information between policymakers and local decision‐makers. For example, while healthcare payers may make reimbursement decisions by choosing the intervention that provides the highest expected net benefit, providers may be more focused on the impact of the technology on their hospital budget, or the (short‐term) health gains for the patients during their first hospitalization event. Behaviors of practitioners according to these criteria are likely to be affected by the uncertainty about the true performance of the device, and therefore, reducing uncertainty through generating new evidence will also affect the companies' expectations on uptake and market shares.

Since the objectives of manufacturers are different from the ones of payers, the expected VOI can be estimated in a similar way to healthcare payers but using a different set of utility functions and decision rules. For example, companies may decide to enter a market if the expectation of the NPV of taking such a decision is positive or higher than a certain threshold. This expectation in turn depends on the initial investment (e.g., the investment required to submit for approval to a national/regional authority), the likelihood of approval, the size of the market, the uncertain level of market shares and revenues if the device gets to the market, or the level of additional direct and indirect operating costs. In addition, to account for other decision levels (e.g., providers) manufacturers may also consider other constraints in addition to a favorable decision from payers. For example, they may consider that providers would not be willing to purchase the device if the costs of the procedure and related hospitalization period were too high, or the performance of the device was below a certain MCD.

Case study on total artificial heart

Heart transplantation (HTx) is the optimal treatment for patients with advanced heart failure (Arabía, 2020; Copeland et al., 2004). Unfortunately, donors' heart availability is not sufficient to satisfy the demand 3antations thus making it necessary to use ventricular assist devices to bridge patients until a new heart becomes available. In most cases, left ventricular assist devices (LVADs) have demonstrated good survival outcomes. However, approximately 40% of patients receiving LVAD suffer from right ventricular (RV) failure (Arabía, 2020; Kormos et al., 2019; Melton et al., 2019) leading to worse quality of life and increased mortality. There exist a variety of devices for temporary RV support, however, worse long‐term outcomes of patients surviving RV failure suggest the need for long‐term biventricular support. The use of Biventricular assist devices, which provide support to both the left and right ventricles of the heart, has shown good outcomes, but for a limited patient population there exists a benefit from totally removing the sick heart and implanting a total artificial heart (TAH; Arabía, 2020). To date, there existed only one TAH licensed as a bridge to transplant (BTT) device, the SynCardia TAH (SynCardia Systems, LLC Tucson, AZ, USA), which has evolved from the Jarvik 7, the first TAH model to be implanted in a human patient in 1981 (DeVries et al., 1984). Despite being a life‐saving device for patients awaiting in the transplant list, survival and complications rates and quality of life outcomes remain relatively poor (Arabía et al., 2018; Vierecke et al., 2019) so that in the quest to improve patients outcomes, other TAH devices are under development or have been recently licensed for BTT therapy. Evidence on ventricular assist devices is generally poor at market launch with very few randomized clinical studies comparing device types and therapeutic strategies. Most of the evidence comes from international registries, such as the INTERMACS, EUROMACS registries (de By et al., 2015; Kormos et al., 2019), or individual cohort studies. For TAH, cost‐effectiveness evaluations are also lacking, perhaps due to the absence of viable therapeutic alternatives, and a higher weight given to the device's life‐saving characteristic over its value for money. Notwithstanding, with the advent of novel models from different companies, more evidence will be needed to inform decisions about which device should be reimbursed, purchased, and implanted. Consequently, device manufacturers will also have an interest in understanding which type of evidence provides the best value for money when building their value proposition.

In the present analysis, we used Italy as the reference jurisdiction for the case study. From the payer side, we adopted the perspective of the national healthcare system considering only direct healthcare‐related costs and consequences. From the manufacturer's side, we adopted the perspective of an individual company planning to submit for reimbursement to the national authority. The utility functions and decision rules for both manufacturers and payers are specified below in Section 2.5.

Cost‐effectiveness model

A discrete‐time semi‐Markov model was adopted to estimate the cost‐effectiveness of two different models of a TAH, the Syncardia TAH and a hypothetical novel TAH. The model was assumed to have three states, alive with TAH support (TAH state), HTx or dead. Patients in the TAH state can either survive until transplant or die with probability that is a function of the time from implant. Patients who receive a donor's heart move to the HTx state and remain in such state until death. The probability of death λ(t) from the HTx state was also modeled as a function of time from transplantation. A graphical representation of the model is reported in Figure 1. Since the probability of transitions across states depend on the time of entrance in each state, following Sharples et al. (2006), the model was decomposed into two simple sub‐processes, one representing time pre‐transplantation and the second representing time post‐transplantation. This approach allows to overcome the memoryless characteristic of Markov models and to correctly estimate transitions of patients according to their time‐dependent probabilities.

FIGURE 1

Graphical representation of the cost effectiveness model. TAH, Alive with total artificial heart implanted; heart transplantation (HTx), alive after HTx

After implant of a TAH the survival of patients was modeled considering death and transplantation as competing events. For the Syncardia TAH, survival data were taken from a retrospective observational study reporting data on 450 patients implanted with a TAH in the INTERMACS registry. Kaplan Maier curves were digitized, and individual patient level data were reproduced using the algorithm proposed by Guyot et al. (2012). The study reported survival curves with censoring at transplantation, however, since no other censoring occurred due to lost to follow up or other reasons, with the only exception of 6 patients still alive at the end of the observation period, it was possible to replicate full information on the survival of patients for both events of death and transplantation. The time to any event, was modeled as a Generalized Gamma distribution based on visual inspection and the Akaike or Bayesian Information Criterion. Then, conditional on experiencing an event, patients were distributed among death or transplantation states based on the estimated number of events in each cycle. The probability of transition to either transplantation or death was calculated as (1−γ(t))*ρ(t) and (1−γ(t))*(1−ρ(t)), where γ(t) is the survival function for any event and ρ(t) is the proportion of deaths conditional on leaving the TAH state.

The cumulative hazard of death rapidly decreases in the first 3 months and then remain almost constant afterward (Arabía et al., 2018). Therefore, the proportions of deaths and transplantations were estimated individually for the first 3 months and then assumed constant thereafter. Estimated and reported cumulative incidence functions from competing risk analysis were then compared to check the validity of the approach resulting in nearly identical curves. Four major complications were considered and defined according to the INTERMACS definitions of adverse events for mechanical circulatory support (Lenderman, 2021): ischemic strokes, major bleedings, major infections and major device malfunctions. Strokes were further classified into disabling (moderate or severe disability) and non‐disabling strokes. Patients surviving to a disabling stroke were assumed to have lower quality of life after transplantation compared to their non‐stroke peers.

Expert clinicians agree in that the novel TAHs under development will improve the quality of life of patients. Such improvement is expected to be realized through a lower rate of non‐fatal adverse events, higher mobility and discharge rates while on support, as well as improved hemocompatibility and flow regulation (Melton et al., 2019). In the absence of evidence on the quality of life of patients on support with Syncardia (Canada et al., 2019; Kormos et al., 2019; Streur et al., 2020) we presumed that data on quality of life was available from the pivotal study on the novel TAH and assumed that the differences in quality of life between TAH could be elicited through expert opinion.

Disease specific mortality after HTx remain high in the first 3 months and then become almost zero (David et al., 2020; Kirsch et al., 2013; Levin et al., 2016; Shah et al., 2016). Similarly to survival curves with the TAH in place parametric extrapolation was done from digitized data of the Kaplan‐Meier curves from David et al. (2020) who provide mortality data with the longest follow up of 12 years. Italian life tables were then used to incorporate all‐cause mortality in the cohort assuming an average age of 50 years (Arabía et al., 2018; Levin et al., 2016).

The cost of the device for the Syncardia TAH was retrieved using the Italian device national classification and the public expenditure data from the Ministry of Health. Patients were assumed to either remain hospitalized until death or HTx occurred or to be discharged after an average stay in the cardiac ward of 50 days (Arabía et al., 2018). Costs of complications were valued using Italian Diagnosis Related Group (DRG) tariffs.

Cost data of patients receiving a TAH were not available in Italy or elsewhere. Therefore, we used data from Sharples et al. who did collect micro‐costing evidence from 70 patients receiving a LVAD in the UK (Sharples et al., 2006). With the only exception of the cost of the devices, we assumed that the cost of the index procedure including implant operating theater costs, initial Intensive Care Unit stay, maintenance drugs and maintenance tests, were similar among VAD and TAHs and had similar standard deviations. Similarly, monthly costs after HTx were assumed equal to patients undergoing LVAD. All cost figures from Sharples et al. have been inflated to 2019 and converted into EURO using the average 2019 conversion rate between Pound sterling and Euro from of 0.878 (Bank, 2021). The model estimated the lifetime costs and consequences of patients until death.

Simulation of data for the novel TAH

A hypothetical comparison was created by micro‐simulating data based on the expected improvements of a novel TAH under design compared to the Syncardia TAH. We presumed that 24 months data from a single arm pivotal study with 35 patients were available and that the study collected information on main clinical endpoint and quality of life but not resource use and costs. The simulated evidence for the study is consistent with the data that is expected for a new MD in this field. For example, the ongoing clinical study to support the Humanitarian Device Exemption application of the Syncardia TAH as destination therapy for patients who are not eligible for transplant (ClinicalTrials.gov identifier: NCT02232659) has an open‐label, single arm design aiming to measure safety and clinical benefit in an expected number of 38 enrolled patients. Similarly, the pivotal study to support CE mark application for the CARMAT TAH (ClinicalTrials.gov Identifier: NCT02962973) envisioned the enrollment of 20 patients in a single‐arm study with a follow up of up to 2 years, measuring safety, and effectiveness endpoints, including quality of life measures.

We assumed that patients would have similar times on support before experiencing either death or HTx, but that the risk ratio of death between the novel and Syncadia TAH at each time point was lower by 0.25. Therefore, time to any event was simulated from a Generalized Gamma distribution and then competing events were classified as death or HTx based on the calculated proportions. Patients still on support at 24 months were considered as censored. Similarly, a 0.15 reduction in the incidence of any of the 4 considered major adverse events was simulated. The number of adverse events in the pivotal study was calculated by multiplying the number of enrolled patients for the adjusted incidence and then by rounding the results to the nearest integer. These numbers were then used as inputs in the model to estimate the posterior probability distribution of adverse events. Procedure costs were considered equal to the Syncardia TAH except for the cost of the device. During support, monthly differences in costs were originated from the model based on differences in the number of discharged patients and the incidence rates of adverse events. Patients surviving to HTx were assumed to have same monthly costs with the two devices. Due to the simulated innovativeness of the device and the likely development costs, we presumed that the cost of the novel TAH would be set at €200,000.

The model estimated the costs and consequences for the whole patients' lifetime with future costs and consequences equally discounted at a discount rate of 0.035. For the full set of parameters, we performed a probabilistic sensitivity analysis (PSA) by running a Bayesian model in Jags from RStudio to obtain samples from the joint posterior distribution of model parameters for both the novel TAH and Syncardia TAH. For the PSA we used a simulation size of 50,000 estimated across three chains with a burn‐in of 5000 each. The full table of model parameters with 95% credible intervals and the distributions used in the PSA is reported in Table 1.

TABLE 1

Parameters of the cost‐effectiveness model

Values	Mean (95% credible interval)	Source	Parametric distribution
Probability of remaining in the TAH state – Survival curve parameters
Location parameter – Syncardia TAH	5.1 (4.6; 5.7)	(Arabía et al., 2018)	Generalized Gamma
Location parameter – novel TAH	4.14 (2.8; 6.02)	Simulated	Generalized Gamma
Shape parameter (assumed equal for both TAH devices)	0.9 (0.8; 0.9)	(Arabía et al., 2018)	Generalized Gamma
Probability of death while on support conditional on leaving the TAH state
1 month	0.7 (0.3; 0.8)	(Arabía et al., 2018)	Beta
Syncardia TAH	0.7 (0.3; 0.8)	(Arabía et al., 2018)	Beta
Novel TAH	0.61 (0.34; 0.84)	Simulated	Beta
2 months
Syncardia TAH	0.5 (0.4; 0.6)	(Arabía et al., 2018)	Beta
Novel TAH	0.12 (0.003; 0.41)	Simulated	Beta
3 months
Syncardia TAH	0.4 (0.2; 0.5)	(Arabía et al., 2018)	Beta
Novel TAH	0.14 (0.004; 0.43)	Simulated	Beta
>3 months
Syncardia TAH	0.2 (0.2; 0.3)	(Arabía et al., 2018)	Beta
Novel TAH	0.13 (0.02; 0.33)	Simulated	Beta
Complications while on support
Strokes
Syncardia TAH	0.23 (0.18; 0.27)	(Arabía et al., 2018)	Beta
Novel TAH	0.21 (0.1; 0.36)	Simulated	Beta
Proportion of disabling strokes	0.12 moderate‐severe disability; 0.39 severe	(Arabía et al., 2018)	Beta
Major bleeding
Syncardia TAH	0.21 (0.10; 0.36)	(Arabía et al., 2018)	Beta
Novel TAH	0.19 (0.08; 0.33)	Simulated	Beta
Major infection
Syncardia TAH	0.7 (0.65; 0.74)	(Arabía et al., 2018)	Beta
Novel TAH	0.59 (0.44; 0.74)	Simulated	Beta
Major device malfunction
Syncardia TAH	0.106 (0.08; 0.13)	(Arabía et al., 2018)	Beta
Novel TAH	0.108 (0.03; 0.22)	Simulated	Beta
Survival after HTx (all devices)
Location parameter	−5.24 (−5.52; −4.96)	(David et al., 2020)	Generalized Gamma
Shape parameter	0.543 (0.051; 5.80)	(David et al., 2020)	Generalized Gamma
Quality of life
Quality of life while on support
Syncardia TAH	0.64 (0.58; 0.69)	Assumed as informed by expert opinion	Beta
Novel TAH	0.70 (0.66; 0.73)	Assumed	Beta
Quality of life after HTx – without disabling stroke	0.76 (0.64; 0.86)	(Sharples et al., 2006)	Beta
Long term utility decrement with disabling stroke	−0.18 (−0.23; −0.13)	(Luengo‐Fernandez et al., 2020)	Beta
Costs
Cost heart transplant procedure and first month in the hospital
TAH cost
Syncardia TAH	€ 100,000	Expenses data from the Italian Ministry of health	‐
Novel TAH	€ 200,000	Assumed	‐
TAH implant procedure + first month hospitalization	156,625 (106,382; 216,435)	(Sharples et al., 2006)	Gamma
Monthly hospitalization cost after the first	Decreasing from 19,224 (3730; 47,241) for the 2nd month to 1744 (31; 6705) in the 8th month	(Sharples et al., 2006)	Gamma
Proportion of TAH patients discharged
Syncardia TAH	0.11 (0.09; 0.14)	(Arabía et al., 2018)	Beta
Novel TAH	0.73 (0.58; 0.86)	Assumed	Beta
Average Length of Stay before discharge (for those discharged)	50 (37; 64)	(Arabía et al., 2018)	Gamma
Costs of stroke	5587 (1039; 17,794)	Italian DRGs 559; 14 and 15 (mean of the three values)	Log‐normal a
Cost of Major bleeding requiring re‐operation	4321 (802, 13,762)	Italian DRG 174	Log‐normal a
Cost of major infection	7148 (1,320, 22,926)	Italian DRG 576	Log‐normal a
Cost of major device malfunction	81,551 (15,259, 260,400)	Italian DRG 103	Log‐normal a

Abbreviation: TAH, total artificial heart.

for all costs of complications standard deviation on the log scale was arbitrarily set at 0.5.

Utility functions and calculation of the value of information

The payer was presumed to be risk neutral, taking decisions based on expected INMB. We assumed that the payer could take “accept” or “reject” decisions and mandating further research by making coverage conditional to further evidence generation such as in “only in research” or “approval with research” schemes. These are the decision rules and policy options that would be available in Italy according to the proposed new national plan for the HTA of MDs (Tarricone et al., 2021). Nonetheless given the low incidence of patients eligible to TAH and the severity of the condition, only “approval with research” was considered as a feasible conditional reimbursement policy, meaning that the payer's decision could prevent access of the device to the market, but not limit its market shares. Following the national appraisal, it was assumed that regional health authorities would comply to the national decision, but practitioners would still be free to choose which device to utilize, that is, implementation would not be guaranteed after a positive reimbursement decision at the national level.

The manufacturer was presumed to make decisions on whether to submit or not for reimbursement based on the expectation of a positive NPV of the investment. Initial investment to enter a country was assumed to be €500,000. This cost may represent for example, the irrecoverable costs required to preparing and going through submission, such as the costs of hiring national consultants to build the submission dossier or the costs of negotiating with the national authorities. If the company took a “no go” decision, NPV was set to zero (i.e., zero revenues and costs). In case of a “go” decision, revenues were modeled as a function of a positive reimbursement decision from the payer, and the parameters that manufacturers believe to affect market shares in case of adoption. In case of a negative appraisal, market access of the technology would be prevented, and therefore market shares were set to zero and the manufacturer incurred a loss of the initial investment. In case of a positive appraisal, market shares were assumed to be based on the risk‐ratio of successful heart transplantations and modeled using a truncated function assuming 0 values for relative risks lower than 1 and following a cumulative normal distribution with mean 1.2 and standard deviation 0.07 otherwise. This function is similar to the one proposed by Willan et al. (Willan, 2008) and was assumed to reflect the company's belief around the link between the expected treatment effect and market shares. Operative income was calculated as revenues, minus direct and indirect operating costs. Direct operating costs were set at 20% of the price of the device whereas indirect operating costs were assumed to be €250,000 per year.

For both manufacturers and payers, we assumed that approximately 10 patients per year would receive the TAH and that the novel device would stay on the market for T = 10 years before becoming obsolete. We made the simplifying assumption that the price of the device, the market shares and the number of eligible patients would remain constant in each year considered for the calculation of the NPV and NMB, however, these assumptions could be easily relaxed in the model. More in general we assumed that during this time horizon no other changes such as price modifications, other competing technologies entering the market, or new evidence becoming available would occur and affect the estimation of the NPV, the NMB and the VOI for both payers and manufacturers.

The population INMB from the healthcare payer perspective was calculated as: Where ∆u and ∆c are differences in costs and consequences between the novel TAH and the Syncardia TAH that are directly relevant to the healthcare payer; k is the applied cost‐effectiveness threshold; I is the annual incidence of patients eligible to an artificial heart; and dr is the annual discount rate which was assumed to be the same for manufacturers and payers. The NPV for the company deciding to enter a market was calculated as: Where and are the direct, indirect operative costs and the initial investment cost incurred by the manufacturer; p is the price of the device; and δ is the company's annual market share.

The expected value of perfect information (EVPI) for manufacturers and payers was then calculated in the standard way by taking the difference between the expected value of a decision made with perfect information and the expected value of a decision made with current knowledge (Rothery et al., 2020). In addition, we introduced 2 constraints for the manufacturer to reflect his beliefs of additional market barriers beyond the payers' approval: a MCD and a maximum budget impact (MBI) constraint. For the MCD of the risk‐ratio of successful transplantation was arbitrarily set to 1.2 representing the threshold below which the manufacturer does not believe that clinical practice (and market shares) would change even if reimbursement was granted. For MBI we arbitrarily set a limit of €80,000 in the overall budget impact while on support until transplantation. This constraint may reflect for example, the fact that hospitals may be reluctant to purchase the novel TAH if the expected budget impact was too high and no immediate adjustments of the DRG tariff to accommodate the extra cost of the device was done. For both constraints, we assumed that the manufacturer would take a “no go” decision if the risk of not complying with any of these constraints was higher than 0.5. Following Koffijberg et al. (2018) the expected VOI in presence of a constraint was calculated through different steps. First, we determined the best option with the current knowledge that provided the highest expected payoffs to the company (i.e., either a positive operating income or zero) while complying with the applicable constraints and with the acceptable risk of exceeding these constraints. Second, for each simulation from the PSA the highest acceptable payoff was derived: the go decision was selected if the operative income was positive, and all applied constraints were met. alternatively, the no‐go decision was selected instead. The EVPI with a constraint is then calculated by subtracting the highest acceptable payoff from the best option with current knowledge (Koffijberg et al., 2018).

Value of information for manufacturers can also be affected by other uncertain parameters such as their direct and indirect operating costs or market dynamics like the time before a competitor would enter the market. Since this uncertainty cannot be resolved through further generation of evidence, they have been considered fixed in the base case analysis. One‐way deterministic analysis was then conducted to estimate the impact of these parameters on VOI. Specifically, the following parameters were varied by plus, minus 25%: the yearly indirect operating costs, the initial cost of the submission (e.g., the initial investment required to enter a specific market); the annual market size, that is the number of potential patients per year, the time horizon considered for calculating the NPV and the interest rate used by the manufacturer to discount future cashflows. In fact, while in the base case we used the same discount rates for payers and manufacturers, these rates may differ as they represent different types of opportunity costs; the opportunity cost related to the future marginal productivity of healthcare spending for the former, and the opportunity costs related to the returns that could be earned from other investments for the latter.

RESULTS

The novel TAH was expected to be more costly (with 0.975 probability) and more effective (0.997 probability) than the Syncardia TAH. At a willingness to pay (WTP) threshold of €30,000 per Quality Adjusted Life Year gained the expected incremental NMB of the novel TAH was €31,277 with a probability of being the most cost‐effective alternative equal to 0.67. At the same threshold, the probability that the manufacturer would benefit from entering the market was lower, at 0.53. This is because the likelihood of positive operative income is affected not only by the probability of HTA approval, but also by market shares which in turn have been modeled as a function of the device efficacy. Therefore, there may be cases in which even though the device would be considered cost‐effective by the healthcare (HC) payer, the estimated clinical difference would not generate a satisfactory level of revenues. The probabilities of a profitable market entry for the manufacturer get even lower if further constraints such as MCD or MBI are introduced. In fact, constraints further limit the cases when manufacturers would choose a go decision. For example, at a CE threshold of €30,000 only 22% of cases would have positive operating income, MCD greater than 1.2 and a 24‐month budget impact of less than €80,000 (Figure 2 and Figure 3).

FIGURE 2

Probability of a positive incremental net monetary benefit (INMB) (for payers) and net present value (NPV) of the decision to enter a market (manufacturers), as a function of payer's willingness to pay (WTP). MBI, Maximum Budget Impact; MCD, Minimum Clinical Difference; NMB, Net monetary benefit; NPV, net present value

FIGURE 3

Samples with positive net present value (NPV) for the manufacturer and compliance with the imposed constraints (blue dots) MCD, Minimum clinical difference; maximum budget impact (MBI), Maximum impact budget. Blue dots represent the simulations of the probabilistic sensitivity analysis (PSA) with a positive NPV for the manufacturer, with or without additional constraints imposed

Market shares had mean value of 0.345 with truncation at 0 (44% of samples) and maximum value at 1. The expected NPV at a payer WTP threshold of €30,000 was approximately 2.8 Million (95% CrI Million € −2.5 – 11). Results of the model for both manufacturers and payers are reported in Table 2.

TABLE 2

Estimation of the net monetary benefit (NMB) (Payer) and net present value (NPV) (manufacturer)

HC payer
	NMB syncardia TAH (95% CrI)	NMB novel TAH (95% CrI)	Incremental net monetary benefit
	€ −31,309 (−148,807; 82,310)	€736.8 (−120,000; 126,000)	€31,277 (−79,324; 158,269)

Abbreviations: Cri, Credible intervals; HC, healthcare.

Expected value of perfect information

The expected VOI varied between manufacturers and payers and depended on whether constraints were added to the manufacturer's decision rules. In the absence of constraints, VOI for manufacturers increases until a payer's WTP of €8500 and then remain constant (Figure 4). At this payer's WTP threshold the uncertainty over the profitability of entering the market is maximum and the expectations of the NPV is close to zero. For WTP values higher than €8500 the expectation of the NPV turns positive and the manufacturer would change its decision and opt for entering the market. Nonetheless, after deciding for entering the market, uncertainty would remain on the level of achievable market shares, which is unrelated to the payers' WTP, so that the VOI remains constant.

FIGURE 4

Expected value of perfect information (EVPI) for healthcare payers and manufacturers and different constraints applied by manufacturers. Value of information (VOI) is estimated using population net monetary benefit (NMB) for healthcare (HC) payers and manufacturer's net present value (NPV), using a common time horizon of 10 years and 10 incident patients per year

At a payer's WTP threshold of €30,000, the population EVPI for payers is approximately €1.4 million whereas for manufacturers the value is lower at €451,335. However, if constraints are added the EVPI of the manufacturer vary considerably. When a MCD constraint is added, the risk of not meeting the desired threshold of 1.2 for the novel TAH is about 0.57 and therefore it exceeds the maximum acceptable risk for manufacturers. Consequently, given the current knowledge, the manufacturer would not submit for approval no matter the expected NPV in the absence of the constraint. In this case, the gains from the VOI becomes higher as having perfect information would allow the manufacturer not to take such conservative decision and apply for reimbursement should the device prove to comply with the constraint and generate a positive income. Since the probability of a positive NPV also depends on payer's behavior, the benefit from perfect information is higher with higher values of the payer's WTP threshold and reaches its maximum when the probability of approval reaches 1 (and the VOI for payers is zero; Figure 4).

With the MBI constraint, the risk of the device having a 24‐month budget impact higher than €80,000 is 0.47 and therefore is below the maximum acceptable risk for the manufacturer. Hence, with current knowledge, the company would choose to enter the market only based on the expectations of a positive NPV. Nonetheless, in cases when the manufacturer would choose to enter the market (i.e., for payers' WTP higher than €8500) the application of the constraint will lower the VOI compared to the scenario with no constraints. This is because the expected value of the NPV in presence of perfect information would be lower due to the inclusion of the constraint in the manufacturer's decision rules. Specifically, any time the device would not meet the constraints in the samples from the PSA the manufacturer would still opt for not entering the market even with positive expected NPV. If the proportion of samples exceeding the constraint is high, the expectation of the NPV with full information would be strongly reduced. Note that due to this effect, over a certain value of the WTP threshold, the expectation of the NPV with full information gets even smaller than the expectation of a go‐decision with current knowledge, resulting in a negative EVPI. The full results of the EVPI for manufacturers and payers and different constraints are reported in the supplementary materials (table S1)

With both MCD and MBI constraint, the EVPI of the manufacturer increases similarly to the case with the MCD constraint because similarly to that case the manufacturer would not opt for entering the market given that the risk of the device not complying with this more restrictive constraint is higher than 0.5. The value of the EVPI however, remains lower due to the effect of the MBI constraint reducing the expected value of the optimal choice.

Univariate sensitivity analysis on those parameters for the manufacturer that are not related to the performance of the device were different depending on whether constraints were applied or not. In the absence of a constraint, variations in the VOI were limited, whereas with any of additional constraints applied the EVPI of the manufacturer varied considerably with the annual market size and the time horizon considered for the NPV being the most influential parameters (Figure 5).

FIGURE 5

Univariate sensitivity analysis of parameters which may affect the value of further research for the manufacturer Parameters were varied by ±25%. MBI, maximum budget impact; MCD, Minimum clinical difference

DISCUSSION

A (more) efficient evidence generation for new MDs would be beneficial for all, including manufacturers, healthcare payers and, most of all, the patients. Particularly, increasing the relevance of evidence for HTA purposes would improve the capacity of payers to take informed decisions at earlier stages and achieve their objective of maximizing population health. More relevant evidence would also be beneficial for manufacturers willing to maximize their probability of accessing markets and having positive incomes. Ultimately, patients will benefit from improved decision making and reduced uncertainty over the true performance of novel technologies. Nonetheless, this analysis, and the case study on a hypothetical TAH, showed that the value of further research may differ between payers and manufacturers. Differences originate from the fact that the two parties pursue different objectives, but also may depend on other assumptions and prior beliefs specific to either one or the other. For example, when constraints were added to the decision rules of manufacturers, the VOI changed considerably (but the same would have happened if constraints were imposed to the payer). The analysis also showed that VOI for manufacturers is dependent on other factors which are not directly related to the uncertainty over the device, such as the market size of the target country or the expected time window during which manufacturers expect to have revenues before the device becomes obsolete and is overtaken by incremental innovations.

While these differences are unavoidable, our results confirm the importance that any potential misalignment in the (perceived) VOI is identified and addressed in order to achieve optimal evidence generation processes along the product life cycle. One ideal moment when such misalignments may be reduced is during EDs when the device is close to obtaining market approval and the evidence for HTA purposes is still to be generated. Although it is unlikely that dialogs on the study design during an ED are based on an explicit and transparent sharing of information and decision rules from both sides, a more consistent use of VOI may reduce asymmetric information and signal the potential reactions of each part to decision uncertainty, thus contributing to align on evidence requirements. In a typical ED for HTA, HTA bodies provide non‐binding recommendations to manufacturers about their proposed pivotal study for HTA submission. The study will be fully funded by industry, so that HTA bodies just need to consider to what extent the results of such study will enable them to make informed decisions by the time the manufacturer submits for reimbursement. HTA bodies could use VOI to evaluate if, according to their utility function, the proposed design and sample‐size exhaust the VOI for the technology under assessment, or whether residual VOI would remain after the study reports its results. If manufacturers, based on their VOI calculations, propose a study design that also exhausts VOI from the HTA bodies and payers' perspective, the incentives for manufacturers and payers are likely to be similar and the appropriate evidence will be generated. If the proposed study leaves residual VOI, then HTA bodies could signal what would be the minimum level of evidence that should be generated pre‐submission for the technology to be adopted. Since the recommendations of HTA bodies on a technology can prevent market access and therefore are usually considered in manufacturers' utility function this information can be used to update manufacturers' analysis on the optimal study design, thus obtaining more refined estimates of the expected payoffs for their marketing strategies and a closer alignment with the evidence requirements of HTA bodies. In theory, more complex interactions between HTA bodies and manufacturers might also allow further discussion on the intended evidence generation plan to seek agreement on the optimal sequential design of studies in a life‐cycle perspective including optimal pre‐launch and post‐launch evidence generation. Nonetheless, such types of interactions are not actually envisioned within a typical ED.

Another moment in the product lifecycle when manufacturers and payers may discuss about evidence needs, is later, at the time of HTA appraisals, if adoption decisions by national or regional payers envisage the possibility of arranging performance‐based risk‐sharing schemes. One of the key elements of these schemes is that the price, or reimbursement of a technology is linked to its performance which is assessed through a purposeful, prospective data collection. Federici et al. (2021) reported that CED programmes for MDs exist in 7 European countries and that 78 such schemes were implemented in the last 5 years. They also showed that these schemes have different arrangements about how they are implemented, and how responsibilities are shared between the parties involved. For example, in France the high authority for health (Haute Autorité de la Santé ‐ HAS) may temporarily register a device in the positive list for reimbursement and make renewal conditional on the generation of further evidence to reduce some of the remaining uncertainty on the performance of the device. The responsibility of the generation of evidence relies entirely on the manufacturer, which must report the new data at the time of the reassessment, usually after three or 5 years. However, it is not uncommon that re‐appraisals from the HAS report that the required data is of insufficient quality or even totally missing. This may reflect the lack of incentives for manufacturers to conduct the new research once their product is approved even for a limited period. For example, the company may be developing and plan to launch an incremental innovation within a short time or may know that other competitors are about to launch similar devices so that investing in further research may not be considered worthwhile. Even if companies' private information and strategies may not be accessible to HTA bodies and payers, considerations of the expected market size for the technology under assessment and historical information on the rate of innovation in the specific field of the technology could provide insights to payers on the likelihood that manufacturers will conduct the desired research. Nonetheless, the difference in goals and incentives together with lack of transparency may still hamper any alignment process so that when agreeing CED schemes, HTA bodies should always ensure that appropriate mechanisms are in place to monitor and eventually sanction uncomplying manufacturers.

This study has estimated the VOI for manufacturers in relation to the specific decision problem of entering a target market. In fact, the analysis included parameters that are generally considered as poorly generalizable (as e.g., costs and patients utilities). In addition, and perhaps even more important, the estimation of VOI for the manufacturers incorporated other aspects that are likely different across countries. These aspects include for example, the decision rules adopted by payers to determine reimbursement (e.g., based on clinical criteria only rather than NMB), and characteristics of the market, such as the market size, or the existence of other competitors. Consequently, the VOI for the company estimated in one market would not be directly applicable to the company's decisions to enter other markets. However, some of the benefits of reducing uncertainty for one specific decision problem, will also have positive spill‐over effects on other (future or parallel) decisions of the company, although as mentioned the extent of these spillover effects will be dependent on the type of uncertainty to be reduced. For example, in the case study proposed, future decisions of the company will benefit from a reduced uncertainty on the rate of adverse events, but less so on the costs of such adverse events, since these are likely to be more context specific. Obviously, the more this reduction in uncertainty is generalizable the higher the spill‐over effects for the company and the higher the global VOI. When the contributions of specific study designs are considered then also the type of study conducted may impact the generalizability of results and therefore how “reusable” that evidence would be for other company's decisions. For example, a Randomized controlled trial would be expected to be more generalizable compared to an observational study based on a registry.

Eckerman and Willan (Eckermann & Willan, 2009) argued that a global trial design that maximizes the VOI across jurisdictions (optimal global VOI trials) would benefit manufacturers and healthcare payers alike, by increasing the transferability of trial evidence and allowing optimal allocation of trial sample across jurisdictions so to minimize the opportunity costs of delay in approval, based on what decision is locally optimal. The authors also extended the framework to design optimal global VOI trials in the context of risk sharing agreements, arguing that such designs would also allow more complete and robust approval with research designs (Eckermann & Willan, 2011). From a manufacturer perspective, an optimally designed trial would allow additional revenue generation in jurisdictions where approval with research is locally optimal while evidence is generated in other jurisdictions where the global trial is ongoing. According to this framework, manufacturers could design their optimal trials within a CED scheme accounting for the expected behaviors of payers in each jurisdiction, the expected revenues in each jurisdiction and the global and local costs of the proposed clinical study.

In any case, while spill over effects at the global level may be difficult to estimate, calculating the VOI for single‐market decisions may be still useful when the evidence required (e.g., by payers) is highly context specific, as is the case of many CED schemes focusing on the real‐world performance of the device in the country's clinical practice. In addition, by calculating single‐market VOI, manufacturers can already say whether the costs of further research would be justifiable even considering the expected returns of this single decision problem. So, in a sense, the single market VOI for manufacturers is the first check for a company on whether further research would be worthwhile. These decisions may also have negative spill‐over effects. For example, a negative appraisal might have consequences on the decisions taken by payers in other markets. Or the decision of not submitting for approval in a country may impact the company's probability of accessing late adopters' markets. The incorporation of these aspects may require more complex models to reflect the global value of research in other markets.

To the best of our knowledge, no other studies applied VOI to manufacturers decision processes using a different maximization objective and incorporating the existing barriers to access and market dynamics. As discussed, this analysis may support manufactures to optimize their evidence generation plan and contribute to reduce misalignments between payers and manufacturers. This analysis has also limitations.

First, the proposed case study uses a hypothetical TAH as the novel device to calculate the VOI and make several assumptions on both payers and manufacturers behaviors and beliefs. In addition, other simplifying assumptions when building the model were required by the paucity of evidence on the existing TAH. For example, it was impossible to characterize the correlations between costs and effects in the model. While these assumptions are likely to affect the results of the economic model and VOI analysis, they do not invalidate the conclusions of the study on how VOI could be used to align research objectives between payers and manufacturers. Nonetheless, the development of real‐world applications is necessary to understand the real entity of the misalignments on VOI between payers and manufacturers, and the extent to which the proposed framework is applicable to other technologies, for example, pharmaceuticals, and health conditions.

Second, even in the case of real applications, several parameters to calculate VOI may not be easily estimated empirically. In the case of manufacturers, the value of these parameters may be informed by the practices that are routinely performed for business intelligence. For example, studies on market dynamics (e.g., competitors' product and pricing strategies, novel devices in pipeline) could inform the time horizon to consider when calculating the NPV. Also, the link between the device effectiveness and the market shares, including any MCD that is required to trigger adoption could be derived from heuristics based on previous experience of the company with similar devices, or by directly eliciting providers preferences for example, using stated preferences techniques such as discrete choice experiments. Similarly, the investment costs as well as the direct and indirect operating costs could be informed by operations management or other analyses to assess investment decisions. In the case of healthcare payers, the estimation of VOI requires an estimate of the present and future population that will benefit from the reduction in uncertainty over the decision being addressed and the time horizon before exogenous changes (such as new treatments, new evidence becoming available, changes in prices etc.) would modify the results of the underlying cost‐effectiveness model and therefore the VOI. These parameters can be estimated sourcing available real‐world data such as for example, epidemiological data, and using approaches similar to the ones used by manufacturers to do horizon scanning of the upcoming technologies.

More in general, several assumptions were required on the expected values of parameters used in the models and their parametric distributions to perform PSA and estimate VOI. Lack of information on model parameters is a common issue when doing early assessment of novel technologies, and it has been argued that performing probabilistic analysis (including VOI) with limited data on model parameters can create pseudo‐certainty (Grutters et al., 2019) and incorrect decisions (Lofgren, 2020). In another work in this supplement, Iskandar et al. explore new methods on how characterize uncertainty based on available information while avoiding imposing parametric assumptions for parameters where evidence is sparse (Iskandar et al., n.d.).

Lastly this study only estimated the EVPI for manufacturers and payers, that is, the benefits of removing all existing uncertainty on model parameters. Further studies may extend the analysis to calculate the payoffs of reducing uncertainty over specific parameters (expected value of partial perfect information EVPPI) or with specific study designs (expected value of sampling information, EVSI; Rothery et al., 2017). Further research may also explore the role of additional constraints on the EVPPI and expected value of sampling information and develop methods to incorporate spill‐over effects on the company's decision making.

CONCLUSIONS

Misalignments between payers and manufacturers about the value of additional research may be considerable and depend on both parties' decisions rules, the type of existing uncertainty and other contextual factors such differences in the perception of the market size/patient population or the time horizon during which the evidence generated will be relevant. Such misalignment may affect the efficiency of evidence generation processes resulting in longer or uninformed introduction of innovation. Therefore, it is vital that these misalignments are reduced along the different stages of the product life cycle and especially in the first stages of product development. Value of information analysis can be used as an explicit framework for both payers and manufacturers to identify and resolve any potential conflict in the perceived value of evidence.

CONFLICT OF INTEREST

The authors have no relevant financial or non‐financial interests to disclose.

Supplementary Material

Click here for additional data file.