Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI's Patient Groups and Clinical Trials Project.

BACKGROUND: While patient groups, regulators, and sponsors are increasingly considering engaging with patients in the design and conduct of clinical development programs, sponsors are often reluctant to go beyond pilot programs because of uncertainty in the return on investment. We developed an approach to estimate the financial value of patient engagement.
METHODS: Expected net present value (ENPV) is a common technique that integrates the key business drivers of cost, time, revenue, and risk into a summary metric for project strategy and portfolio decisions. We assessed the impact of patient engagement on ENPV for a typical oncology development program entering phase 2 or phase 3.
RESULTS: For a pre-phase 2 project, the cumulative impact of a patient engagement activity that avoids one protocol amendment and improves enrollment, adherence, and retention is an increase in net present value (NPV) of $62MM ($65MM for pre-phase 3) and an increase in ENPV of $35MM ($75MM for pre-phase 3). Compared with an investment of $100,000 in patient engagement, the NPV and ENPV increases can exceed 500-fold the investment. This ENPV increase is the equivalent of accelerating a pre-phase 2 product launch by 2½ years (1½ years for pre-phase 3).
CONCLUSIONS: Risk-adjusted financial models can assess the impact of patient engagement. A combination of empirical data and subjective parameter estimates shows that engagement activities with the potential to avoid protocol amendments and/or improve enrollment, adherence, and retention may add considerable financial value. This approach can help sponsors assess patient engagement investment decisions.

Introduction

Increasingly during the past decade, stakeholders in the biopharmaceutical research and development (R&D) enterprise have sought ways to improve engagement and partnership with patients throughout the development, regulatory, and postapproval life cycle. To this end, regulatory agencies and numerous public and private collaborative initiatives have developed recommendations, frameworks, and resources to engage patients and solicit and incorporate the patient perspective in the life cycle.[1-10]

While the moral and ethical case for patient engagement has been made,[11-15] there are also strongly pragmatic reasons for bringing the patient perspective into drug development. These include improving the relevance of study questions to patients living with (or trying to prevent the onset of) disease, selecting the most appropriate endpoints for clinical trials, improving the clinical trial experience for patients, eliminating barriers to participation, and optimizing strategies for recruitment, adherence, and retention.[14,16-21] Although some companies that are engaged in, or sponsoring, R&D also have actively participated in planning and piloting patient-centric initiatives, many others struggle with the fact that patient-centric approaches represent a profound departure from the more familiar and paternalistic (“physician as the only expert”) approaches that have characterized the pharmaceutical R&D enterprise for more than 50 years.[3,16,20,22-26]

Several factors may be delaying the adoption of patient-centric initiatives. Some organizations have pointed to a high level of confusion around how to operationalize a patient-centric approach or achieve the necessary culture change.[14,15,22,27] Other organizations may not have internal mechanisms to coordinate and monitor these activities or may have concerns about conflicts of interest and other legal issues.[1,2] However, a frequently mentioned factor delaying adoption relates to uncertainty around the financial value that patient-centricity provides. In the absence of published evidence of financial value or a clearly defined value proposition, sponsors may be reluctant to allocate substantial capital and personnel resources.[3,21,22,28,29] Providing quantitative assessment of the value of patient engagement would help sponsors identify the opportunities that are good from the patient perspective and that also generate value and revenue.

In this paper, we present a method for assessing the value of patient engagement using standard risk-adjusted financial modeling techniques, with the novel component being its application to patient engagement. The approach, expected net present value (ENPV), accounts for the key business drivers of cost, time, revenue, and risk. It is a methodology commonly used in high-risk fields such as the pharmaceutical and oil and gas industries.[30-38] To our knowledge, no quantification of the patient-centric value proposition has been published.

Our goals are to develop and demonstrate a means to assess the financial value of patient engagement and to provide an additional mechanism to assist R&D sponsors in deciding whether to invest in patient-centric initiatives, to determine how many resources to allocate to these initiatives, and to anticipate their impacts. First, we review business drivers that link patient engagement to value generation. We then summarize the ENPV model and how patient engagement changes the value drivers in the model. Next, we introduce the pre–phase 2 and pre–phase 3 base case generic oncology examples used in the rest of the paper. Our results show the impact of engaging with patients on these base cases and provide a methodology that can be refined as more objective data on the impact of patient engagement are collected. Finally, we describe implications and limitations of the analyses and results.

Methods

Value Drivers

The value of a drug development project to a pharmaceutical company is generally based on five key drivers: revenue, costs, time, risk, and intangibles (Table 1). Approaches for estimating these drivers, other than intangibles, and combining them into composite measures for value are well known, widely accepted, and commonly used across industries to inform corporate strategies and portfolio management.[33] The novel aspect of our work is the application of such a composite valuation measure to patient engagement, and the means by which the value drivers are adjusted to account for such engagement.

Table 1.

Pharmaceutical Project Value Drivers.

Value Driver	Definition
Revenue	The financial benefit from project success, with success defined as achieving both regulatory approval and commercial viability.
Costs	The out-of-pocket spending and staff costs by phase during development, as well as sales, marketing, manufacturing, and administrative costs.
Time	The intervals between successive development phase transitions, with cumulative time reflecting time to approval.
Risk	Technical risk is the probability that a sponsor will choose to advance development after a development stage completes. Each phase of development has a probability of technical success (PTS). Regulatory risk is the probability that health authorities will approve the treatment as planned—probability of regulatory success (PRS). The probability that a drug will successfully complete all remaining planned trials and be approved by a health authority is the “probability of technical and regulatory success” or PTRS. Other risks that can be incorporated into this approach include operational, cost projection, and forecasting risk.
Intangibles	The benefits obtained that are impossible or unrealistic to characterize financially. Typical intangibles are the strategic value of a particular project, the value of setting a precedent, patient satisfaction with a sponsor’s decision to invest in a product, and improvement in reputation.

Expected Net Present Value

A revenue stream with accompanying development, manufacturing, and marketing costs can be summarized using net present value (NPV).[33-35] NPV is defined here as the after-tax, inflation-adjusted present value of the future net cash flow, assuming successful development and regulatory approval. Because a project in development can fail before it ever gets to the market—resulting in a loss of investment—the expected net present value (ENPV) is used to reflect the NPV, accounting for the various paths that lead to failure or success. ENPV is a widely recognized metric that combines the revenue, cost, and time value drivers with the associated risks of getting to market and reflects the risk-adjusted NPV.[30-38]

ENPV is assessed by considering each possible path (ie, failing at different points in development or achieving regulatory approval) and estimating for each path taken (Figure 1). The present value of the costs and revenues (or just costs if failure occurs before launch) for each path is multiplied by the probability of that path occurring. The sum total of all these risk-adjusted NPVs is the ENPV of the entire project. For example, a drug currently in phase 2 will either succeed or fail phase 2. Success creates the opportunity to move into phase 3, while failure entails spending of costs committed through the end of phase 2 or to premature termination of a study. Similarly, the phase 3 probability of technical success and probability of regulatory success determine how likely it is that funds are lost due to failure or that revenue is gained due to regulatory approval. The ENPV example in Figure 1 is described in detail in Supplement S1. As noted in the supplement, many real-world pharmaceutical projects are more complex than shown in Figure 1 and allow for multiple development paths to failure or success (such as an interim analysis or early termination of studies because of safety concerns), multiple definitions of success for a study, and submission to multiple regulatory agencies. For simplicity, we confine ourselves to a single development path, single revenue stream, and success/fail step examples as shown in Figure 1.

Figure 1.

Example of a simple ENPV model for a project in phase 2. Circles represent uncertain events; values adjacent to each path from the circles indicate the probability that the project will follow that path. Values on the right indicate the probability that the project will terminate following that path and the NPV for that path (eg, probability of technical and regulatory success = 22%). ENPV is calculated by totaling the product of the NPV and probability of each path. In this case, ENPV = (0.22 × $400) + (0.02 × –$45) + (0.16 × –$40) + (0.60 × –$3) = $77 MM [details in S1].) ENPV, expected net present value.

While ENPV has its limitations (see Discussion), it can be particularly valuable when comparing alternative strategies for a particular project, such as with or without different forms of patient engagement.

Impact of Patient Engagement

Patient engagement activities are incorporated into the ENPV assessment by examining how the value drivers are affected by the activity. For example, if the patient experience in the trial can be improved (eg, prompt responses to people who inquire about a trial, less arduous screening, less demanding study design) on the basis of patient input (eg, patient advisory panels or crowd-sourced review), there may be more rapid enrollment, increased adherence to the protocol, and/or reduced dropout rate from the study.[39,40] Faster enrollment will reduce the time to complete the study, which will in turn increase the present values for revenue and costs for each development path, though reduced development time may also decrease costs (Figure 1). Assuming a drug is efficacious and safe, increased patient adherence is likely to increase the probability that the drug will meet the efficacy and safety criteria of the study and can advance to the next phase. The reduced dropout rate may thereby increase the phase success probability. Reduced dropout rate will also increase the quality of the data from the study overall, which, when combined with better demonstrations of efficacy and safety, may increase the probability of regulatory success. Overall, patient engagement that improves the patient experience can potentially increase the likelihood of a drug succeeding in the clinical trials and gaining regulatory approval.

The ENPV assessment allows estimating whether the incremental value of the changes from patient engagement outweigh the up-front investment (time, resources, and cost) of the patient engagement activity. The challenge with applying the ENPV model to patient engagement is that it is difficult to characterize the relationships between value drivers and patient engagement. Below we show 2 examples of such relationships: (1) a model for the avoidance of a protocol amendment that is well quantified based on empirical data and (2) a model for improving the patient experience in a trial based on subjective estimates of the impact on the value drivers. The Results section assesses the impact for patient engagement activities that yield at least 1 of these 2 benefits. The Discussion addresses application of these results to the more realistic circumstance when engagement does not always yield either of these benefits.

Base Case Assumptions

Base case project: oncology clinical trial

Our base case is a generic oncology new molecular entity. We consider 2 situations: a pre–phase 2 drug and a pre–phase 3 drug. Pre–phase 2 (or pre–phase 3) refers to the time during protocol development for phase 2 (or phase 3) studies during which a patient engagement activity could take place and influence phase 2 (or phase 3) plans.

Risks, costs, and cycle times (defined in Table 1) are based on data from the Tufts Center for the Study of Drug Development (Table 2).[41-45] Details for these base case assumptions are in Supplement S2.

Table 2.

Base Case Cost, Time, and Technical and Regulatory Probabilities of Success.

Phase	1	2	3	Regulatory
Cost ($MM)	4.4	7.3	61.8	52.0a
Cycle time (mo)	21.8	33.3	33.7	9.6
Probability of success, %	59.4	33.0	52.4	87.5

a$52MM reflects $2MM for regulatory expenses and $50MM[46] (5%-10% of peak sales) for preapproval marketing expenses incurred before a regulatory decision.[47,48]

Revenue is based on a logarithmic revenue growth curve through 10 years to peak, with peak sales assumed at $1B.[49,50] Patent expiration with entry of generic competition is at 11 years postlaunch, following which revenue is assumed to erode exponentially, with 50% revenue lost in the first year.[51,52]

The pre–phase 2 base case assumes that phase 1 has been completed and a decision is being made whether to advance to phase 2. Cost to launch is $121MM ($7.3 + $61.8 + $52.0), time to launch is 77 months (33.3 + 33.7 + 9.6), and probability of technical and regulatory success (PTRS) is 15% (33% × 52.4% × 87.5%). Using a discount rate of 11%, the midyear convention for discounting, sales and marketing at 39% of net sales, preapproval marketing expenses at 5% of peak sales, cost of goods sold at 30% of net sales, and a tax rate of 37.5%,[47] the NPV for a successful launch is $493MM.

The pre–phase 3 base case assumes the drug has already succeeded in phase 2 and now the decision is being made whether to advance to phase 3. Calculations are similar to those for the phase 2 base case but exclude phase 2 values. Cost to launch is $114MM ($61.8 + $52.0), time to launch is 44 months (33.7 + 9.6), and PTRS is 46% (52.4% × 87.5%). The NPV for successful launch is $640MM. The value is higher than in the pre–phase 2 scenario for several reasons: phase 2 risks have now been resolved, phase 2 costs are considered “sunk,” and the drug is closer to launch (causing the revenue stream to start sooner, resulting in a higher present value of income).

Results

Patient Engagement Pre–phase 2

Avoidance of a protocol amendment

Numerous benefits may occur when a sponsor works with a patient group to revise a protocol. Patient input may identify eligibility criteria that can be relaxed to increase the number of patients able to participate, modify the schedule of events to make the study more attractive for patients by making the protocol less arduous for patients to follow, and provide more mechanisms to offer engaging feedback to patients during the study, and so on.[2,46,53] Almost every one of these changes, if implemented after submission of the protocol, requires a protocol amendment. The top reason for amending a protocol is to modify study eligibility criteria as a result of changes in study design strategy and difficulties recruiting patients.[54,55]

As a starting point, we assume a protocol review with a patient group results in the avoidance of one phase 2 amendment. On average, one amendment adds 90 days to the development timeline and entails direct costs of $141,000 for phase 2 trials.[55] Indirect costs (primarily costs associated with sponsor and contract research organization personnel time to implement an amendment) are estimated to be 3 to 4 times larger.[55] Assuming the base case includes the amendment that is being avoided, these time and cost changes to the base case result in a 3-month earlier launch, a reduction in cost to launch by $0.5MM (additional cost for the patient engagement activity is considered separately in the Discussion), an NPV increase of $25MM, and an ENPV increase of $4MM (Table 3). The earlier approval increases the present value of revenue and increases cumulative revenue since revenue generally tapers very quickly after patent expiration. There is no impact on risk. Of note, with a 3-month acceleration but no change in total cost, NPV is $517MM and ENPV is $70MM—values very similar to those with the cost changes, reflecting that most of the benefits from avoiding an amendment come from the 3 months’ reduced development time rather than the reduced development cost.

Table 3.

Changes in Key Metrics Resulting from Patient Engagement Pre–phase 2.

	Base Case	Effect of Avoiding a Phase 2 Amendment	Chg	Effect of Improving Phase 2 Patient Experience	Chg	Combined Impact	Chg
Launch date	–	3 mo earlier		6 mo earlier		9 mo earlier
NPV ($MM)	493	518	25	531	38	555	62
PTRS, %	15	15	0	20	5	20	5
ENPV ($MM)	67	71	4	97	30	102	35
Cost to launch ($MM)	121.1	120.6	–0.5	121.1	0.0	120.6	–0.5

Abbreviations: Chg, change from base case; ENPV, expected net present value; NPV, net present value; PTRS, probability of technical and regulatory success.

Improvement of patients’ study experience

Patient group protocol review can provide benefits beyond potentially avoiding a protocol amendment.[39] Better communication with people who inquire about a trial, making the trial more palatable to patients, and relaxing the eligibility criteria may increase enrollment, shortening the time to launch. Making the informed consent easier to understand, making the trial less demanding for patients, and providing in-trial feedback to patients may increase protocol adherence and reduce patient dropout rates. For a medical treatment that is effective and safe, increased adherence and reduced dropout will improve the quality of data used for sponsor go/no-go decisions and for regulatory review, potentially increasing the probabilities of technical success and regulatory success.

There are few documented examples of patient engagement for protocol review, and essentially no literature from which to draw empirical estimates for a review’s effect on enrollment, adherence, and dropout. Instead, we make subjective assumptions of changes in time to launch and technical and regulatory probabilities. These assumptions were developed by 2 authors (B.L. and M.G.) who have extensive experience with decision analysis and risk-adjusted financial analysis in pharmaceutical development.

Actual enrollment time exceeds planned enrollment more than 50% of the time, and sponsors use numerous tactics to improve recruitment.[56] We assume patient engagement in protocol design will reduce enrollment time by 25%. For patient engagement pre–phase 2, the benefits gleaned are likely also to impact phase 3. Phase 2 and 3 oncology cycle times are 33.3 and 33.7 months, on average. If 12 months of these cycle times are spent on enrollment, the assumption is that patient engagement in pre–phase 2 reduces enrollment time by 3 months for both phase 2 and phase 3. We can assume both phases benefit, because without a patient engagement activity, a missed opportunity to improve the patient experience and enrollment in phase 2 would also be missed in phase 3.

On average, nearly 1 in 5 patients drop out of clinical trials, with dropout rates sometimes above 30%.[56,57] Assessments of efficacy and safety are sensitive to missing data due to dropouts and reduced data quality from poor adherence. For the impact of increased adherence and reduced dropout,[57] we assume an increase in probability by 5 percentage points for the phase 2 and phase 3 probabilities of technical success, that is, an increase of phase 2 probability from 33% to 38% and of phase 3 probability from 52% to 57%. Similarly, we assume a 3-point increase in probability of regulatory success, from 88% to 91%. We conduct a sensitivity analysis of these assumptions in the Discussion section.

Applying these assumptions to the phase 2 base case, patient group protocol review results in a 6-month-earlier launch, an NPV increase of $38MM, a PTRS increase of 5 percentage points, and an ENPV increase of $30MM (Table 3).

Combined impact

Patient group review of a phase 2 protocol has the potential to provide both the avoidance of an amendment and improvement of the patient experience. The combined impact is a 9-month-earlier launch, a reduction in cost to launch by $0.5MM, an NPV increase of $62, a PTRS increase of 5 percentage points, and an ENPV increase of $35 MM (Table 3).

Patient Engagement Pre–phase 3

Amendments add an average of 90 days to development and entail direct costs of $535,000 for phase 3 trials, suggesting indirect costs of $2.1MM (4 times larger than direct costs).[55] Applying these time and cost changes to the phase 3 base case results in a 3-month earlier launch, a reduction in cost to launch by $2.1MM, an NPV increase of $32MM, and an ENPV increase of $15 M (Table 4).

Table 4.

Changes in Key Metrics Resulting from Patient Engagement Pre–phase 3.

	Base Case	Effect of Avoiding a Phase 3 Amendment	Chg	Effect of Improving Phase 3 Patient Experience	Chg	Combined Impact	Chg
Launch date	–	3 mo earlier		3 mo earlier		6 mo earlier
NPV ($MM)	640	672	32	671	31	705	65
PTRS, %	46	46	0	52	6	52	6
ENPV ($MM)	275	290	15	332	57	350	75
Cost to launch ($MM)	113.8	111.7	–2.1	113.8	0.0	111.7	–2.1

Abbreviations: Chg, change from base case; ENPV, expected net present value; NPV, net present value; PTRS, probability of technical and regulatory success.

Applying the same assumptions described for the pre–phase 2 base case for improving the patient study experience, patient group phase 3 protocol review results in a 3-month earlier launch, an NPV increase of $31MM, a PTRS increase by 6 percentage points, and an ENPV increase of $57MM (Table 4).

Finally, the combined impact of avoiding an amendment and improving the patient experience for patient group review of a phase 3 protocol is a 6-month-earlier launch, a reduction in cost to launch by $2.1MM, an NPV increase of $65MM, a PTRS increase by 6 percentage points, and an ENPV increase of $75 MM (Table 4).

Discussion

While many pharmaceutical and biotechnology companies are eager to embrace patient-centric drug development, a key concern delaying its adoption is uncertainty around the financial value that patient engagement activities provide. Sponsors may be reluctant to allocate substantial capital and personnel resources to patient engagement in the absence of a clearly defined value proposition, particularly given the limited experience most sponsors have with patient engagement and the currently limited regulatory and legal guidance. Additionally, industry typically perceives less value than patient groups in patient engagement, resulting in potential conflicts of interest between sponsors and patient groups.[40] To help address this barrier, we developed an approach to estimate the financial value of patient engagement, accounting for the business drivers of cost, risk, revenue, and time.

Patient Engagement: Cost Versus Value

There are a multitude of ways a sponsor could engage with patients for review and feedback on a clinical protocol. Some approaches use 1- or 2-day facilitated discussions, with the sponsor costs including preliminary research, staff preparation and participation time, facilitator and facility fees, and travel and lodging expenses for several patients or patient advocates. We will use $100,000 as an estimate for sponsor expenditures on such a meeting.

The reductions in cost and increases in NPV and ENPV all greatly exceed an investment of $100,000 in patient engagement (Tables 3 and 4). The reduction in development cost alone ($0.56MM for pre–phase 2, $2.1MM for pre–phase 3) as a result of having 1 less amendment exceeds the costs of engagement by a factor of 5 for pre–phase 2 and 21 for pre–phase 3. The gains in NPV and ENPV can be several hundred times the investment for engagements that avoid an amendment and improve the patient experience (Table 5).

Table 5.

Ratio of Reduction in Cost to Launch, Gain in ENPV, and Gain in NPV to a $100,000 Investment in Patient Engagement.

	Avoiding an Amendment	Improving Patient Experience	Combined
Pre–phase 2
Cost gain	5×	–	5×
ENPV gain	38×	301×	349×
NPV gain	245×	382×	619×
Pre–phase 3
Cost gain	21×	–	21×
ENPV gain	150×	570×	750×
NPV gain	320×	309×	649×

Abbreviations: ENPV, expected net present value; NPV, net present value.

To put the patient engagement–induced changes in NPV and ENPV in context, we can consider changes in these metrics from other activities in drug development. For example, if a sponsor meets FDA requirements for conducting studies in children, the sponsor is granted an additional 6 months’ patent life—a policy known as Pediatric Exclusivity.[58] In the pre–phase 2 base case, an additional 6 months would increase NPV by $15 MM and ENPV by $2.2 MM. In the pre–phase 3 base case, an additional 6 months would increase NPV by $19 MM and ENPV by $9 MM. These increases are considerably smaller than those gained from the patient engagement activities (Tables 3 and 4). Another comparison is in terms of the equivalent ENPV benefit from reducing time to launch. The ENPV increase from the combined amendment/patient experience impact is equivalent to reducing time to launch by 30 months for the pre–phase 2 case and by 17 months for the pre–phase 3 case.

Relationship Between Value Drivers and Patient Engagement

Quantifying the relationship between a particular act of patient engagement and the changes it induces in business value drivers is a challenge, both in estimating the sizes of the changes and in estimating whether there will be any meaningful changes at all. In some cases, such as the avoidance of an amendment, empirical data are sufficient to roughly characterize the probability of avoiding an amendment, cost savings, and time savings likely to be caused by patient group review of a protocol. However, for most forms of patient engagement, such as the impact of improving the trial experience for patients or of developing a more patient-friendly enrollment process, there are insufficient empirical data to estimate the changes in business drivers. In these types of cases, judgment from experience conducting clinical trials and debriefing with patients in trials can provide an estimate.

While amendments are common, not all protocols have them, and it is not evident how often patient engagement activities can avoid amendments. A rough assessment comes from noting that 66% of phase 2 trials and 77% of phase 3 trials have at least 1 amendment. Twenty-two percent of these amendments are classified as “somewhat avoidable” and are due to recruitment difficulty or feedback from sites or investigators.[55] Hence, assuming a sponsor engages with a patient group to review a protocol independent of whether there is a potential need for an amendment, about 70% of the protocols reviewed may have at least 1 amendment, and 22% of these amendments are potentially avoided by the protocol review. Assuming protocols are selected for patient review at random, based on these values, an amendment could be avoided in approximately 15% of cases when a patient group reviews a protocol. This is likely an underestimate, since protocols are more likely to be selected for patient review when there are concerns about the protocol and similar or earlier-stage protocols have not been recently reviewed by patients. However, assuming this 15% rate, the average reduction in development cost is 15% of that for definitely avoiding an amendment, or $85,000 for a phase 2 amendment and $321,000 for a phase 3 amendment. Similarly, the gains in ENPV from avoiding a protocol amendment can be adjusted for this uncertainty. Of all scenarios considered, the smallest gain in ENPV is $3.8MM for a phase 2 patient engagement activity that avoids one amendment but does not result in improvement of patient trial experience (Table 2). The average ENPV gain when only 15% of protocol reviews result in avoidance of an amendment is $720,000 (15% of 4.8MM). For all other scenarios, the ENPV gains are much higher. In all scenarios, the average reductions in cost and the gains in ENPV both greatly exceed a $100,000 investment.

While the limited empirical data for the impact of patient engagement can be frustrating for sponsors looking for a definitive value proposition, particularly for the impact on the patient experience, the ability to test a range of assumptions—ranging from conservative to radical—serves as a sensitivity analysis that provides confidence in a value proposition. In the analyses above, even if the benefits of increased probabilities of technical and regulatory success were far smaller, simply the NPV and ENPV increases from the time savings alone far exceed the investment. For example, if the increase in probabilities of technical success in phase 2 and phase 3 were only 3 percentage points, rather than 5 percentage points as in Table 2, and the increase in probability of regulatory success were 2 percentage points rather than 3, the changes in ENPV in the combined impact scenario in Table 2 is $24MM instead of $35MM—still several orders of magnitude above the investment. Similar sensitivity analyses can be conducted for each assumption, which are planned for future work.

Limitations

In addition to challenges in quantifying the relationships between patient engagement activities and their impact on the value drivers, there are a variety of other limitations to this work. The impact of an amendment on ENPV may be an underestimate. The survey from which the amendment costs were derived noted that companies only reported partial costs of the amendment, suggesting that the actual cost savings would be larger. Additionally, amendments can reduce the number of patients screened and enrolled, potentially diminishing statistical power.[55] Reduced statistical power can lessen the trust sponsors and regulators have in the trial results, potentially reducing the probabilities of technical and regulatory success. The NPV and ENPV changes in Tables 2 and 3 could be considered a conservative estimate from this perspective.

In general, ENPV penalizes projects in early development compared with those in late development, because of the greater risk and longer time to launch of the earlier project. This limitation is not a significant concern when comparing versions of projects in the same stage of development, as is done in this work, but if a sponsor were choosing between investing in patient engagement in projects that are in different phases, this artifact of ENPV will penalize the value of patient engagement for the project earlier in development. One approach to mitigating this concern is using separate budgetary constraints for each phase when considering the allocation of funds to patient engagement.

The modeling in this work assumes that a sponsor is able to fully implement the insights gained from a patient engagement activity. This may not always be the case, especially for novel suggestions that do not fit into a sponsor’s normal development process. In these cases, the gains in business drivers may be less than assessed, with lower gains in NPV and ENPV than described above as a consequence.

Another limitation of our study is that we did not explicitly account for operational, cost projection, or forecast risk. Rather, we used average costs and time for development, assessed by averaging data for oncology compounds in the Tufts Center for the Study of Drug Development database and adjusted for the number of indications (Supplement S2), as well an average revenue model. A more detailed ENPV assessment would account for the uncertainty in these inputs and compute a distribution of ENPV values. Additionally, in many development programs, there are activities and studies that support all the indications, and these are typically associated with the lead indication. This can result in a negative NPV for the lead, though this value is somewhat misleading given that the costs are shared by all the future indications. These issues need to be considered when interpreting an ENPV model.

Most compounds are studied for several, usually related, indications. This is especially true for oncology drugs, where the same mechanism of action can benefit patients for many different cancers. When multiple indications are pursued for a given molecular entity, the benefits of a particular act of patient engagement can increase manyfold if the insights gained from the activity are applied to concurrent or future trials for additional indications. Including line extensions in the analysis would therefore provide additional value from early patient engagement activity.

Finally, there are numerous intangible benefits from patient engagement that cannot realistically be represented with a financial model. The strategic value to a sponsor building a relationship with a patient group, the reputational boost from setting a precedent in patient engagement, patient satisfaction with a sponsor designing a trial using patient input, and similar impacts are challenging to quantify. Using quantitative methods alone to evaluate the impact of patient engagement does not allow for a valuation of the intangibles that are gained through interaction with patients.[59] By excluding the context in which the engagement occurred from its evaluation, the ENPV assessment does not show the potential benefit if the same engagement efforts will work for different projects. Not including this context also misses the wisdom, skills, and relationships that are developed through the engagement efforts. Hence, we believe quantitative evaluation of the effects of patient engagement will generally underestimate the full value of activity.[59]

Follow-up work is planned to develop models for the impact of different forms of patient engagement, sensitivity analyses on the assumptions, assessments in other therapeutic areas, and comparison with real-world case studies.

It has been argued that, while the moral case for patient engagement has been made, evidence of the value of patient engagement is mostly anecdotal.[11-16] For the impact of patient engagement to be fully realized, robust and rigorous study of its quantitative effects on R&D is needed.[60,61] We believe this work will help with understanding and adoption of patient engagement activities.

Click here for additional data file.

Supplementary Material - TIRS716715_Supplement_S1 - Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project by Bennett Levitan, Kenneth Getz, Eric L. Eisenstein, Michelle Goldberg, Matthew Harker, Sharon Hesterlee, Bray Patrick-Lake, Jamie N. Roberts, and Joseph DiMasi in Therapeutic Innovation & Regulatory Science

Click here for additional data file.

Supplementary Material - TIRS716715_Supplement_S2 - Assessing the Financial Value of Patient Engagement: A Quantitative Approach from CTTI’s Patient Groups and Clinical Trials Project by Bennett Levitan, Kenneth Getz, Eric L. Eisenstein, Michelle Goldberg, Matthew Harker, Sharon Hesterlee, Bray Patrick-Lake, Jamie N. Roberts, and Joseph DiMasi in Therapeutic Innovation & Regulatory Science