Application of a decision analytic framework for adoption of clinical trial results: are the data regarding TARGIT-A IORT ready for prime time?

The results from randomized clinical trials are often adopted slowly. This practice potentially prevents many people from benefiting from more effective care. Provide a framework for analyzing clinical trial results to determine whether and when early adoption of novel interventions is appropriate. The framework includes the evaluation of three components: confidence in trial results, impact of early, and late adoption if trial results are reversed or sustained. The adverse impact of early adoption, and the opportunity cost of late adoption are determined using Markov modeling to simulate the impact of early and late adoption in terms of quality of life years and resources gained or lost. We applied the framework to the TARGIT-A randomized clinical trial comparing intraoperative radiation (IORT) to standard external beam radiation (EBRT) and considered these results in the context of trials comparing endocrine therapy with and without radiation therapy in postmenopausal women. Confidence in the TARGIT-A trial 4 year results is high because the peak hazard for local recurrence in the trial is between 2 and 3 years. This is consistent with most trials, and no second peak has been observed in similar patient populations, suggesting that the TARGIT-A trial results are stable. The interventions offer approximately equivalent life expectancy. If IORT local recurrences rate were as high as 10 % at 10 years (which is higher than expected), we would project only 0.002 fewer expected life years (less than 1 day) compared to EBRT if IORT is adopted early. However, there is a $1.7 billion opportunity cost of waiting an additional 5 years to adopt IORT in low risk, hormone-receptor-positive, postmenopausal women. EBRT costs an additional $1467 in indirect costs per patient. Applying an evaluative framework for the adoption of clinical trial results to the TARGIT-A IORT therapy trial results in the assessment that the trial results are stable, early adoption would lead to minimal adverse impact, and substantially less resource use. Both IORT and no radiation are reasonable strategies to adopt.

Introduction

Clinical adoption should occur when high-level randomized data clearly show the efficacy of one treatment against another, without serious adverse effects. However, several factors other than evidence regarding these outcomes influence the adoption of new findings from clinical trials. For example, when less treatment is shown to have an equivalent outcome, change in practice is highly variable. It is typically easier to adopt or add new agents or treatments, but can be hard to abandon what is seen as standard treatment.

In the United States, new business for physicians, hospitals, and manufacturers that result from a positive trial often helps drive adoption. However, when new approaches disrupt the routine of practice, or when interventions reduce the use of particular services, there may be less incentive for early adoption [1]. Reasons for slow adoption include financial disincentive, disruption of practice routines, resistance to change, fear of abandoning a standard treatment, and skepticism about trial results [1].

The question of how to approach patients with early stage, favorable risk breast cancer illustrates the complexity of forces influencing decision making with regard to the adoption of new approaches. Whole-breast external beam radiotherapy (EBRT) remains the standard of care following breast-conserving surgery. However, multi-dose partial breast radiation is increasingly offered as an alternative for eligible women. Emerging technologies have provided impetus for shifts in radiation approaches despite lack of randomized clinical trial data for these devices [2]. The frequency of brachytherapy use increased from ~1 % in 2001 to 10 % in 2006, despite concerns about long-term efficacy [3].

Meanwhile, the finding from a randomized trial (CALGB 9343) that older women with hormone-receptor-positive breast cancer could be effectively treated with tamoxifen without radiation therapy have yet to be adopted into clinical practice [4]. The results demonstrated that with or without radiation, distant recurrence, breast cancer mortality, and mastectomy rates are the same and very low in the two arms [5]. Despite these results with more than 10 years of follow-up, and corroborating evidence from a similar Canadian trial in all postmenopausal women, [6] radiation is rarely omitted and post-lumpectomy radiation is considered a quality measure by the American College of Surgeons for older women [7]. Fear of omitting therapy and being less aggressive often makes both physicians and patients uncomfortable, even in the face of supporting evidence to the contrary. There is a cultural bias which dictates that more aggressive treatment for cancer is better, and this bias colludes with the fear of malpractice and financial rewards to encourage intervention.

With this as prelude, it is not surprising that when the results of the international randomized trial comparing a single intraoperative radiation (IORT) using low-energy photons delivered by the TARGIT device were published in 2010, there was a great deal of criticism and arguments that it was too early to adopt the findings [8]. Indeed, early adoption of technology that turns out to be inferior to the status quo can be harmful, but late adoption of technology that turns out to be equivalent or superior to the status quo can be a missed opportunity that also harms individuals and society. Clearly, a rational approach is needed to put early trial results in perspective and facilitate adoption decisions.

We propose a decision framework for the adoption of clinical trial results that includes three components. First, we ex amine the level of confidence we have in the trial results, second, we predict the impact of early adoption of trial results, and third, we predict the impact of late adoption of those trial results. Predicted impact is measured through life expectancy, quality-adjusted life years (QALYs), and cost. We apply the proposed framework to the example of the TARGIT-A intraoperative radiotherapy trial to illustrate this approach in the hopes of providing a better platform for decision making for the medical community.

Methods

The adoption framework (shown in Fig. 1) relies on predictive modeling, which can aid policy decisions by projecting possible outcomes. Within this framework, sensitivity analyses are used to determine the impact of initial assumptions and how possible changes in data might impact conclusions from clinical trials.

Fig. 1

Overview of the adoption framework—the decision of whether or not to adopt trial results is based on consideration of the confidence in the trial results and an assessment of the impact of early versus late adoption of the results according to life expectancy, QALYs, and cost

The example

The TARGIT-A trial randomized women with invasive hormone-positive ductal breast carcinoma (<3.5 cm) undergoing breast-conserving surgery to receive either one-time targeted IORT (TARGIT) or whole-breast EBRT [9]. In a risk-adapted design, women with invasive lobular, hormone-receptor-negative, or positive nodes received EBRT in addition to TARGIT. Early results were published in The Lancet in 2010: the Kaplan–Meier estimate of local recurrence in the conserved breast at 4 years was 1.20 % (95 % CI 0.3–2.71) and 0.95 % (0.39–2.31) for TARGIT and EBRT arms, respectively [9]. The IORT treatment was deemed not inferior to EBRT as the IORT recurrence rate was within the prospectively defined non-inferiority margin of 2.5 %. (difference between groups 0.25 %,–1.04 to 1.54; p = 0.41). Toxicity (Radiation Therapy Oncology Group grade 3) was lower in the TARGIT arm. The updated results 2 years later show a 2 % difference overall. When TARGIT is applied at the time of lumpectomy as a single surgery, there is no difference in the local recurrence rates when compared to EBRT [10].

The framework examines the following three questions:

(1) What is the likelihood that the trial results are stable?

Analyzing the hazards for disease recurrence or progression over time to determine if the hazard for recurrence has peaked is one way to generate confidence in whether the trial results will change. The trial in question should also be compared to results from other published trials with similar patient populations, to determine consistency of results. A more rigorous alternative is to predict the Bayesian likelihood or predictive probability that the trial results will change.

(2) What is the impact of early adoption?

Predictive modeling was used to quantify the negative impact of adopting too early and the possible changes in results—such as an increase in toxicity or recurrence rate—on life expectancy, QALYs, and cost. Sensitivity analyses were conducted to predict outcomes over a range of possible costs, probabilities, and utilities.

(3) What is the impact of late adoption?

Predictive modeling was also used to determine the cost associated with delaying the adoption of a less expensive treatment. Opportunity cost to an individual may include lost wages, lack of convenience, or decreased life expectancy or QALYs. Societal costs include the aggregate cost of the more expensive treatments during the period that the adoption was delayed.

We applied the adoption framework to the published 4-year results of the international TARGIT-A to illustrate how the proposed framework can be used to inform policy. We used Markov modeling to simulate life expectancy, QALYs, and costs as a cohort of individuals’ transitions between expected health states over a relevant time period [11, 12] The model uses 2011 Medicare reimbursement rates, discounted at 3 %, and published utility values [13].

Results

(1) Confidence in the long-term durability of early trial results

The peak of local recurrence occurred at 3 years in the TARGIT-A trial and the annual hazards for recurrence in both arms were quite low [9]. The hazards for local recurrence were 0.0, 0.3, 1.2, and 0 versus 0.0, 0.15, 1.1, and 0.0 for IORT and EBRT for years 1–4, respectively. More than 1,500 women had a median follow-up of 3 years, and more than 800 women had a median follow-up of 4 years. Comparing to trials with similar postmenopausal populations and longer follow-up periods, we see peak hazards at similar times. The adjuvant arimidex versus tamoxifen versus combination study, as well as the early breast cancer trials meta-analysis also show early peak hazards for local risk of recurrence (LRR) between 2 and 3 years and within 2 years, respectively. No second LRR peaks are seen within the 10-year follow-up period [14, 15]. In another IORT trial of 1,200 patients randomized to radiation versus no radiation (ELIOT), median local recurrence occurred at 26 months [16]. The British Association of Surgical Oncology II trial, which randomized 1135 low-risk patients to XRT vs. tamoxifen using a 2 × 2 design, found that either EBRT or tamoxifen was associated with a 10-year survival of 96 %, both 100 %, and none 87 %. The annual % local recurrence rate was 2.1, 0.7, 0.8, and 0.0 for no treatment, XRT alone, tamoxifen alone, or both, respectively [1]. Therefore, for postmenopausal women with low risk hormone-positive cancers, the local recurrence rates seen in the TARGIT-A trial are highly likely to remain low.

(2) Potential adverse impact of early adoption

The adverse impacts of early adoption were predicted using a Markov model to simulate the outcomes (life expectancy, QALYs, and cost) of the three different treatment options, IORT, EBRT, or no radiotherapy (tamoxifen only). The 4-year TARGIT-A results were used as baseline inputs for the model and then varied. We used the “tamoxifen only” results from the tamoxifen alone versus tamoxifen plus EBRT to establish a 10-year LRR upper bound [6] to be used in the sensitivity analyses in our Markov model as the vast majority of patients with hormone positive disease receive some form of endocrine therapy (either tamoxifen or aromatase inhibitors). Toxicities were included as previously described [13]. Mortality rates were assumed to be the same in the two treatment arms as has been observed from trials of XRT versus none with a similar population of patients (node-negative and hormone-receptor-positive) [5, 6, 17].

Life expectancy

At the projected IORT LRR (3 % over 10 years), the two treatments are essentially equivalent in terms of life expectancy, with IORT resulting in an expected 0.0002 fewer life years (less than 2 h) than EBRT. The LRR was varied well beyond the published annual rate from the TARGIT-A trial in sensitivity analysis [9]. Even at the highest clinically relevant 10-year LRR of 10 % (the predicted difference in LRR without any radiation), the impact on life expectancy would be just 0.002 fewer life years (less than 1 day) with IORT compared to EBRT (Fig. 2). This assumes all patients receive hormonal therapy.

Fig. 2

Predicted life expectancy based on a possible range of LRR values for IORT (assuming no change in LRR for EBRT). LRR for “No RT” is assumed to be 10 % at 12 years based on the CALGB 9343. So when the LRR for IORT is extended beyond 10 %, it appears that IORT life expectancy is worse than “no RT”, but that is artificial. This figure demonstrates that the impact of early adoption, even if the results change significantly (5 × more than expected to 10 %), is negligible. The 10-year IORT LRR projected from the results published in 2010 and 2012 are 3 and 6 %, respectively

Quality of life impact

Side effects occurred less frequently with the single intraoperative treatment, Radiation Therapy Oncology Group grade 3) events were lower in the IORT group than in the EBRT group (0.5 and 2.1 %, p = 0.002). This was not incorporated into the analysis, but favors application of IORT given that there is no adverse impact on life expectancy.

(3) Impact of late adoption

Cost to society

To calculate the opportunity cost to society of delaying adoption of trial results by an additional 5 years, we first determined the magnitude of the population for whom this technology would be applicable. We estimated that 70,136 women are eligible each year for IORT or no RT (N0, grades 1 and 2 ductal cancers in postmenopausal women >50 years old) from SEER and US Census Bureau data (Table 1).

Table 1

Projected incidence of breast cancer in the US in women of all races, limited to grade 1 and 2, node-negative

Ages	Frequency of grade 1 or 2, node-negative breast cancer (%)	Projected incidence, US per year
50–54	37.0	8,512
55–59	39.7	9,835
60–64	43.5	10,390
65–69	47.9	10,268
70–74	49.4	9,601
75–79	49.7	9,320
80–84	49.3	7,236
85+ years	43.1	4,974
Total		70,136

For each age group, the percent of patients meeting these criteria is shown, and the absolute number of women is projected based on annual incidence of breast cancer in the United States (SEER 2003–2007)

Early adoption of the TARGIT-A trial results after the report of the 4-year outcomes as opposed to waiting the 10 years was calculated to save the health care system $1.7 billion. If the same women avoided RT altogether the savings are expected to be $3.2 billion.

In calculating the incremental cost-effectiveness ratios (ICERs) for each of the three treatment arms (EBRT, IORT, and no radiation therapy), we found that EBRT was the most costly option, offered the least QALYs, and was, therefore, dominated by both IORT and the absence of any radiation therapy. The ICER for the IORT compared to no radiation therapy was $10,186/QALY.

Cost to patient:

Patients face numerous indirect costs that may have a significant impact on their daily lives. Instead of the one-time intraoperative dose of radiotherapy with IORT, patients who receive EBRT must travel to a radiation therapy facility every day for upto 3–6 weeks. We incorporated these additional burdens, including miles traveled and wages lost, as indirect costs in the Markov model for the EBRT [13]. As estimated from the literature, total treatment time (travel, parking, and treatment) was modeled to be 2 h per treatment. The average hourly 55-year-old female wage ($13.68) was taken from the US Bureau of Labor Statistics and adjusted to 2011 US dollars. The travel distance was estimated to be 20 miles at 19 cents/mile. Per-day costs were added for 28 treatments for 5 weeks of EBRT following IORT and 33 treatments for the 6-week EBRT regimen. The total indirect cost to a patient of undergoing EBRT instead of IORT or no radiation therapy was $1467 (this amount was included in the $1.7 billion stated above).

Summary of results (Fig. 3)

Fig. 3

Adoption framework: summary of results

Given that the peak hazard for local recurrence is early in all postmenopausal women, we would not expect the result of the TARGIT-A trial to change. The results of CALGB-9343 are now well past 10 years, and the 5-year results indeed predicted the 10.5-year results. IORT is not expected to be associated with an adverse impact on life expectancy, even in the highly unlikely event that the long-term results did change. There would be no increase and likely a decrease in toxicity. In contrast, continuing with EBRT results in a $1.7 billion opportunity cost compared with early adoption of the TARGIT-A trial results.

Discussion

Decision making in low-risk patients is often more complicated than for those with high risk. The reason is that the benefit from an intervention and its side effects may not outweigh the risk of the disease. Thus, a postmenopausal woman with a favorable risk hormone-receptor-positive breast cancer has several options after breast conservation, including hypofractionated EBRT, IORT, or no radiation, which are all supported by the literature as at least equivalent, if not superior, strategies to EBRT [18, 19]. And yet, trial results have not significantly influenced practice in the United States. When the CALGB 9343 results were presented in 2002, they met a similar reaction to the results of the TARGIT-A results in 2010. Many demanded 10 years of data prior to adoption.

In the CALGB 9343 trial, the findings reported at 5 years were predictive of the results at 10.5 years (see Table 2). In June of 2005, CALGB 9343 was presented, showing equal impact on life expectancy of EBRT versus no radiation in settings of hormone-receptor-positive women age 70 or older. In December of 2010, the CALGB 9343 10.5-year results were published but the trial results have yet to be adopted, suggesting that the factors other than evidence and patient benefit are driving practice changes. This suggest that we need more rigorous methods of how we evaluate trial results and how we choose to use them. The results of the potential harm and opportunity cost from failure to adopt early are striking and suggest that we should be doing less for women with lower risk tumors.

Table 2

The results of local recurrence rates at 5, 10, and 10.5 years of follow-up in recent clinical trials

STUDY	Accrual dates	N	Study arms	5 year LRR (%)	10 year LRR	12.6 year LRR
TARGIT-A 2010	2000–2012	2,232	IORT	1.20a	N.A.	N.A.
			EBRT	0.95a
Studies comparing XRT versus NO XRT
CALGB C9343 Hughes et al. [5]	1994–1999	636	Tam	4	7	10
			Tam+RT	1	1	2
Fyles et al. [6, 26]	1992–2000	611 (all T1 patients)	Tam	5.5	13.8b	N.A.
			Tam+RT	0.4	5.3
		114 (subset of G1/2, lum A patients)	Tam	2c	4.9	N.A.
			Tam+RT		5.5
Studies comparing hypofractionated 3-week EBRT versus 5-week EBRT
Whelan et al. [19]	1993–1996	1,234	3 week EBRT	2.33d	6.2	N.A.
			5 week EBRT	2.17d	6.7
START-B 2008	1999–2001	2,215	3 week EBRT	2.2e	NA	NA
			5 week EBRT	3.3e

Patient populations were largely postmenopausal women with node-negative, hormone-receptor-positive, early stage invasive breast cancer. Two of the trials, Hughes et al. and Fyles et al., compared treatment regimens of XRT versus no XRT. The other two trials, Whelan et al. and START-B, compared standard 5-week EBRT to hypofractionated 3-week EBRT

a 4-year follow-up data

b Includes ER-patients with a higher recurrence rate than ER + patients

c The ER+/HER2-subset was presented at ASCO in 2002

d No significant difference between treatment groups

e Rates are for locoregional recurrence, not local recurrence

Over time, the local recurrence rates have been coming down from the original reported risk of 10 % after IORT. Recent studies show rates largely less than 5 % and even lower when endocrine therapy is used. Data from 4 trials with populations of similar biology (postmenopausal women with node-negative, hormone-receptor-positive, early stage breast cancer) are shown in Table 2. Local recurrence rates are low even in the less intensive EBRT arms or the complete absence of radiation. Fyles et al. found that women over 50 in the low risk group—immunohistochemistry subtype luminal A—had a local recurrence of 4.9 % at 10 years. Thus the IORT results, which show a very low LRR, are consistent with other modern trial results. It should be noted that there was no difference in metastatic rates or overall survival for any of the trial arms.

We did not account for the cost of purchasing the IORT equipment, which would be an added cost if the LRR eventually is shown to be equivalent to no radiation. Equipment, however, can be leased and not purchased and is included in the cost of IORT. We also did not have the ability to assess the utility of avoiding 3–6 weeks of daily radiation, and therefore, likely underestimated the improvement in QALYs with IORT.

The 2012 TARGIT-A trial update shows a very small but statistically significant increase in non-breast cancer (cardiac) mortality for EBRT compared to IORT [10]. Our analysis assumes that IORT and EBRT are expected to be equivalent in terms of life expectancy; an improvement in expected mortality for either treatment will shift the results to favor that strategy. If over time, the small mortality benefit in favor of IORT persists, the IORT would be favored on the basis of mortality benefit as well.

Breast cancer is now recognized as being comprised of several distinct diseases. One of the reasons that the LRR appears to be so much lower is likely due to the impact of mammography screening and the identification of tumors that are biologically more indolent [20-22]. The results from older trials where there was a 40 % LRR in the absence of radiation applied to younger women and tumors with more aggressive biology [23]. Thus, reframing risk and options must occur as we have the ability to better characterize the biology of newly diagnosed breast cancers [24] and offer better options for women with low risk disease, and where recurrence is not life threatening.

The risk for distant breast cancer recurrence for hormone-receptor-positive women does extend over a 20-year period of time [25]. This well-known fact leads to an erroneous conclusion that the TARGIT-A study results were “too early” to adopt. The statisticians were confident in the TARGIT-A results because data clearly show that the peak hazard rate for local recurrence is early, and is then low and stable, thus the overall conclusions are extremely unlikely to change. This assumption was further validated by the recent update of the TARGIT-A results, where the conclusions of non-inferiority remain unchanged for patients treated at the time of initial resection [10].

The demonstration that less aggressive interventions are equally effective for women with lower risk tumors is a critical breakthrough for women and a major advance in our ability to tailor treatments for women according to the biology of their tumors. If we can safely accomplish the same goal (preventing cancer recurrence) in a much more efficient, less invasive, and less personally time-consuming manner for women, the physician community should be the first to embrace this therapy. The complaint about IORT seems to be that the data are immature; however, clinical practitioners can switch to IORT and watch the data mature. The chance that the results will change has a very small probability, and practitioners could switch back to EBRT, or switch to something that in the interim has been shown to be better than EBRT. As our analysis indicates, little will have been lost for any patient who receives IORT, as many are also eligible for hormone therapy alone.

We have an opportunity to take an important step forward in patient-centered care and to address the challenges women face in returning for daily radiation treatments over many weeks. It is reasonable to assume that patients would prefer no radiation, IORT or shorter courses of EBRT even simply on the basis of convenience. If women are concerned only about distant recurrence, no radiation is the appropriate treatment. If a 5–10 % difference in local recurrence is important to avoid, IORT is an excellent alternative to no radiation.

Having a robust method to ascertain the reliability of results enables the community to move forward with adopting new treatments, especially if we are confident that they are safe. If they are also far less expensive, time and resource-intensive, and could impact hundreds of thousands of women in the US, Europe, and Australia, that should be added incentive to drive adoption.

There is no point in putting the effort into funding and conducting randomized clinical trials if at the end of the process we feel uncomfortable making a change and allowing new evidence to change our practice. It is reasonable to question the results. However, we suggest that rigor in evaluating the impact of trials can make a difference in the policy for adoption by looking at the likely stability of results, the impact of early adoption if wrong and the opportunity cost of late adoption.