The effect of the frequency and duration of PSA measurement on PSA doubling time calculations in men with biochemically recurrent prostate cancer.

BACKGROUND: PSA doubling time (PSADT) is an attractive intermediate end point for assessing novel therapies in biochemically recurrent prostate cancer (BRPC). This study explores whether PSADT calculations are influenced by frequency/duration of PSA measurements, and whether statistical variability leads investigators to find false significant results.
METHODS: In retrospective analyses of two BRPC cohorts: Johns Hopkins Hospital (JHH) patients who deferred therapy and placebo patients on a randomized clinical trial (RCT), we calculated changes in PSADT from early measurements to later measurements using subsets of available PSAs for patients with ≥6 and ≥9 PSAs. We simulated hypothetical single-arm trials using randomly selected, 50-patient subsets and simulated two-arm RCTs.
RESULTS: JHH cohort (n=205) had median follow-up 58 months, median age 61 years and median Gleason 7. PSA variability changed with duration of PSA measurement as median within-patient PSADT increases for men with >6 PSAs ranged from 1.0 to 1.4 months by PSA subset while increases for men with ≥9 PSAs ranged from 3.9 to 4.1 months. Frequency of measurement did not change PSA variability as PSADT increase was unchanged when odd values were used instead of all values. Approximately 30% of JHH men experienced >200% increases in PSADT. Up to 62% of 50-patient single-arm simulations detected a significant PSADT change, whereas simulated RCTs did not. Results were supported in the RCT placebo cohort; 46% of patients experienced PSADT increases >200%.
CONCLUSIONS: These data suggest that calculated PSADT in BRPC may naturally increase over time in the absence of therapy and may be influenced by duration of PSA follow-up. As a result, single-arm trials could show false significant increases despite the lack of active treatment of these patients. Placebo-controlled RCTs including clinical end points are recommended to screen novel agents in men with BRPC to mitigate bias because of natural PSADT variability.

RCT Entities:   Population Interventions Outcomes

Introduction

Approximately one-third of prostate cancer patients who undergo primary local therapy will experience biochemical relapse documented by rising prostate specific antigen (PSA) level.[1-3] Salvage radiation may be curative in a portion of prostatectomy patients who experience PSA recurrence.[4] However, PSA recurrence following local therapy and salvage treatment is indicative of micrometastatic disease that is no longer curable.[5] Because of the high sensitivity of PSA measurement, most biochemically recurrent prostate cancer (BRPC) patients do not initially present with any radiological or physical evidence of local recurrence or distant metastases. In BRPC patients, treatment is aimed at slowing cancer progression. More than 30 clinical trials have evaluated treatments for BRPC patients.[6] These treatments range from drugs approved for other indications, Dutasteride;[7] Gefitinib;[8] to natural products including soy[9] or diet and exercise;[10] to novel targeted agents including immunotherapies and antibodies.[11]

PSA doubling time (PSADT), defined as the length of time [in months] for the PSA level to double,[5] is a controversial endpoint in clinical trials of BRPC patients. Using PSADT as an intermediate endpoint in this population is attractive because clinical endpoints (i.e. metastasis-free survival or overall survival) are prolonged, often making trials infeasible. Retrospective studies have shown that PSADT is a strong predictor of metastasis-free survival,[12,13] overall survival,[2] or both.[14] However, patients on placebo arms of two randomized controlled trials (RCT) experienced a significant increase in PSADT that was not induced by any active therapy. In a study of rosiglitazone (n= 106), 73% of placebo patients experienced a >100% increase in PSADT and 31% experienced a >200% increase in PSADT,[15] and in a study of celecoxib (n=78) patients, 20% of placebo patients experienced a >200% increase in PSADT.[16] PSADT increases of these magnitudes in patients without medical intervention suggest natural and significant variation of PSADT in BRPC patients. More clinically meaningful, accurate conclusions may result from clinical trial designs informed by the availability of reliable information on the natural variability of PSADT, especially the magnitude of statistical variability in determining PSADT.

To this end, we performed a retrospective analysis of men with BRPC who chose to defer hormonal therapy and are not undergoing other known treatments for prostate cancer until after the occurrence of metastases. We had three goals: (1) to characterize the natural history of PSADT using a group of patients seen by medical oncologists at the Kimmel Cancer Center at Johns Hopkins (JHH) and a similar patient population on the placebo arm of a phase II clinical trial; (2) to examine the ramifications of statistical variability in PSADT measurement on the outcome of a series of simulated clinical trials; and (3) to determine whether PSADT calculations in these patients could be influenced by the frequency and duration of PSA measurements.

Methods

We performed a longitudinal observational retrospective analysis of two groups of patients: (1) “JHH medical oncology patients”—prostate cancer patients seen by JHH medical oncology staff between 1990 and 2010; and (2) “placebo patients”—prostate cancer patients who were randomized to the placebo arm of a phase II RCT of an experimental targeted non-hormonal agent conducted in 41 sites in the United States and Canada.[11]

Johns Hopkins Hospital Medical Oncology Patients

This retrospective analysis used data extracted from the JHH cancer registry, selecting prostate cancer patients whose paper or electronic records showed medical oncology visits between January 1990 and June 2010 and who had undergone local therapy with prostatectomy or radiation (n=1,973). All charts were reviewed to identify those that indicated the patient was experiencing BCRP, and that had at least 6 PSA measurements ≥ 0.2 ng/ml at least 21 days apart when the patient was not on systemic therapy. Of those excluded, 1,569 patients were undergoing systemic therapy (ADT or clinical trial), and 199 did not have 6 eligible PSA measurements (Figure 1). No patient’s PSA measurements were included in the trial until he experienced biochemical relapse following completion of local therapies including surgery or radiation. Thirty-six patients had salvage therapy at least one year prior to their first included PSA measurement and 9 patients had salvage therapy following the last included PSA measurement. For patients who had androgen deprivation therapy (ADT) with local therapy, PSA values were excluded for 12 months following the completion of ADT. PSA values were included on study until the patient was placed on therapy (e.g. ADT or clinical trial). The protocol was approved by the JHH Institutional Review Board.

Figure 1

Rising PSA Patient and Subset Selection

Six PSA measurements were required for inclusion because we were simulating clinical trials in the BRPC patient population. Inclusion criteria for BRPC trials routinely require 3 PSA measurements for PSADT calculation prior to enrollment and at least 3 additional PSA measurements on study.

Phase II Clinical Trial Placebo Patients

We obtained PSA measurements from the clinical trial database accumulated from the clinical case reports of 108 patients randomized to the placebo arm of a double-blind, placebo-controlled phase II trial of an experimental targeted agent in prostate cancer patients with biochemical recurrence after radical prostatectomy. One patient was removed from the analysis, as he had no on-treatment PSA values. “On-treatment” refers to the period during which the patients were receiving placebo pills while enrolled on a clinical trial. Eligibility criteria for the clinical trial included BRPC after radical prostatectomy, and baseline PSA between 0.3–6 ng/mL with a PSADT <12 months based on two PSA values at least two weeks apart. Median treatment duration was 21.9 months. For the 46 patients who had undergone salvage radiation therapy, between 3 months and 10 years (median 2.3 years) elapsed from the end of radiation therapy to eligible PSA levels. Eligibility for inclusion in this PSA variability study was ≥3 on-treatment PSA values collected every 12 weeks while on placebo. All placebo patients had completed all potentially curative therapies including radiation, surgery and salvage therapy. PSA measurement terminated when hormone therapy began or when the men terminated participation.

Statistical Analysis

PSADT was calculated for the 205 JHH and 108 placebo patients, using selected subsets of available post-baseline PSA values, including odd-numbered visits, initial one-third of recorded values, last two-thirds, first-half, last-half, first-three, and remaining recorded values. To examine the influence of the frequency of PSA measurement on PSADT, we calculated median within-patient change (difference) in PSADT for complementary combinations of subsets of available PSA values (first-half versus second-half, first-third versus second two-thirds, first three versus remainder, and first three versus remaining odd-numbered values) (Figure 1). We also calculated median within-patient change (difference) in PSADT for patients where the baseline values were all recorded in the first 12 months after biochemical recurrence, with post-baseline values recorded in the following 24 months. To examine the influence of the number of PSA measurements, we performed the within-patient change in PSADT calculation separately for patients who had ≥6 PSA values and those who had ≥9 PSA values. PSADT was calculated using the formula PSADT = ln(2)/slope of the linear regression fit of log PSA versus time. Patients experiencing declining PSA values (negative PSADT) were assigned a PSADT of 100 months and included appropriately in the analyses.[2,6]

To determine how frequently researchers conducting a hypothetical uncontrolled clinical study of an ineffective therapy would encounter a significant change between the subsets of PSA values within patients, we performed 2,000 simulations of hypothetical single-arm, 50-patient studies using change in PSADT from baseline as the primary outcome. The computer simulations of hypothetical clinical studies used the actual PSA follow-up data in our database. For each simulated single-arm clinical study, we took a random sample of 50 men in our database. For each sampled man, we computed a hypothetical pre-study PSADT and on-study PSADT and the difference using his observed PSAs in the database. For the 50 patient sample, we then tested whether this difference was zero with the Wilcoxon Signed Rank test, calling a difference significant if the one-sided p value was less than 0.05. We repeated these hypothetical clinical studies 2000 times and determined the proportion of these simulated studies that reached statistical significance. If there is no difference between the pre-study and on-study PSADTs, only 5% of the simulated studies should reach statistical significance. Simulations of randomized clinical trials followed a similar procedure, except that these simulations also included assigning each sampled individual to one of two hypothetical treatments. All analyses and simulations were carried out in the R statistical environment. We also calculated the percentage of patients who experienced >100% and >200% increases in PSADT.

Results

Patient Demographics

Clinical features of the 313 men with BRPC who formed our two analysis cohorts are summarized in Table 1. For the JHH cohort (n=205), the median baseline PSADT was 13.8 months (range: 2.1 to 569 months). The majority of patients had Gleason scores ≤7, with only 19% above 7. Seventy percent of the JHH patients had radical prostatectomy as their primary therapy, while the rest had radiation as initial therapy. Median follow-up was 58 months and mean and median number of PSA measurements were 12.8 and 10 respectively.

Table 1

Patient Demographic and Clinical Characteristics

Characteristic	Johns Hopkins Hospital Data Set	Placebo Data Set
Total Number of Patients (number)	205	108
Age at Diagnosis (years)
Mean	61.5	64.6
Range	45–82	44–86
SD	7.2	7.8
Median	62	65
Unknown	23	-
Ethnicity (number and percent)
White	156 (76%)	NA
African American	43 (21%)	NA
Asian/Other/Unknown	6 (3%)	NA
Pre-Operative Gleason Score (number and percent)
Low grade (score 4–6)	61 (30%)	26 (24%)
Intermediate grade (score 7)	82 (40%)	53 (49%)
High grade (score 8–9)	38 (19%)	27 (25%)
NA	24 (12%)	3 (3%)
Mode of Primary Therapy (number and percent)
Radical prostatectomy	141 (69%)	108 (100%)
Radiation therapy	48 (23%)	-
Radiation and hormone therapies	12 (6%)	-
Other	4 (2%)	-
Salvage Therapy (number and percent)*
Radiation Therapy	44 (31%)	46 (42%)
Cryotherapy	1 (1%)	0
Follow-up Time (months)**
Mean (Standard deviation)	66.0 (42.0)	22 (8.0)
Range	5.2–184.8	5.8–41.7
Median	58.1	21.9
Initial PSA Value (ng/mL)
Mean (Standard deviation)	1.2 (2.1)	1.4 (1.2)
Median	0.5	0.9
Range	0.2 – 14.5	0.3 – 6
Baseline PSADT (months)***
Mean (standard deviation)	27.8 (51.8)	10.2 (14.4)
Range	2.1 – 568.7	1.0 – 100.0
Median	13.8	7.1

9 of the 45 patients who underwent salvage therapy did so after the end of the study; the remainder completed salvage therapy at least 1 year prior to the beginning of the study.

Follow-up time is defined as the time difference between a patient’s last recorded PSA and his first PSA value used in this study. Negative PSADTs are set to 100.

Baseline for JHH was based on 1st 3 PSA values.

Abbreviations: NA = not available; PSA = prostate specific antigen; PSADT = PSA doubling time.

For the patients on the placebo arm of the phase II clinical trial (n=108), median baseline PSADT was 7.1 months (range: 1 to 100 months). The majority of patients had Gleason scores of ≤7, with 25% ≥7. All of the placebo patients had radical prostatectomy as their primary therapy; median follow-up was 22 months.

PSADT Changes in Johns Hopkins Medical Oncology Patients

In the cohort of JHH medical oncology patients with ≥6 PSA measurements (n=205, Table 2), the median “post-baseline” to “baseline” within-patient increase in PSADT was 1.0–1.4 months. More than 50% of the men in this cohort experienced ≥100% increase in PSADT and approximately 30% of the men experienced ≥200% increase in PSADT. The frequency of significant PSADT lengthening using differences in simulated single-arm studies ranged from 17% to 37% using Wilcoxon Signed-Rank Tests to test for increased PSADT. Unlike the single-arm studies, the simulated randomized trials declared a significant difference around 5% of the time, as expected for a 5% level test. Of note, seven of 205 patients in the JHH cohort experienced declining PSA values (negative PSADT) in the post-baseline period.

Table 2

PSADT in Months Using Data from JHH Oncology Patients Who Have at Least Six Recorded PSA Values

		Mean PSADT (standard deviation)	Median PSADT	Median Within-Patient PSADT Difference	Percent of Simulations Significant	Number (%)of Men with > 100% Increase	Number (%)of Men with > 200% Increase
	All Values (n = 205)	25.1 (30.6)	16.6
First Half versus Second Half of Values	First half	21.6 (19.9)	16.8
	Second half	40.7 (99.6)	17.0
	Within-Patient PSADT Difference	19.1 (95.5)		1.0	37%	114 (56%)	56 (27%)
First Three Values versus Remainder of Values	First Three	27.8 (51.8)	13.8
	Remaining Values	29.1 (40.1)	18.5
	Within-Patient PSADT Difference	1.2 (61.3)		1.4	17%	114 (56%)	65 (32%)

For JHH patients with ≥9 PSA values (n=127, Table 3), increases in median within-patient PSADT ranged from 3.9 months in the “first-third versus second two-thirds” comparison to 4.1 months in the “first-three PSA values versus the remaining-PSA-values” comparison. The difference in median within-patient PSADT was unchanged at 4.0 months when we removed every other value from the remainder (thereby comparing the first three values with remaining odd-numbered values). Significant differences were found in 27% to 62% of 50-patient simulated trials when comparing median within-patient PSADT differences. Furthermore, approximately 60% of the men in this cohort experienced >100% increase in PSADT and 28%–38% of the men experienced >200% increase in PSADT.

Table 3

PSADT in Months Using Data from JHH Oncology Patients Who Have at Least Nine Recorded PSA Values

		Mean PSADT (standard deviation)	Median PSADT	Median Within- PSADT Difference	Percent of Simulations Significant	Number (%)of Men with > 100% Increase	Number (%)of Men with > 200% Increase
	All Values (n = 127)	31.9 (36.1)	22.2
First Half versus Second Half of Values	First half	26.0 (22.4)	19.8
	Second half	51.2 (122.4)	22.1
	Within-Patient PSADT Difference	25.2 (117.9)		4.1	62%	78 (61%)	36 (28%)
First Three Values versus Remainder of Values	First Three	34.5 (63.3)	16.1
	Remaining Values	35.1 (46.2)	22.3
	Within-Patient PSADT Difference	0.6 (74.2)		4.1	27%	78 (61%)	48 (38%)
First Third versus Second Two- Thirds of Values	First Third	33.4 (57.9)	18.0
	Second Two-Thirds	39.1 (55.0)	23.2
	Within-Patient PSADT Difference	5.7 (74.0)		3.9	40%	75 (59%)	45 (35%)
First Three Values versus Remaining Odd- Numbered Values	First Third	34.5 (63.3)	16.1
	Remaining Odd- Numbered	34.0 (37.7)	23.3
	Within-Patient PSADT Difference	−0.5 (67.5)		4.0	35%	78 (61%)	45 (35%)

PSADT Changes in Placebo Patients

Median treatment duration for patients on the placebo arm of the phase II study was 16.3 months (Table 4). The median baseline PSADT was 7.1 months while the median on-treatment PSADT was 14.9 months. The median within-patient increase in PSADT from baseline to on-treatment was 5.9 months. Increases in PSADT of >100% and >200% occurred in 79% and 46% respectively, for these patients on placebo.

Table 4

PSADT in Months Using Data from Placebo Patients (in months)

		Mean PSADT (standard deviation in parentheses)	Median PSADT	Median Within-Patient PSADT Difference	Percent of Simulations Significant	Number (%)of Men with > 100% Increase	Number (%) of Men with > 200% Increase
Pretreatment versus On- treatment Values	All Values (n = 108)	26.2 (62.1)	12.7
	Pretreatment	10.2 (14.4)	7.1
	On-treatment	30.1 (40.5)	14.9
	Within-Patient PSADT Difference	19.9 (42.5)		5.9	100%	85 (79%)	50 (46%)

Discussion

This retrospective analysis of patients with BRPC after definitive local therapy illustrates the variability in the natural history of PSA and has significant implications for clinical trial design and interpretation in this population. A hypothetical clinical study of untreated BRPC patients seen by JHH medical oncologists between 1990 and 2010 could show PSADT increasing by ≥4 months in the absence of additional therapy and may be influenced by duration of PSADT follow up. Confirmatory data comes from a randomized phase II clinical trial involving BRPC patients where patients on the placebo arm experienced PSADT increases of nearly 6 months, and in both of these populations ≥50% of patients experienced increases of PSADT >100% and approximately one third experienced PSADT increases of >200%. These results are consistent with findings in prior studies showing 73% of placebo patients in clinical trials having >100% and 20–31% having >200% increase in PSADT.[15,16]

The similarity in within-patient increases in PSADT when using every other PSA value vs. using all PSA values implies that frequency of measurement will not impact PSA variability. However the larger within-patient increase in PSADT for the patients with ≥ 9 PSA measurements vs. patients with ≥ 6 PSA measurements implies that duration of measurement is associated with greater increases in PSADT, and therefore that single arm trials with longer durations are more likely to reach false conclusions of significance.

Thus, these data suggest that uncontrolled single-arm studies using change in PSADT as a primary endpoint may frequently show a statistically significant lengthening of PSADT that may be a false positive (i.e. might not be at all related to drug activity). This was demonstrated when at least 29% and as many as 55% of hypothetical single-arm, 50-patient studies, using subsets selected from the JHH cohort, showed statistically significant “baseline” to “post-baseline” PSADT changes. These simulated trials compared median within-patient PSADT change from “baseline” to “post-baseline” and showed apparently significant increases despite the lack of active treatment of these patients. In contrast, simulated randomized comparative trials using randomly sampled untreated patient trajectories maintained the 5% Type I error of a false positive result. Thus, statistically significant increases in PSADT of 6 months or less, or changes of up to 60% in the numbers of patients experiencing 100% increases in PSADT are unlikely to be reliable measures of clinical impact for advancing experimental treatments into randomized phase III trials. Single-arm phase II studies are commonly used to determine if a treatment has activity against a disease. The significant variability found in this study reinforces the value of including placebo arms in clinical trials in BRPC patient populations.

Smaller changes in median PSADT were seen when the JHH population was expanded to include patients for whom at least six PSA measurements were available: 17%–37% of the simulations were significant in this expanded population versus 27%–62% in the smaller group with at least nine PSA measurements. The smaller increase in median PSADT differences and the reductions in the percentage of simulated trials that were significant are artifacts of sampling. Patients might have fewer PSA measurements eligible for inclusion in this study if they have more rapidly progressing disease and require early hormone therapy. Therefore, data for the subset of patients with at least nine PSA values may have a bias because they exclude patients with a worse prognosis (i.e., a faster PSA doubling time). A second source of similar bias is introduced by our exclusion of patients receiving additional systemic therapies shortly after local therapy, including androgen deprivation. Excluding both of these groups of men retrospectively removes patients non-randomly, and it could be argued that the men removed from the analysis had the more biologically aggressive disease. A further limitation of this data set was the absence of other data on biologic activity that may be affecting the production of PSA over time.

Although patients were excluded if they were on any medications approved for treatment of prostate cancer, available data did not allow us to determine when patients were also taking natural products or other medications (such as non-steroidal inflammatory agents) that could potentially decrease PSA values. Despite this limitation and the biases described above, the natural history of PSADT in BRPC patients with slower PSADTs is illuminating because this is the BRPC patient population most likely to remain on trials of therapies designed to delay the onset of metastases, especially when potential toxicities outweigh the benefits of hormone treatment. This population is targeted in current trials of acai berries, disulfiram, brassica vegetables, kanglaite (Chinese grass seed oil), and vorinostat, all of which are either single-arm or dose-finding studies without placebo arms.[6] In the current setting where no proven therapy exists, a placebo control is appropriate in these trials and appears to be warranted.

Although a large single-institution patient population was culled for this study and a second multi-institution trial population provided confirming data, the overall numbers of patients in the analysis is small. The results require additional validation in other cohorts before they can be used as guidance for endpoint determination in clinical trials.

Conclusion and Implications

This retrospective study shows that PSA variability in BRPC patients who are monitored for nine or more PSA measurements, on average three months apart, leads to their PSADT naturally lengthening over time even in the absence of treatment. Single arm trials could therefore show apparently significant increases despite the lack of active treatment of these patients. Thus, using PSADT as the primary endpoint in clinical trials requires a placebo control in the absence of effective proven therapy, and the magnitude of the benefit needs to be clinically meaningful (e.g. ≥6 months). Further, when PSADT change is used as an endpoint in clinical trials, all PSA values recorded at a specified frequency should be used in computing a baseline PSADT as that is the same protocol that will be used in computing post baseline PSADT. For instance, one could envision a trial for men with low-risk BRPC in which patients would remain off therapy for the first several months of the study (i.e. during a lead-in period), where PSA data would be collected at the same interval (e.g. monthly) as would be collected after the initiation of treatment, and analyzed by one central laboratory. Moreover, the frequency of PSA measurement does not appear to affect PSADT determinations, a finding that would be expected given the log-linear relationship between PSA and time in the computation of PSADT. In conclusion, PSADT is a useful measure of PSA dynamics in men with BRPC and should be evaluated only in randomized controlled clinical trials, preferably including conventional clinical and radiological endpoints that allow for proper validation. In this way, post-treatment changes in PSADT in an active treatment arm and in a placebo arm could both be associated prospectively with clinically-meaningful endpoints such as time-to-first-metastasis or metastasis-free survival.