Comment on: 'evaluation of chemoresponse assays as predictive markers'.

Sir,

We read with great interest the recent Short Communication by Korn and Freidlin (2015), which considers hypothetical examples challenging the ‘match/mismatch' analysis presented in Tian . In Tian , we proposed and applied a novel match/mismatch analysis approach for evaluating the predictive value of a chemoresponse assay from an observational study, by investigating the assay's association with outcome. The match analysis was performed using the assay result for the administered therapy (assayed therapy=administered chemotherapy); the mismatch analysis was performed using the assay result for a randomly selected therapy from all assayed treatments for a given patient, not necessarily matching the administered therapy (assayed therapy≠administered chemotherapy). If the match association is stronger than mismatch association, then the association is potentially drug specific and the assay may have predictive value. Using three examples in which a hypothetical chemoresponse assay is assumed to have only prognostic value, Korn and Freidlin (2015) have indicated that this analytical method may incorrectly conclude that the assay has predictive properties.

We agree with Korn and Freidlin (2015) that the match/mismatch method employed in Tian should be applied in limited circumstances and likely cannot be generalised to all chemoresponse, or more generally to all predictive biomarker assessment studies. As Korn and Freidlin (2015) point out, in situations where either (1) the treatments being considered have meaningful differences in efficacy in the unselected population or (2) specific treatment selection for a given patient is based on factors that have prognostic importance, the match/mismatch approach is inappropriate. However, we believe that neither of these cases are present in the clinical situation of recurrent ovarian cancer considered in the study by Rutherford .

Specifically, in their hypothetical examples 2 and 3, Korn and Freidlin (2015) assumed different efficacies across treatments. This is inconsistent with the clinical situation in recurrent ovarian cancer (to which the match/mismatch analysis was applied), where more than ten different drugs are recommended, but evidence from clinical trials fail to demonstrate that any one is superior to any other (National Comprehensive Cancer Network, 2014). In their hypothetical example 1, Korn and Freidlin (2015) assumed similar treatment effects for drugs A and B, but they also assumed that the patients treated by drug A were different from those treated by drug B in terms of patient prognostic profiles. In Korn and Freidlin's (2015) example, due to differences in subpopulations (pattern of assay results and sampling fraction can also be different), the match/mismatch analysis method is indeed inappropriate. However, in the study by Tian , 15 drugs were evaluated and, as such, the heterogeneous pattern of assay results across treatments was far more complex than Korn and Freidlin's (2015) example that included two drugs. In addition, although it is possible that the treatment groups differ in prognostic profile, it is more likely, as demonstrated in clinical practice, that patients with similar prognoses have multiple therapeutic options, and there are no clear prognostic factors which dictate treatment decisions for individual patients. Taking all of these considerations together, after resampling, the likelihood that patients included in the mismatch analysis have similar prognostic profiles (on average), compared with those included in the match analysis, is quite high. Table 1 shows the comparison of patient prognostic profiles between match and mismatch analyses in the study by Tian , demonstrating strong similarity between the two analysis groups. For the mismatch analysis used in Tian , patients with heterogeneous patterns of in vitro response were assigned either ‘sensitivity (S)' or ‘resistance (R)' assay results by resampling. For match analysis, 28.6% were treated with an S drug and 71.4% were treated with an R drug, with mean multiple drug response index (MDRI) of 0.68 vs 0.10, respectively; for mismatch, 25.2% were assigned as S and 75.8% were assigned as R, with mean MDRI of 0.71 vs 0.11, respectively. Other major prognostic factors also had similar distributions (S vs R) between match and mismatch. These results indicate that if the assay is only prognostic, the association with patient outcome should be consistent between match and mismatch analyses. In other words, the difference reported was unlikely explained by the confounding effects of prognostic factors. To further control the potential confounding factors, we also included a multivariate analysis in our study which further demonstrated the differences in patient outcome between the match and mismatch analyses (Tian ).

Table 1

Comparison of prognostic profiles between match and mismatch analyses (sensitivity vs resistance)

	Match analysis		Mismatch analysisa
	Sensitivity (28.6%)	Resistance (71.4%)	Sensitivity (25.2%)	Resistance (74.8%)
MDRIb (mean)	0.68	0.10	0.71	0.11
Age (mean, years)	57.3	63.3	58.9	62.5
ECOG PS (%)
0	68.0	70.6	69.8	71.0
1 or 2	32.0	29.4	30.2	29.0
Cell type (%)
Serous	65.3	69.0	65.9	69.8
Others	34.7	31.0	34.1	30.2
Tumour grade (%)
1 or 2	15.9	23.3	17.9	23.1
3	84.1	76.7	82.1	76.9
TFIc (%)
<6 months	38.7	47.1	38.9	47.0
⩾6 months	61.3	52.9	61.1	53.0

Abbreviations: MDRI=multiple drug response index; TFI=treatment-free interval.

Mismatch analysis: results representing the averages of 3000 simulations.

MDRI representing the percentage of all assayed therapies to which a patient scored as sensitive.

TFI defined as the time interval from the end of treatment until disease progression in the first-line treatment setting.

We agree with Korn and Freidlin (2015) that evaluating the predictive value of chemoresponse assays is challenging. As with any observational study, it is impossible to entirely exclude bias, and a definitive answer relies on randomised clinical trials. However, randomised trials to evaluate predictive markers are highly challenging in rare tumour types such as recurrent ovarian cancer, particularly when a large number of treatment options are available. The length of time for patient accrual alone is likely to obfuscate clinical utility. Thus, observational studies and other non-randomised prospective studies must continue to have an important role in evaluating chemoresponse assays in this cancer type. We feel that in the appropriate circumstance, our proposed match/mismatch analysis can provide helpful information regarding an assay's potential prognostic and/or predictive value.