Novel Approach for Clinical Validation of the cobas KRAS Mutation Test in Advanced Colorectal Cancer.

AIM: Our objective was to assess the performance of the cobas test versus comparators for KRAS mutation status and predicting clinical response to anti-epidermal growth factor receptor (EGFR) therapy in patients with metastatic colorectal cancer (mCRC).
METHODS: mCRC samples from 398 patients from Roche study NO16968 (XELOXA) and 82 supplemental samples were tested with the cobas(®) KRAS mutation test (cobas test), the therascreen(®) KRAS RGQ PCR kit test (therascreen test), and Sanger sequencing as the reference method for detecting mutations in codons 12/13.
RESULTS: For 461 eligible samples, the cobas test, therascreen test, and sequencing had invalid results for 5.2, 10.8, and 2.6 % of specimens, respectively. Valid cobas and therascreen test results had similar KRAS mutation-positive rates (37.3 vs. 36.3 %, respectively); sequencing was 28.5 %. Positive and negative percent agreement (PPA/NPA) between the cobas test and sequencing was 96.9 % (95 % confidence interval [CI] 92.2-98.8), and 88.7 % (95 % CI 84.7-91.8), respectively. PPA/NPA between the cobas and therascreen tests was 93.3 % (95 % CI 88.1-96.3) and 96.5 % (95 % CI 93.5-98.1), respectively. Bridging analysis from NCIC-CO.17 and NCT00113763 using the cobas test yielded modeled hazard ratios for overall survival and progression-free survival (PFS) of 0.558 (95 % CI 0.422-0.752) and 0.413 (95 % CI 0.304-0.550), respectively, for cetuximab and 0.989 (95 % CI 0.778-1.299) and 0.471 (95 % CI 0.360-0.626), respectively, for panitumumab, demonstrating significant efficacy in the KRAS-negative population for PFS.
CONCLUSION: The cobas test showed similar accuracy to the therascreen test for detecting KRAS mutations and could appropriately identify mCRC patients ineligible for anti-EGFR therapy as demonstrated by bridging analysis results.

Key Points

Introduction

In vitro diagnostic (IVD) tests are playing an increasingly important role in selecting patients with cancer for targeted therapies, both to direct treatment to eligible patients and to avoid prescribing unnecessary and potentially detrimental therapies. The CO.17 pivotal study and subsequent analyses showed that cetuximab, a monoclonal antibody that inhibits activity of the epidermal growth factor receptor (EGFR), provided a survival benefit to patients with advanced colorectal cancer (CRC), i.e., those for whom all chemotherapy for CRC had failed and no other standard anticancer therapy was available. Compared with best supportive care (BSC) alone, treatment with cetuximab yielded a significant improvement in overall survival (OS) [1]. Subsequent analyses showed that the survival benefit applied only to patients with metastatic colorectal cancer (mCRC) whose tumors harbored a wild-type KRAS gene, and tumors with a mutation in codon 12 or 13 of exon 2 of the KRAS gene were insensitive to anti-EGFR therapy [2-5]. Several prospective randomized controlled trials and retrospective analyses have confirmed this conclusion [4, 6–9]. Similar results were observed for another anti-EGFR monoclonal antibody, panitumumab, in study NCT00113763. In this study, panitumumab plus BSC yielded a significant improvement in progression-free survival (PFS) compared with BSC alone in the subgroup of patients with wild-type KRAS [10]. Also, there was no significant improvement in the subgroup with KRAS mutation [10].

The mechanism of anti-EGFR failure in patients with EGFR-expressing tumors can be explained by the fact that KRAS acts downstream of EGFR, and KRAS-activating mutations can supersede EGFR inhibition.

Recent studies have shown that mutations in KRAS outside exon 2 and mutations in NRAS are also associated with low response rates for cetuximab and panitumumab therapy [11-13]. It is now recognized that KRAS and NRAS mutations in exons 2, 3, and 4 are predictive of resistance to anti-EGFR therapy. Based on these findings, the National Comprehensive Cancer Network (NCCN) guidelines strongly recommend KRAS/NRAS genotyping of tumor tissue in all patients with mCRC [14]. Also, limiting EGFR-directed treatment to patients with wild-type KRAS is supported by clinical treatment guidelines from the NCCN, the American Society of Clinical Oncology, and the European Society for Medical Oncology [14-16].

The therascreen® KRAS RGQ PCR Kit (therascreen test) for use with the Rotor-Gene Q MDx instrument was approved by the US FDA in July 2012 for determining KRAS mutations status in patients with mCRC [3]. The test is designed with primers to specifically detect mutations in KRAS codons 12 and 13, since these codons account for the majority of activating mutations (mutations in codons 12 and 13 account for ~82 and ~17 %, respectively, of all reported KRAS mutations in exon 2) [17].

To expand the available IVD options for testing the anti-EGFR therapy response of patients with mCRC, a study was designed to compare the performance of the cobas® KRAS mutation test (hereafter referred to as the cobas test) with that of Sanger sequencing and that of the therascreen test.

Clinical utility of a new companion diagnostic test is demonstrated when used within the context of a corresponding clinical trial to identify the subpopulation of patients who are most likely to benefit from an investigational drug. Alternatively, retrospective testing of patient samples from the trial can be conducted to evaluate the efficacy of the drug. For the cobas test, a potential companion diagnostic for the treatment with cetuximab of patients with mCRC, this approach was not feasible because samples from the pivotal clinical trial were not available and conducting a new randomized controlled trial (with a placebo arm) to select patients using the cobas test would be unethical.

Samples from another clinical cohort (bridging study cohort) for CRC were tested by the cobas test and Sanger sequencing as well as with another FDA-approved test (the therascreen test). In the absence of clinical outcome data for patients in the bridging study cohort, the drug efficacy (estimated by hazard ratio [HR]) for cetuximab using the cobas test was evaluated by transporting the drug efficacy results from the pivotal study using a non-differential misclassification (NDMC) assumption. Under this assumption, the HR for the cobas wild-type and mutation-positive subsets were assumed to be the same as the HR conferred by the sequencing test result. The difference in log HRs for the cobas test results can be estimated by utilizing agreement between the cobas test and Sanger sequencing. This difference was also estimated for the FDA-approved test, and the result for the cobas test and the FDA-approved test were compared.

The primary objective of this study was to determine whether the analytical and clinical performance of the cobas test was comparable to that of the FDA-approved therascreen test in determining KRAS mutation status and predicting clinical response to anti-EGFR therapy. The study also aimed to demonstrate the clinical utility of the cobas test to predict cetuximab and panitumumab efficacy using a novel bridging analytical strategy.

Materials and Methods

Study Design

The analytical performance of the cobas test was compared with that of the therascreen test and Sanger sequencing for detecting mutations in KRAS codons 12 and 13. Tissue samples from the pivotal trials (NCIC-CO.17 [2, 3] and NCT00113763 [10]) were not available at the time of this study. Therefore, to assess analytical performance, formalin-fixed, paraffin-embedded tissue (FFPET) samples from the XELOXA (Xeloda and Oxaliplatin in Adjuvant Setting) trial [18], which enrolled patients with stage III CRC and a supplemental cohort of mCRC patient samples, were analyzed using both the cobas test and the therascreen test, with Sanger sequencing performed on all samples as the reference standard.

However, patients in the XELOXA trial were not treated with cetuximab or panitumumab. Therefore, in the absence of clinical outcome data from targeted therapy for patients in the study, we used a bridging analytical strategy to evaluate efficacy-based on data from separate trials [19]. Specifically, the drug efficacy of cetuximab and panitumumab using the cobas test was evaluated by transporting the HRs for each drug from two separate trials. The HRs for cetuximab were based on the Sanger sequencing result from the NCIC-CO.17 study [2], and the HRs for panitumumab were based on the therascreen result from the NCT00113763 study. Under an NDMC assumption, efficacy was determined by performing bridging statistical analyses [10]. In short, the HRs for the cobas wild-type and mutation-positive subsets were assumed to be the same as the HR conferred by the sequencing test result. The difference in log HRs for the cobas test results compared with Sanger sequencing and the therascreen test was estimated by utilizing agreement between the cobas test and Sanger sequencing and between the cobas test and the therascreen test, respectively. The positive and negative predictive values (PPVs and NPVs, respectively) were estimated by utilizing mutation prevalence from the NCIC-CO.17 study and the NCT00113763 study. These estimates were then used to calculate the attenuation factors for the cobas and reference tests and are described in full in Sect. 2.8.

Additional analyses were performed (1) to demonstrate the comparability of the clinical trial cohorts (covariate analysis), (2) to evaluate the “influence criterion” and sensitivity analyses, and (3) to evaluate the robustness of the results.

This approach was necessary because it would have been unethical to perform a randomized controlled trial similar to those studies now that the relationship between KRAS mutation status and cetuximab or panitumumab efficacy in CRC patients is clear.

All specimens were sent to one external testing laboratory for KRAS mutation testing using the cobas test and the therascreen test. A second clinical laboratory selected by Roche Molecular Systems performed Sanger sequencing. To reduce bias, all slides were coded by the sponsor to mask the identity of individual patients and comparator method results from the testing lab. Sample processing and mutation testing were performed in accordance with the respective manufacturer’s instructions for the cobas test (Roche Molecular Systems, Inc., Peasanton, CA, USA) and the therascreen test (Qiagen Inc., Valencia, CA, USA). The study protocol was submitted to an Institutional Review Board in accordance with FDA and local regulatory requirements before the start of the study.

Study Population

Tumor specimens were obtained from 398 patients enrolled between April 2003 and October 2004 in Roche study NO16968 (XELOXA), a global, randomized, phase III study conducted at approximately 226 sites worldwide [18]. In addition, NO16968 samples were supplemented with tumor specimens from 82 mCRC patients. From this initial set of 480 screened samples, 461 eligible FFPET samples were tested for KRAS mutations with the cobas and therascreen tests, since 15 samples had no tumor content and four were not CRC, as confirmed by pathology evaluation. Additionally, all 461 samples underwent Sanger DNA sequencing as the reference method. Eligible patients were aged at least 18 years and had histologically confirmed, late-stage CRC. Sufficient archival FFPET material had to be available for mutation analysis, and written informed consent was obtained from all patients or their legal guardian. Patients were excluded if there was insufficient material to perform testing with the cobas test, the therascreen test, and Sanger sequencing.

Tumor Block Sectioning and Pathology Assessment

FFPET blocks were sectioned and ten serially cut 5-μm sections were mounted on slides with one section per slide. Four of ten slides were designated for cobas testing and four were designated for therascreen testing. All slides were coded by laboratory personnel to mask the identity of the individual patients. One slide from the middle of the serial sections underwent hematoxylin and eosin (H&E) staining. A laboratory pathologist used the H&E-stained slide to determine the percentage tumor cell content by area. Manual macro-dissection was required if tumor cell content by area was <10 % for the cobas test and <20 % for the therascreen test. One of the ten slides was saved as back-up for each of the assays, whereas one was reserved for H&E testing and four slides each were reserved for the cobas test and the therascreen test and for repeat testing, if necessary (i.e., in the event of an invalid result).

cobas Test Protocol

The cobas® KRAS Mutation Test kit (cobas test) is a TaqMelt™-based polymerase chain reaction (PCR) assay [20, 21] based on two major processes: (1) manual specimen preparation to obtain genomic DNA from FFPET using the cobas DNA Sample Preparation Kit; and (2) PCR amplification of target DNA. The test is designed to detect 19 mutations in codons 12 and 13 of exon 2 and 61 of exon 3 of the KRAS gene. However, because of the lack of a predicate device for codon 61, we compared the cobas test performance for only codons 12/13.The cobas test has a ≥95 % correct mutation call rate across all specimen types with ~5 % mutant alleles at a DNA input of 0.8–6.3 ng/PCR [20].

Genomic DNA was isolated from at least one slide designated for the cobas test. DNA isolation was performed using the cobas test manufacturer’s instructions to ensure that sufficient DNA was recovered. If the concentration of DNA obtained from a slide was ≥4 ng/μl, testing with the cobas test was performed. If the concentration of the purified sample was <4 ng/μl, DNA isolation was repeated using two slides. Tissue from these two slides was added to one tube and DNA isolation and testing with the cobas test was performed if the concentration of DNA obtained was ≥4 ng/μl. All residual DNA stock from the sample preparation process was quantified and stored at –20 °C for future shipment to a separate clinical laboratory where all samples underwent Sanger sequencing.

therascreen Test Protocol

The therascreen® KRAS RGQ PCR Kit (therascreen test) is a real-time PCR assay that combines an amplification refractory mutation system (ARMS) and a Scorpion fluorescent primer/probe system [22]. The therascreen test has high sensitivity (~1 % mutant allele), but requires >160 ng of amplifiable DNA (equivalent to 800 ng of the total DNA input) and eight PCR reactions to detect the seven most common mutations in codons 12 and 13 of the KRAS gene and may not detect uncommon KRAS mutations [22, 23]. Genomic DNA was isolated from at least one slide from each patient designated for therascreen testing. DNA was isolated using the QIAamp DSP DNA FFPE Tissue Kit (Qiagen, Inc.) and then tested with the therascreen test, according to the manufacturer’s instructions. Based on the therascreen test package insert, a PCR-based sample-assessment step for DNA quality was performed prior to the mutation detection round. Based on the assessment step, samples could be designated as adequate for testing, interpret with caution because of low levels of DNA, very few amplifiable copies of DNA, or reject sample. Mutation calls were based on the manufacturer’s recommendations using a delta Ct method.

Sanger DNA Sequencing

After a valid cobas test result was confirmed, residual DNA from the cobas test sample-preparation process was combined with DNA extracted and quantified from the remaining slides as needed to achieve a minimum of 5 μl DNA with a DNA concentration ≥10 ng/μl. DNA stock intended for sequencing was shipped at –20 °C to a separate clinical laboratory selected by Roche Molecular Systems.

Agreement Analysis and Statistics

All analyses were performed using SAS® Enterprise Guide software version 5.1 (SAS Institute, 2012). Mutation detected (MD) was defined as the presence of KRAS mutations in codons 12 and/or 13 and no mutation detected (NMD) was defined as the absence of KRAS mutations in codons 12 and 13.

The analytical performance of the cobas test compared with the therascreen test and the reference method (Sanger sequencing) for detection of codon 12 and 13 mutations was evaluated by positive, negative, and overall percent agreement (PPA, NPA, and OPA) with 2-sided 95 % confidence intervals (CIs).

Analyses of Clinical Effectiveness of cobas Test

In concordance with recommendations from FDA, the following five criteria were used to establish the clinical utility of the cobas test, specifically, the ability of the cobas test to select patients who are likely to benefit from treatment with cetuximab or with panitumumab:

Attenuation factors for cobas test

Attenuation factors for reference KRAS test

Clinical effectiveness and/or influence condition evaluation

Covariate assessment

Sensitivity analysis

The first three criteria were evaluated using a key assumption that the drug efficacy (of cetuximab or panitumumab) using the cobas test is non-differential given the reference method result. This assumption leads to the following relationships based on the PPV and NPV estimated from initial studies:here δ. = log(HR) for cobas test NMD result, δ. = log(HR) for cobas test MD result, δ.0 = log(HR) for Sanger NMD result, δ.1 = log(HR) for Sanger MD result, p = PPA; p = 1–NPA

Difference in log HR for cobas test = difference in log HR for reference method MD and NMD result × (NPV + PPV − 1) was calculated as follows:

Log HR for cobas KRAS MD result = log HR for reference method MD result × PPV + log HR for reference method NMD result × (1 − PPV)

Log HR for cobas KRAS NMD result = log HR for reference method MD result × (1 − NPV) + log HR for reference method NMD result × NPV

Criteria 1 and 2 The attenuation factors for the cobas test and reference test were calculated by the quantity PPV + NPV − 1. The predictive values of the cobas test were calculated by combining the PPA and NPA of the cobas test relative to a reference method together with the prevalence of a KRAS MD result by the reference method in the published clinical studies for cetuximab or panitumumab [2, 10]. The formulas for PPV and NPV are as follows:

The estimates and CIs were calculated using the parametric bootstrap method.

Criterion 3 The clinical effectiveness of the cobas test, measured by HR, to select mCRC patients for treatment with cetuximab or panitumumab was supported by demonstrating significant efficacy in the KRAS NMD population and lack of significant efficacy in the KRAS MD population based on PFS, with supplementary analysis of OS. Drug efficacy for cetuximab or panitumumab using the cobas test was evaluated by transporting the drug efficacy results from the pivotal study NCIC-CO.17 and NCT00113763, respectively, under the NDMC assumption.

The estimate of the HR for the cobas test for the NMD and MD cohorts and the difference of the log HR and their corresponding 95 % CIs were estimated using a parametric bootstrap method.

Criterion 4 The distribution of covariates between XELOXA and the supplemental samples and NCIC-CO.17 for cetuximab (or NCT00113763 for panitumumab) were compared. Drug efficacy was re-evaluated after adjusting for covariates, which were significantly different between the two studies.

Criterion 5 A sensitivity analysis was performed to determine how many agreements between cobas testing and Sanger sequencing would have to be changed to disagreements before the study failed to show clinical effectiveness [19].

Results

Patient Characteristics

The median age of the 480 enrolled patients was 61 years (range 26–89). Approximately 89 % of patients were Caucasian and 54 % were male; 95 % of patients had an Eastern Cooperative Oncology Group (ECOG) score of 0 or 1 and 98 % had stage III disease. The vast majority of tissue samples were from primary tumors, with the remaining 3 % being metastatic (Table 1). A total of 86 % of tumor specimens contained at least 20 % tumor cells, and 54 % had no tumor necrosis. Patient demographics and baseline characteristics are shown in Table S1 in the Electronic Supplementary Material (ESM).

Table 1

Baseline tumor characteristics and specimen availability

Characteristics	Overall (n = 480)	XELOXA samples (n = 398)	Supplemental samples (n = 82)
Tumor type
Primary	463 (96.5)	394 (99.0)	69 (84.1)
Metastatic	12 (2.5)	0 (0)	12 (14.6)
Unknown	5 (1.0)	4 (1.0)	1 (1.2)
Tumor contents
<10 %	26 (5.4)	25 (6.3)	1 (1.2)
10–20 %	42 (8.8)	38 (9.5)	4 (4.9)
>20 %	412 (85.8)	335 (84.2)	77 (93.9)
Necrosis
No necrosis	261 (54.4)	225 (56.5)	36 (43.9)
<10 %	76 (15.8)	59 (14.8)	17 (20.7)
≥10 %	143 (29.8)	114 (28.6)	29 (35.4)

All data are presented as n (%)

KRAS Mutation Analysis: Invalid Samples and Mutation Detection Rates

All of the 461 eligible samples were tested by the cobas test, the therascreen test, and Sanger sequencing. Specimens for which the codon 12/13 reaction well was invalid or for which an out-of-range melting temperature or peak height was observed (neither wild-type nor mutant range) were reported as ‘invalid’. The cobas test had an invalid rate of 5.2 %, the therascreen test had an invalid rate of 10.8 %, and Sanger sequencing had an invalid rate of 2.6 % (Table 2). The therascreen invalid rate included 8 % DNA sample assessment failures and 2.8 % mutation testing failures. If the initial test result was invalid, retesting by the cobas test or therascreen was performed up to two times; however, re-testing was not limited unless there was insufficient remaining sample DNA for Sanger sequencing. Among the samples with valid results, the cobas and therascreen tests had similar KRAS mutation-positive rates (37.3 vs. 36.3 %, respectively), while Sanger sequencing had a positive rate of 28.5 %.

Table 2

Invalid sample rates and mutation detection rates from the three testing methods

	cobas test	therascreen test	Sanger sequencing
Samples tested (n)	461	461	461
Invalid result	24 (5.2)	50 (10.8)	12 (2.6)
Valid result	437 (94.8)	411 (89.2)	449 (97.4)
No mutation detected	274 (62.7)	262 (63.7)	321 (71.5)
Mutation detected	163 (37.3)	149 (36.3)	128 (28.5)

All data are presented as n (%) unless otherwise indicated

Analytical Performance

The analytical performance of the cobas test was compared with Sanger sequencing or the therascreen test for the ability to detect mutations in KRAS codons 12 and 13 in CRC tumor samples by determining PPA, NPA, and OPA. Results are shown in Table 3. The four samples where Sanger sequencing identified mutations in KRAS but the cobas test missed them were also missed by therascreen. Of 34 samples where the cobas test detected codon 12/13 mutations and Sanger sequencing missed, 26 were confirmed as codon 12/13 mutations by therascreen. When comparing the cobas and therascreen tests, the cobas test missed mutations in ten samples called codon 12/13 mutations by therascreen. All these samples were confirmed as mutation not detected by Sanger sequencing. Of the nine samples called MD by cobas and NMD by therascreen, six samples were NMD and three were MD by Sanger sequencing. The PPA/NPA between the cobas test and Sanger sequencing was 96.9 % (95 % CI 92.2–98.8) and 88.7 % (95 % CI 84.7–91.8), respectively. The PPA/NPA between the cobas and therascreen tests was 93.3 % (95 % CI 88.1–96.3) and 96.5 % (95 % CI 93.5–98.1), respectively. The PPA between therascreen and Sanger sequencing was 94.2 % (113/120) and the NPA was 87.5 % (253/289).

Table 3

Comparison of the cobas® KRAS mutation test with reference methods for detection of KRAS mutations in codons 12 and 13

cobas KRAS mutation test (test method)	Reference method
	Sanger sequencing				therascreen KRAS RGQ PCR Kit
	Mutation positive	Mutation negative	Invalid	Total	Mutation positive	Mutation negative	Invalid	Total
MD	124	34	5	163	139	9	15	163
MND	4	268	2	274	10	248	16	274
Invalid	0	19	5	24	0	5	19	24
Total	128	321	12	461	149	262	50	461
PPAa	96.9 % (124/128) [92.2–98.8]				93.3 % (139/149) [88.1–96.3]
NPAa	88.7 % (268/302) [84.7–91.8]				96.5 % (248/257) [93.5–98.1]
OPAa	91.2 % (392/430) [88.1–93.5]				95.3 % (387/406) [92.8–97.0]

CI confidence interval MD mutation detected, MND mutation not detected, NPA negative percentage agreement, OPA overall percentage agreement, PPA positive percentage agreement

aData are presented as % (n/N) [95% CI]

Since Sanger sequencing is less sensitive than both the cobas and the therascreen tests, the lower NPA between the cobas test and Sanger sequencing (88.7 %; 268 of 302 samples; 95 % CI 84.7–91.8) was expected. The improved NPA between the cobas and therascreen tests (96.5 %; 248 of 257 samples; 95 % CI 93.5–98.1) supports this expectation.

Clinical Utility

Attenuation Factor Evaluation

Clinical effectiveness of the cobas test was established by modeling the difference in the log HR from the pivotal clinical trials for cetuximab and panitumumab. Under the NDMC assumptions, the difference in log HR between NMD and MD populations was projected to be attenuated relative to the corresponding difference for the comparator method by the factor (PPV + NPV − 1). The attenuation factor refers to the fraction of expected difference in response that is preserved when the new test is used instead of the reference test.

The attenuation factors (PPV + NPV − 1) for cobas versus the reference methods are shown in Table 4, which indicated that the difference in the log HRs of the cobas test NMD and MD subsets was projected to be 83.3–90.4 % of the corresponding difference for the reference method. The attenuation factor for the therascreen test versus Sanger sequencing was 79.5 %, which indicated that only approximately 80 % of the difference in log HR was preserved from the drug efficacy based on Sanger sequencing.

Table 4

Attenuation factors (PPV + NPV − 1) for the cobas® KRAS mutation test

Comparison	Drug	Codon 12/13 prevalence in the pivotal study (%)	PPV	NPV	Attenuation factor
cobas test vs. Sanger sequencing	Cetuximab	41.6	0.858 (0.811–0.902)	0.975 (0.946–0.994)	83.3 % (77.7–88.3)
cobas test vs. therascreen test	Cetuximab	41.6	0.957 (0.927–0.981)	0.945 (0.909–0.978)	90.2 % (85.6–94.4)
cobas test vs. therascreen test	Panitumumaba	40.1	0.949 (0.914–0.977)	0.956 (0.927–0.981)	90.4 % (86.1–94.4)
therascreen test vs. Sanger sequencing	Cetuximab	41.6	0.840 (0.790–0.888)	0.956 (0.918–0.986)	79.5 % (73.4–85.2)

Data in parentheses are 95 % CIs

CI confidence interval, NPV negative predictive value, PPV positive predictive value

aSamples in the NCT00113763 trial for panitumumab were only tested by the therascreen test

Estimated Efficacy of Anti-Epidermal Growth Factor Receptor (EGFR) Therapy by cobas Test

For cetuximab, the HRs for PFS and OS in patients with wild-type KRAS were estimated to be 0.413 (95 % CI 0.304–0.550) and 0.558 (95 % CI 0.422–0.752) (Table 5). These HRs indicated that PFS and OS were both significantly improved in a CRC population of KRAS NMD as determined by the cobas test. For the KRAS MD population selected by the cobas test, the HRs for PFS and OS were 0.869 (95 % CI 0.670–1.138) and 0.908 (95 % CI 670–1.209), indicating that PFS and OS were not improved in a CRC patient population with mutated KRAS as determined by the cobas test.

Table 5

Estimated drug efficacy by KRAS mutation status

Efficacy outcome	cobas KRAS mutation status	Cetuximab HR (95 % CI)	Panitumumab HR (95 % CI)
PFS	NMD	0.413 (0.304–0.550)	0.471 (0.360–0.626)
	MD	0.869 (0.670–1.138)	0.964 (0.709–1.293)
OS	NMD	0.558 (0.422–0.752)	0.989 (0.778–1.299)
	MD	0.908 (0.670–1.209)	1.027 (0.767–1.375)

CI confidence interval, HR hazard ratio, MD mutation detected, NMD no mutation detected, OS overall survival, PFS progression-free survival

Similarly, the estimated HRs for PFS in the KRAS NMD and KRAS MD populations identified by the cobas test were estimated to be 0.471 (95 % CI 0.360–0.626) and 0.964 (95 % CI 0.709–1.293), respectively, for panitumumab. These HRs indicate that PFS was significantly improved in CRC patients with wild-type KRAS status and not improved in patients with mutated KRAS status.

Covariate Assessment and Sensitivity Analysis

Tables evaluating sample attribute covariates and sensitivity analysis are presented in the ESM (Tables S2–S5). Covariate analysis showed some significant sample-type characteristic differences between the bridging cohort and the trial cohorts, including Duke’s stage (Table S2 in the ESM); however, KRAS mutation status appears to occur at a relatively constant rate regardless of clinical stage [24, 25] and primary tumors are often used to determine KRAS status for ‘intention-to-treat’ patients [17, 26]. The sensitivity analysis showed that the bridging comparison was robust since the estimated HRs for OS and PFS for the MD population decreased to <1.0 (with an upper limit of the 95 % CI of 0.99) only after the frequency of discordance exceeded 21 % and 12.6 % beyond the measured discordance, respectively (Table S5 in the ESM).

Discussion

The study was undertaken to evaluate the performance of the cobas test in selecting advanced CRC patients for treatment with cetuximab and panitumumab. Determination of KRAS mutation status is recommended by the NCCN guidelines [14] to select appropriate treatment for patients with mCRC. Using bridging statistical analysis [19] and NDMC assumption to impute efficacy from the pivotal phase III clinical trials CO.17 for cetuximab and NCT0013763 for panitumumab, the results demonstrate that the cobas test shows similar accuracy to the therascreen test for detecting KRAS mutations in CRC FFPET samples for the treatment decision.

The study design was determined by the fact that CRC tumor specimens from the therascreen pivotal trial, CO.17, were unavailable. Moreover, it would have been unethical to perform a clinical trial with the same design as the CO.17 trial, which used BSC in one of the two treatment arms. Therefore, in close collaboration with FDA, the current study design was adopted. The study analyzed KRAS mutations in CRC tumor specimens from the Roche clinical trial NO16968 (XELOXA) because the clinical attributes from patients in the XELOXA trial were similar to those used in both pivotal trials. The cobas test demonstrated an analytical performance that was comparable to that of the therascreen test based on PPA, NPA, and OPA. The differences in analytical performance noted between the cobas and therascreen tests could be due to uncommon KRAS mutations that will not be detected by the therascreen design (in the case of the nine samples called MD by cobas and MND by therascreen), while low tumor cellularity (<10 %) could be the reason the cobas test missed mutations in ten samples called codon 12/13 mutations by therascreen. Clinical utility of the cobas test was demonstrated by showing maintenance of cetuximab and panitumumab treatment efficacy in the KRAS NMD patient population and the lack of treatment efficacy in the KRAS MD patient population—those with mutations in KRAS codons 12 and 13. The cobas test and the therascreen test are essentially comparable in their ability to correctly select patient populations for either prescribing or withholding cetuximab or panitumumab treatment based on the absence or presence, respectively, of mutations in KRAS codons 12 and 13. In another study comparing the cobas and therascreen tests and Sanger sequencing, the cobas test was demonstrated to have greater analytic sensitivity in detecting KRAS mutations than Sanger sequencing and also detected more uncommon KRAS mutations than the therascreen test with a lower sample input volume (100 ng of total DNA input with the cobas test vs. 800 ng of total DNA with the therascreen test) due to its broader reportable range [22].

In the current study, a lower attenuation factor for therascreen versus Sanger sequencing was noted. This is expected as, among the three assays, Sanger sequencing has the lowest analytic sensitivity [22]. Low tumor cellularity of specimens could lead to false-negative results with Sanger sequencing [23] and would require more sensitive assays for mutation detection. In the current study, 26 samples were recorded as KRAS NMD with Sanger sequencing and MD with the cobas test (and confirmed MD with the therascreen test); the negative results with Sanger sequencing could possibly be due to poor analytical sensitivity of the method. For Sanger sequencing, residual DNA from the cobas sample preparation process was combined with DNA extracted and quantified from the remaining slides as needed to obtain the required sample processing volume. This meant that samples undergoing Sanger sequencing were macro-dissected in accordance with the cobas test package insert instructions, i.e., macro-dissection was required for specimens with tumor cell content by area of <10 %. If macro-dissection had been performed for the 42 specimens with tumor cell content by area of 10–20 %, it is possible that the mutation detection rate with cobas and Sanger sequencing could have been improved. The four samples where Sanger sequencing identified mutations in KRAS but the cobas test missed them were also missed by therascreen because of uncommon KRAS mutations that are not detected by the cobas and therascreen designs.

The study design of the panitumumab pivotal trial (NCT0013763) was very similar to that of the cetuximab trial (CO.17) [2, 10]. Both studies evaluated efficacy of the anti-EGFR antibody versus BSC in patients with advanced CRC. The one striking difference was that, in the panitumumab pivotal trial, patients randomized to BSC were allowed to cross over to panitumumab upon progression. This difference affected the HRs for OS, which showed a non-significant difference between the two treatments. The results indicate that the influence condition does not hold due to improved drug efficacy in the KRAS NMD population and no improvement in the KRAS MD population as selected by cobas testing.

Some of the challenges with the current study are that the true clinical performance of the cobas test relative to the other methods has not been prospectively tested, the patient cohort represented stage III and IV patients, and invalid and discrepant results were not sufficiently resolved. Because the efficacy of anti-EGFR therapy has been demonstrated to have significant benefit in patients with KRAS wild-type mutations, it would be unethical to withhold therapy for such patients. Conversely, treatment of patients with KRAS mutants with anti-EGFR therapy leads to no benefit and can have a potential negative impact on survival. Therefore, we evaluated samples from primary tumors of patients with stage III and IV disease. For patients who have tumors excised with curative intent in stage III disease, that tissue is often used to establish KRAS mutation status prior to or at the time of disease recurrence or after progression when treated with anti-angiogenic therapies such as bevacizumab. Finally, discrepant cases may be due to the different methods, tumor heterogeneity, or varying sensitivity for the assays. Invalid cases for either technology may have been a function of the quality of the sections or the methods used for DNA isolation or the handling of the sample. In clinical practice, when an invalid result occurs, the testing laboratory would repeat the assay or use an alternative technology. Any diagnostic test is prone to invalid results, but the workflow and ease of use for the cobas test, along with the low invalid rate, would likely be manageable. Another potential reason for invalid results could be the age of the tissue blocks or oxidation of the tissue. In routine practice, patients with freshly prepared FFPET blocks may encounter fewer invalid results than in this retrospective study.

The study indicates that the HRs for PFS and OS from patients with wild-type versus mutated KRAS are equally comparable between the cobas test and another FDA-approved test. This novel approach to test validation using a bridging analytical strategy demonstrated that the cobas test could appropriately identify patients with mCRC ineligible for anti-EGFR therapy.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 35 kb)

Targeted therapy with cetuximab provides a survival benefit to patients with metastatic colorectal cancer (mCRC) whose tumors harbor a wild-type KRAS gene.

To expand the available diagnostic options for testing the anti-epidermal growth factor receptor (EGFR) therapy response of patients with mCRC, the current study compared the performance of the cobas test with that of the therascreen test and Sanger sequencing. Bridging analysis of existing published data was performed through a method comparison study to validate clinical value.

The cobas test demonstrated an analytical performance comparable to that of the therascreen test based on positive, negative, and overall percent agreements and also demonstrated clinical utility in predicting clinical response to anti-EGFR therapy in patients with KRAS wild-type mCRC.