Validation and implementation of a modular targeted capture assay for the detection of clinically significant molecular oncology alterations.

OBJECTIVES: The rapid discovery of clinically significant genetic variants has translated to next-generation sequencing assays becoming out-of-date by the time they are designed, validated, and implemented. UW-OncoPlex addresses this through the adoption of a modular panel capable of redesign as significant alterations are identified. We describe the validation of OncoPlex version 6 (OPXv6) for the detection of single nucleotide variants (SNVs), insertions and deletions (indels), copy number variants (CNVs), structural variants (SVs), microsatellite instability (MSI), and tumor mutational burden (TMB) in a panel of 340 genes.
DESIGN: One hundred twelve samples with diverse diagnoses were comprised of formalin-fixed-paraffin-embedded tissue, fresh-frozen tissue, plasma, peripheral blood, bone marrow, saliva, and cell-line DNA. Libraries were prepared from genomic and cell-free DNA, hybridized to a custom panel of xGen Lockdown probes, and sequenced on Illumina platforms. Sequences were processed through a custom bioinformatics pipeline, and variant calls were compared to prior orthogonal clinical results.
RESULTS: Accuracy was 99% for SNVs ≥5% allele frequency, 98% for indels, 97% for SVs, 99% for CNVs, 100% for MSI, and 100% for TMB (compared to previous OncoPlex versions). Library preparation turnaround time decreased by 40%, and sequencing quality improved with a 2.5-fold increase in average sequencing coverage and 4-fold increase in percent on-target.
CONCLUSIONS: OPXv6 demonstrates improvements over prior UW-OncoPlex versions including reduced capture cost, improved sequencing quality, and decreased time to results. The modular capture probe design also provides a nimble laboratory response in addressing the expansions necessary to meet the needs of the continuously evolving field of molecular oncology.

Introduction

In oncology, a primary goal of precision medicine is to deliver the right drug to the right patient at the right time. Initially this approach utilized immunohistochemistry-based diagnostic tests to identify patients who might benefit from specific “targeted therapies”—as was the case with tamoxifen, a repurposed failed anti-estrogen contraceptive with significant therapeutic response in patients with estrogen binding protein (presumably estrogen receptor) positive breast cancer, and trastuzumab, a monoclonal antibody that received FDA approval in the 1990s for the treatment of human epidermal growth factor receptor 2 (HER2) positive breast cancer [1,2]. Immunohistochemistry, however, had limitations, as highlighted by the poor response rate of epidermal growth factor receptor (EGFR) expressing non-small cell lung cancer to erlotinib and gefitinib, small molecule tyrosine kinase inhibitors directed against EGFR, in early randomized clinical trials [3]. The subsequent identification of activating EGFR mutations in the small subset of lung cancer patients who did respond would serve to highlight the need for genetic profiling as a predictive biomarker for therapeutic enrollment [4]. Consequently, each discovery of new, non-overlapping alterations driving tumorigenesis and the development of corresponding clinically effective inhibitors, resulted in numerous single-gene tests, which were necessary to adequately classify a tumor subtype, accompanied by an increase in tissue requirements and cost.

The need to comprehensively assess clinical cancer specimens for an expanding list of alterations critical to therapeutic decision making led to the adoption of large “fixed-content” genetic panels that utilized massively parallel sequencing, more commonly referred to as next-generation sequencing (NGS). However, the rapid pace of discovery of clinically significant genes and variants translates to ever-larger panels that are out-of-date by the time they are designed, validated, and clinically implemented. UW-OncoPlex version 6 (OPXv6) addresses this deficiency through the adoption of modular hybridization capture probes that can be rapidly redesigned as significant diagnostic, prognostic, and/or therapeutic alterations are identified.

We herein report the analytical validation of OPXv6, a modular targeted NGS assay designed to detect single-nucleotide variants (SNVs), insertions and deletions (indels), copy number variations (CNVs), and select structural variants (SVs) in 340 genes selected for their clinical significance in cancer, as well as evaluate for microsatellite instability (MSI) and tumor mutational burden (TMB). This assay is intended to comprehensively characterize a wide variety of clinical specimens, improving on earlier approaches through expansion of the spectrum of detected alterations and generation of higher quality sequencing data more efficiently and at decreased cost, while accommodating for the continued evolution of the molecular oncology field.

Material and methods

Gene panel

OPXv6 employs a modular panel of Integrated DNA Technologies (IDT) xGen Lockdown Probes (IDT, Coralville IA), targeted against 340 genes, including 24 genes with recurrent structural alterations, selected for their clinical significance in relation to diagnosis (somatic or germline), prognosis, and/or therapy selection. The majority of the capture probes were designed and procured through a consortium of academic laboratories (GOAL project), supplemented with additional IDT xGen probes custom designed to provide complete coverage of non-repetitive intron and exon sequences of BRCA1, BRCA2, mismatch repair (MMR) genes (MSH2, MSH6, PMS2, MLH1), additional select regions recurrently involved in fusions, and genes not included in the GOAL consortium probe design (Table 1).

Table 1

OncoPlex v6 Gene Panel.

ABL1	BRAFa	CRX	ESR2	GLTSCR2	MAP2K1	NBN	PLK4	RPS20	STAT6
ABL2	BRCA1b	CSF1R	ETV6a	GNA11	MAP2K2	NF1	PML	RPTOR	STK11
ACVR1	BRCA2b	CSF3R	EZH2	GNAQ	MAP2K4	NF2	PMS2b	RRM1	SUFU
AKT1	BRIP1	CTCF	FAM175A	GNAS	MAPK1	NKX2-1	POLD1	RRM2	SUZ12
AKT2	BTK	CTNNA1	FANCA	GREM1	MAX	NOTCH1	POLE	RSPO2	TACC3
AKT3	C11ORF95a	CTNNB1	FANCB	GRIN2A	MC1R	NOTCH2	PPM1D	RSPO3a	TACSTD2
ALKa	CALR	CUX1	FANCC	GRM3	MCL1	NOTCH3	PRKAR1A	RUNX1	TCF3
ANGPTL1	CARD11	DAXX	FANCD2	H3F3A	MDM2	NPM1	PRPF40B	SAMD9L	TERC
ANKRD26	CBL	DDR2	FANCE	H3F3B	MDM4	NPRL2	PRPS1	SDHA	TERT
APC	CBLB	DDX41	FANCF	HDAC4	MED12	NPRL3	PTCH1	SDHB	TET1
AR	CBLC	DEPDC5	FANCG	HDAC9	MEGF6	NR4A3	PTEN	SDHC	TET2
ARAF	CCND1	DICER1	FANCI	HIF1A	MEN1	NRAS	PTPN11	SDHD	TET3
ARID1A	CCND2	DNAJB1a	FANCL	HIST1H3B	METa	NT5C2	PTPRD	SETBP1	TFE3
ARID1B	CCNE1	DNMT3A	FANCM	HNF1A	MIOS	NTHL1	QKIa	SETD2	TFG
ASXL1	CD19	DOCK7	FBXW7	HRAS	MITF	NTRK1a	RAC1	SF1	TGFBR2
ASXL2	CD274	DPYD	FGFR1a	HSPH1	MLH1b	NTRK2a	RAD21	SF3B1	TLX1
ATM	CD33	EBF1	FGFR2a	ID3	MLH3	NTRK3a	RAD51B	SH2B3	TMPRSS2a
ATR	CD74a	EGFRa	FGFR3a	IDH1	MN1	NUDT15	RAD51C	SHH	TP53
ATRX	CDC27	EIF3Ea	FGFR4	IDH2	MPL	PAK1	RAD51D	SLX4	TP73
AURKA	CDH1	ELF1	FH	IGF1R	MRE11A	PALB2	RAF1a	SMAD2	TRAF7
AURKB	CDK12	EML4a	FKBP1A	IKZF1	MSH2b	PAX5	RARA	SMAD3	TRRAP
AXIN2	CDK4	EP300	FLT1	IL7R	MSH6b	PBRM1	RB1	SMAD4	TSC1
AXL	CDK6	EPAS1	FLT3	JAK1	MSLN	PDCD1LG2	RECQL	SMARCA4	TSC2
BABAM1	CDK8	EPCAM	FLT4	JAK2	MTAP	PDGFRA	RELA	SMARCB1	TTYH1
BAK1	CDK9	EPHA3	FOXA1	JAK3	MTOR	PDGFRB	RETa	SMC1A	TYMS
BAP1	CDKN1A	EPHA5	FOXR2	KDM6A	MUTYH	PHF6	RHEB	SMC3	U2AF1
BARD1	CDKN2A	EPHB2	GAB2	KDR	MYB	PHOX2B	RICTOR	SMO	U2AF2
BCL2	CDKN2B	EPHB6	GALNT12	KIF5B	MYC	PIK3CA	RINT1	SPOP	VHL
BCL2L11	CEBPA	ERBB2	GATA1	KIT	MYCL1	PIK3CB	RIT1	SPRY4	WRN
BCOR	CHD1	ERBB3	GATA2	KLF4	MYCN	PIK3R1	ROR1	SRC	WT1
BCORL1	CHEK1	ERBB4	GATA3	KMT2Aa	MYD88	PLCG2	ROS1a	SRP72	XRCC2
BCR	CHEK2	ERCC2	GEN1	KMT2C	MYOD1	PLK1	RPL10	SRSF2	YAP1a
BIRC3	CREBBP	ERG	GLI1	KMT2D	NAB2	PLK2	RPS14	STAG2	ZBTB16
BMPR1A	CRLF2	ESR1	GLTSCR1	KRAS	NAT2	PLK3	RPS15	STAT5B	ZRSR2

Select introns sequenced in addition to all coding regions.

Full genes sequenced (introns and exons).

Sample selection and orthogonal testing

DNA samples used for the validation of OPXv6 were derived from 108 unique specimens composed of 29 different adult and pediatric neoplasms (including CNS malignancies, leukemia/lymphoma, melanoma, sarcoma, and carcinomas of the lung, breast, endometrium, bowel, and prostate) and 5 germline samples (Fig. 1). All samples had prior molecular characterization via orthogonal clinical tests including both laboratory-developed amplicon-based [5] and hybrid-capture-based NGS assays and/or a custom commercial RNA sequencing assay (FusionPlex, ArcherDx, Boulder, CO, USA). Validation samples were comprised of formalin-fixed paraffin embedded (FFPE) tissue (including biopsies, resections, and cell blocks) as well as fresh-frozen tissue, peripheral blood, bone marrow, cell-free DNA, and saliva; details regarding diagnosis, specimen type, and preparation can be found in Supplementary Table 1. Four HapMap samples (NA12891, NA12892, NA18517, and NA12878) (Coriell Cell Repositories, Camden, NJ) were also used to evaluate the overall performance characteristics of OPXv6, the latter (NA12878) as the positive control for every sequencing run. Collectively, these 112 samples comprised a validation cohort that included multiple known SNVs, Indels, CNVs, SVs, MSI statuses, and a wide range of TMBs.

Fig. 1

Distribution of specimen types tested on OPXv6.

DNA isolation, library preparation, hybrid capture, and sequencing

DNA was extracted using one or more of the following kits from Qiagen (Qiagen, Valencia, CA) depending on specimen type and nucleic acid extraction desired: Qiagen GeneRead DNA FFPE Kit (Catalog #180134), QIASymphony DSP DNA Midi Qiagen Kit (Catalog #937255), QIAsymphony Circulating DNA Kit (Catalog #93756), Gentra Puregene DNA Isolation Kit (Catalog #158489), or Qiagen AllPrep DNA/RNA kits (Catalog #80234, 80204, 80284).

Sequencing libraries from non-cell free DNA samples were prepared from at least 250 ​ng of extracted DNA when available (range: 78–1090 ​ng), with a median of 270 ​ng and a standard deviation (SD) of 154 ​ng, and sheared to an average of 250 bp via sonication (Covaris, Woburn, MA). Post shear DNA input into library preparation ranged from 11 to 520 ​ng (median ​= ​160 ​ng, SD ​= ​120 ​ng). Cell-free DNA (n ​= ​3), which did not require shearing, ranged from 6 to 23 ​ng in total. Both the amount of DNA sheared and the sheared DNA input for library preparation for each specimen can be found in Supplementary Table 1. Fragmentation was followed by enzymatic repair, A-tailing, and simultaneous adapter ligation and barcoding with i5 and i7 indexes to allow for multiplexing (KAPA HyperPrep Kits, Roche Sequencing and Life Science, Wilmington, MA) prior to pooled multiplex capture according to the “xGen hybridization capture of DNA libraries for NGS target enrichment” protocol [6]. In brief, adapter ligated barcoded DNA samples were pooled together with xGen blocking oligonucleotides and human Cot-1 DNA® (Thermo Fisher, Waltham, MA) and then hybridized to the custom-designed panel of individually synthesized 5′-biotinylated complementary DNA oligos targeting a ~1.9 ​Mb capture of 340 targeted genes (Table 1). Samples were subsequently amplified, purified on the Agilent BRAVO automated workstation (Agilent Technologies Inc, Santa Clara, CA) using AMPure beads (Beckman Coulter Life Sciences), and quantified prior to sequencing on the Illumina NextSeq 500 and/or HiSeq 2500 systems (Illumina Inc, San Diego, CA). A total of 14 sequencing runs were performed with 11 NextSeq runs containing up to 12 samples each and 3 HiSeq runs containing up to 24 samples each, including a standard NA12878 control in each.

Data analysis (bioinformatics pipeline)

After sequences were demultiplexed using bcl2fastq version 2.17 (https://support.illumina.com/sequencing/sequencing_software/bcl2fastq-conversion-software.html), sequencing reads were mapped against the human reference genome (hg19/GRCh37) using BWA version 0.6.1 (http://bio-bwa.sourceforge.net/) and aligned using SAMtools version 0.1.18 (http://www.htslib.org/). Duplicate reads were removed using PICARD version 1.113 (https://broadinstitute.github.io/picard/) prior to local read realignment for indel detection and ensuing recalibration of quality scores using Genome Analysis Toolkit (GATK, https://www.broadinstitute.org/gatk/). These recalibrated realignments were processed through a custom bioinformatics pipeline including SNV callers GATK and VarScan version 2.3.7 (http://varscan.sourceforge.net/), SV callers Pindel version 0.2.5a3 (http://gmt.genome.wustl.edu/pindel/0.2.4/index.html), BreakDancer version 1.4.5 (https://github.com/genome/breakdancer), and GRIDSS version 1.8.1 (https://github.com/PapenfussLab/gridss), CNV caller CONTRA version 2.5 (http://contra-cnv.sourceforge.net/), and microsatellite instability detection algorithm mSINGS version 3.5 (https://bitbucket.org/uwlabmed/msings) as previously reported and described in more detail below, as well as custom scripts for TMB calculation [[7], [8], [9]].

GATK default parameters were utilized for SNV and indel analysis, while VarScan2 user-defined parameters for these variants were 0.03 minimum variant frequency and a minimum of 5 variant reads for SNVs and 0.01 minimum variant frequency and a minimum of 4 variant reads for indels. SNV and indel calls from GATK and VarScan2 were annotated with gene-based annotation, conservation scores, predicted effects at the protein level, population frequency, and internal UW database frequency using ANNOVAR version 2015_12_14 (http://www.openbioinformatics.org/annovar/). To ensure that key clinically important and/or pathogenic variants are reliably identified, a custom script utilizing SAMtools mpileup interrogated specific, predefined genomic positions, regardless of read depth. Using these parameters and scripts, SNVs and indels are routinely called from 5 to 7 bidirectional reads, though a molecular pathologist may, at their discretion, choose to call a variant with fewer reads depending on the background at the position in question across the history of the OPXv6 assay (internal UW frequency database).

Pindel was used to identify tandem duplications and indels greater than 10 bp in length [10]. Insertions, deletions, inversions, and inter- and intra-chromosomal translocations were detected using BreakDancer, which identifies paired-end sequencing reads mapped at unexpected separation distances or in an unexpected orientation [11], and GRIDSS, which filters out properly aligned reads and then uses the remaining discordantly aligned read pairs, split reads, soft-clipped reads, one-ended anchored read pairs, and indel-containing reads to perform genome-wide break-end assembly. In this process GRIDSS decomposes reads into k-mers, places them in a directed acyclic graph, identifies overlapping breakpoints, and then calls and scores variants (https://genome.cshlp.org/content/early/2017/11/02/gr.222109.117.abstract) [12]. Structural alterations detected by GRIDSS were annotated by AnnotSV version 1.1.1 (http://lbgi.fr/AnnotSV/). Prior to reporting, structural rearrangements are confirmed by manual review and evaluation of associated quality metrics including the number of split and discordant paired end reads (both of the sample in question and other samples within the run), the magnitude of the GRIDSS quality score, and mapping of the consensus sequence via the UCSC genome browser (BLAT).

CNV analysis was performed using CONTRA, which calculates and normalizes the read count and log ratio on a per-probe basis with NA12878 acting as reference [13]. Data was further analyzed by custom scripts to average log2 ratios across exons, with adjacent exons merged into larger segments for read depths that are not significantly different (p ​> ​0.0001 by Student’s t-test).

An mSINGS baseline was established for MSI detection for both NextSeq 500 and HiSeq 2500 by sequencing 18 MSI-high and 20 microsatellite stable (MSS) tumor samples that were previously characterized from the initial validation cohort on both instruments to identify mononucleotide microsatellite loci present within the capture design that were stable in negative control samples and unstable in positive control samples [9]. Fifty-nine loci were empirically identified with a cutoff fraction of 0.2 (20%) unstable loci determined to be the optimal threshold for detection of potential MSI-high samples. The mSINGS score (fraction of unstable loci out of 59 tested loci) of the 18 MSI-high samples used to establish the baseline ranged from 0.17 to 0.68 (median ​= ​0.34, SD ​= ​0.13) and 0 to 0.18 (median ​= ​0.07, SD ​= ​0.04) for MSS samples (Fig. 2).

Fig. 2

Distribution of mSINGS scores (the number of unstable loci out of 59 possible loci, expressed as a fraction) in the MSI-high and Microsatellite stable (MSS) samples used to establish the baseline for detection of MSI-high tumors. The threshold used for interpreting MSI status is indicated by a dashed line.

TMB classification was determined using a custom script to identify all coding or splice site variants absent from population databases (ExAC, 1000 genome, COSMIC, dbSNP), present in at least 8 reads with a variant allele frequency (VAF) of at least 5%, with an internal UW database frequency of less than 0.005 present in the tested sample (tumor only).

The complete clinical OPXv6 workflow from sample acquisition to data analysis and reporting is diagrammed in Fig. 3.

Fig. 3

The OncoPlex v6 Workflow consists of 6 main steps including genetics preanalytical services, sample preparation, library preparation, sequencing, processing through the bioinformatics pipeline, and data analysis.

Results

Sequencing metrics

Overall sequencing quality was greatly improved across the 14 validation sequencing runs compared to prior versions of UW-OncoPlex, with a 2.5-fold increase in non-duplicate average coverage (NextSeq: from 291X for OPXv5 to 541X for OPXv6, HiSeq: from 255X for OPXv5 to 747X for OPXv6) and a four-fold increase in average percent on-target sequencing (from 15% for OPXv5 to 60% for OPXv6 on NextSeq, and from 18% for OPXv5 to 62% for OPXv6 on HiSeq).

Samples passed the optimum quality control threshold if the average read depth across >90% of the genes was at least 150X, as this level of coverage is routinely capable of identifying single nucleotide variants with a 5% allele frequency. The corresponding overall mean target coverage of the full panel required to achieve this gene-level coverage was empirically determined by plotting the fraction of genes that met or exceeded the minimum coverage (150 or 200X) per sample against the sample’s respective average coverage across all 340 genes (Fig. 4). To achieve a minimum of 150X coverage for 90% of the 340 genes, approximately 350X average sample coverage is required, while an average coverage of 450X is required to obtain 200X coverage for 90% of the genes.

Fig. 4

Fraction of Genes Meeting or Exceeding Different Minimum Coverage Thresholds. Each data point represents one sample. Blue ​= ​200X, Orange ​= ​150X.

Four well-characterized HapMap reference samples were sequenced as part of establishing the performance characteristics of OPXv6 (NA12891, NA12892, NA18517, and NA12878), and detected variants were compared against results from the International HapMap SNP Consortium array and the 1000 Genomes Project [14,15]. Within the genomic regions targeted by OPXv6, 539, 521, 651, and 565 SNVs were present and detected in NA12891, NA12892, NA18517, and NA12878, respectively. Variants excluded from this analysis included alterations where complex indels resulted in misannotated SNP genotyping. The 565 variants present in NA12878 were included as a quality control measure by evaluating their recovery as a run-specific control sample. Across the 14 runs, accuracy at the 565 positions in NA12878 was 100% (7910/7910).

Variant analysis

The performance characteristics of OPXv6 are excellent for all tested variant classes (SNVs, Indels, SVs, and CNVs), both using standard protocols and in the setting of decreased DNA input and multiple methods of nucleic acid extraction (Table 2). Data used for the generation of this and subsequent tables were derived from variants identified in previously tested clinical samples, with the exception of specificity (which included both clinical and HapMap samples as discussed in additional detail below).

Table 2

Performance characteristics by alteration type.

Mutation Type	Sensitivity	Specificity	Reproducibility
			Inter-run	Intra-run
All SNVs, All VAFsb	98.7% (446/452)ad	100% (31,888/31,888)	99.7% (307/308)	98% (49/50)
excluding indels	98.9% (344/348)a	100% (28,085/28,085)	99.5% (220/221)	97.5% (39/40)
VAF ​≥ ​5%b	99% (440/444)ad		99.7% (306/307)	98% (48/49)c
VAF 5–10%b	96.4% (27/28)		95.2% (20/21)	100% (4/4)
VAF <5% b	75% (6/8)		100% (1/1)	100% (1/1)
SNVs 95% CI f (for VAF ≥5%)	97.7%–99.8%		98.2%–99.9%	89.2%–99.9%
All Indels	98.2% (110/112)ad	100% (3865/3865)	100% (89/89)	100% (10/10)
Indels ≥12 bp	100% (17/17)	100% (1349/1349)	100% (8/8)	100% (1/1)
Indels <12 bp	97.9% (93/95) d	100% (2516/2516)	100% (81/81)	100% (9/9)
Indels 95% CI f	93.7%–99.8%		95.9%–100%	69.2%–100%
All SVs	97.1% (34/35)		92.3% (12/13)	83.3% (5/6)e
SVs 95% CI f	85.1%–99.9%		64%–99.8%	35.9%–99.6%
All CNVs	98.8% (163/165)		98.2% (54/55)	92.9% (13/14)
Amplification	100% (17/17)		100% (8/8)	100% (1/1)
Homozygous deletion	100% (16/16)		100% (7/7)	100% (3/3)
Focal gain	100% (15/15)		83.3% (5/6)	100% (1/1)
Focal loss	100% (13/13)		100% (6/6)
Sub-chromosomal gain	97.7% (43/44)		100% (2/2)	100% (1/1)
Sub-chromosomal loss	98.3% (59/60)		100% (26/26)	87.5% (7/8)
CNVs 95% CI f	95.7%–99.9%		90.3%–100%	66.1%–99.8%

One sample was from DNA extracted from a separate tumor block that was analyzed at an outside hospital (OSH); it contained one SNV and one indel that were not detected by OPXv6.

Small indels, typically less than <12bp are identified by VarScan2 and GATK. They are therefore included both in the SNV calculations above (as specified) and in the indel and SV calculations depending on size.

Variant missed due to low coverage, exclusion results in 100% concordance for all variants and SNVs only with VAFs ≥10%.

PHOX2B (detected by an outside lab via Sanger sequencing) was not detected by OPXv6.

A single fusion was missed due to poor DNA quality using the standard DNA extraction method.

All CIs were calculated by the Clopper-Pearson method.

Single nucleotide variants and indels

To evaluate the performance characteristics of OPXv6 in the detection of SNVs and Indels, OPXv6 results were compared to those from orthogonal clinical test results, excluding results of the four HapMap samples. The analytical sensitivity of OPXv6 was excellent with 98.7% (446/452, 95% CI [97.1%–99.5%]) of the SNVs and indels detected at all variant allele frequencies (VAFs) (Table 2); moreover, VAFs determined by OPXv6 were highly correlated by linear regression with an R2 of 0.92 compared to previous clinical results (Fig. 5). For the six SNVs not detected, two were detected using DNA extracted from a different tumor block at an outside hospital, one was an indel in a complex homopolymer region (likely incorrectly annotated at an outside laboratory), and three were present at VAFs below 5%. SNVs and indels were further sub-categorized according to their VAFs: 5% or greater, 5–10%, and less than 5% (Table 2). OPXv6 detected 99% of the SNVs and indels (440/444) with VAFs above 5%; however, sensitivity decreased to 75% for VAFs below 5%. Our conservative limit of detection (LOD) for SNVs and indels was therefore 10% tumor purity for detection of a heterozygous variant.

Fig. 5

Linear regression demonstrating the correlation of VAFs between OPXv6 and orthogonal results.

Inter-run and intra-run concordance of all SNVs and indels showed excellent agreement, with only one discordant variant in each group (Table 2). The single discordant inter-run SNV had an allele frequency of 5%, while the intra-run SNV discordance was due to low coverage of 93X at the specific position in one replicate. For all indels, both inter-run and intra-run concordance was 100%, regardless of the size of the alteration.

An interrogation of 355 SNVs and indels that are clinically important and/or analytically difficult to detect was also employed, and was used to determine the specificity of OPXv6 at clinically-relevant genomic positions for all tested samples including HapMaps. Specificity was determined by comparing previously detected variants with VAFs greater than 5% on OPXv6. The specificity of OPXv6 was 100% (31,888/31,888 variants) for the 355 positions assessed across 90 validation samples, including 44 samples utilized for inter-run comparison and 9 samples for intra-run repeats (Table 2).

Structural variants (SVs)

OPXv6 detected 97.1% (34/35, 95% CI [85.1%–99.9%]) of targeted SVs, with only an MSH2-STAMBP1 rearrangement not detected (Table 3). Near-perfect repeatability and reproducibility was also observed for the tested SVs, with a single fusion discordant in each category. Specifically, an MSH2 inversion was not detected in one of the inter-run replicates, and an SQSTM1-NTRK1 fusion was not detected in an intra-run replicate due to low coverage of only 120X, due to poor DNA quality (Table 2).

Table 3

Structural variant assessment.

Type	Gene 1	Gene 2	Clinically Reported Breakpoints		Clinical Detection Method	Detected on v6
Fusion	ALK	STK32B	chr2:29447949	chr4:5475193	OPXv5	Y
Fusion	ALK	EML4	chr2:29446985	chr2:42543449	OPXv4	Y
Deletion	AR	AR	chrX:66935652	chrX:66943378	OPXv5	Y
Fusion	BRAF	KIAA1549	chr7:138554584	chr7:140491365	OPXv5	Y
Translocation	BRCA2	intergenic	chr13:32929064	chr1:31605622	OPXv5	Y
Fusion	DNAJB1	PRKACA	chr19:14222945	chr19:14628284	OPXv4	Y
Deletion	EGFR	EGFR	chr7:55120823	chr7:55223569	OPXv4	Y
Fusion	ERG	TMPRSS2	chr21:39871035	chr21:42874907	OPXv5	Y
Fusion	FGFR1	TACC1	chr8:38271170	chr8:38692626	OPXv5	Y
Fusion	FGFR2	NPM1	chr5:170,814,856	chr10:123,263,530	OPXv5	Y
Fusion	FGFR3	TACC3	chr4:1808681	chr4:1737103	OPXv5	Y
Inversion	KDM6A	Intergenic	chrX:44921996	chrX:68498474	OPXv5	Y
Fusion	MET	EXOC4	chr7:116435878	chr7:133440326	OPXv4	Y
Fusion	MET	TBC1D10C	chr11:67171961	chr7:116411487	OPXv5	Y
Deletion	MLH1	Intergenic	chr3:37091968	chr3:37237643	OPXv5	Y
Inversion	MSH2	intergenic	chr2:47669528	chr2:38121107	ColoSeq	Y
Inversion	MSH2	Intergenic	chr2:47647684	chr2:49777102	OPXv5	Y*
Fusion	MSH2	SRBD1	chr2:47655221	chr2:45816237	OPXv5	Y
Fusion	MSH2	STAMBP1	chr2:47641412	chr2:74060505	OPXv5	N**
Fusion	NTRK1	Intergenic	chr1:156843724	chr1:162342951	OPXv5	Y
Fusion	NTRK1	TPM3	NA	NA	ARCHER	Y
Fusion	NTRK3	ETV6	chr12:12018895	chr15:88614738	OPXv5	Y
Deletion	PDGFRA	PDGFRA	chr4:55136850	chr4:55139049	OPXv5	Y
Tandem Dup	PIK3R1	PIK3R1	chr5:67587753	chr5:67591150	OPXv5	Y
Fusion	RAF1	MKRN2	chr3:12616316	chr3:12641373	OPXv5	Y
Fusion	RELA	C11orf95	NA	NA	c/w histology	Y
Fusion	RELA	C11orf95	NA	NA	c/w histology	Y
Fusion	RELA	C11orf95	NA	NA	c/w histology	Y
Fusion	RET	CLIP2	NA	NA	OncoPanel and ARCHER	Y
Fusion	RET	KIF5B	chr10:43610740	chr10:32313817	ARCHER	Y
Fusion	RET	CCDC6	exon 12	exon 1	ARCHER	Y
Fusion	RET	SNRPN70	NA	NA	ARCHER	Y
Fusion	RET	PRMT7	NA	NA	OncoPanel and ARCHER	Y
Fusion	ROS1	CD74	chr5:149783823	chr6:117646454	OPXv5	Y
Fusion	RSPO3	PTPRK	chr6:127441024	chr6:128785759	OPXv5	Y

Y* Fusion detected in 2 of 3 runs, see concordance studies.

N** Fusion detected below cutoff quality (GRIDSS filter kept intact secondary to false positive rate).

Copy number variants (CNVs)

Clinically reported CNVs were assessed to determine the performance of OPXv6. Gene amplifications and homozygous deletions were detected with 100% sensitivity and precision (both between and within runs), while less robust single gene copy gain or copy loss had mildly decreased sensitivity (Table 2). Additionally, one focal gain was not detected with an inter-run replicate and one sub-chromosomal loss was not detected on an intra-run replicate (Table 2). Such findings are not surprising as low copy number gains and losses (both focal and sub/chromosomal alterations) are challenging to detect in the presence of FFPE artifact.

Microsatellite instability (MSI) and tumor mutational burden (TMB)

After establishing an mSINGS baseline for OPXv6 MSI detection for both the NextSeq 500 and HiSeq 2500 instruments using 18 MSI-high and 20 MSS FFPE tumor samples, we evaluated MSI in an additional 54 unique samples. We intentionally selected several cases that were very close to the positive threshold (0.2 mSINGS units) and/or demonstrated high signal-to-noise ratios on OPXv5. MSI by OPXv6 correctly classified 53/54 cases, including two cases that were false positives by prior clinical testing (Table 4). One case, a renal cell carcinoma, had an mSINGS score falsely increased by OPXv6 secondary to low mean sample coverage (mSINGS score 0.24 with an average coverage of 199X).

Table 4

Sensitivity and specificity of OPXv6 MSI and TMB.

Alteration	Concordance	Sensitivitya	Specificityb
mSINGS score	97.8% (90/92)c	96% (24/25)	98.5% (66/67)
MSI interpretation	100% (92/92)	100% (25/25)	100% (67/67)
MSI interpretation 95% CI e	96.1%–100%
TMB	97.7% (43/44)d
TMB 95% CI e	88%–99.9%

Number of OPXv6 positive mSINGS/number of OPXv5 positive mSINGS.

Number of OPXv6 negative mSINGS/number of OPXv5 negative mSINGS.

The OPXv6 MSI false positive and OPXv6 false negative mSINGS score (one each) were both corrected by director review.

A single major discordance was detected (changes across 2 categories).

All CIs were calculated by the Clopper-Pearson method.

In clinical practice, MSI status using OPXv6 is determined by evaluation of the sample in totality, including evaluation of TMB, tumor hypermutation signature, and assessment for the presence of MSI-inducing alterations. Using this approach, MSI interpretation concordance reached 100% after director’s review and blinded evaluation of the 54 samples (Table 4). Over the course of the validation and additional clinical runs, we noted that lower DNA input trended towards an increased mSINGS score. This trend did not result in any incorrect MSI classifications, when using the comprehensive approach outlined above, supporting the MSI interpretation lab policy requiring director review in conjunction with the mSINGS algorithm for determining MSI status.

Tumor mutational burden (TMB) is best established by exome sequencing [8]. Systematic comparison of TMB to exome sequencing was performed on OPXv5 for 20 samples (5 mismatch repair-deficient and 15 mismatch repair-intact tumor samples). From these 20 samples, the tumor mutation qualitative categories of low, intermediate, and high were established as 0–6 mutations/Mb of coding sequence, 7–11 mutations/Mb, and ≥12 mutations/Mb respectively. An additional validation set of 251 tumor samples from 32 distinct tumor types and 25 germline samples were subsequently evaluated on OPXv5, 21 of which were mismatch repair-deficient and 255 of which were mismatch repair-intact. This data recapitulated known trends in total mutation burden according to tumor type [16]. Paired OPXv5 and OPXv6 tumor cases were assessed for concordance of TMB and qualitatively classified into low, intermediate, and high according to the classification scheme above. Major discordances of TMB were defined as changes across 2 categories (from low to high or high to low), and minor discordances were defined as changing by only one category (e.g. from low to intermediate).

TMB was more challenging to analyze in initial test batches, as the historical error profile from PCR and sequencing artifacts of OPXv6 was still being established. Among OPXv6 samples with OPXv5 validated TMB data, 22/23 (96%) samples had no major discordances and 21/23 (91%) samples were perfectly concordant by TMB category (Table 4). The single major discordance was in a prostate cancer sample with clear evidence of MSI that is most likely a false negative result by OPXv5. We subsequently evaluated TMB in 21 samples prospectively on three clinical OPXv6 runs spanning a variety of tumor types (Fig. 1, Supplementary Table 1) in which TMB was adjusted based on well-established artifact profiles. While the TMB range did not appreciably change, concordance appeared to improve. For these cases, 19/21 (90%) had no major discordances and 21/21 (100%) were perfectly concordant with an R2 correlation of 0.97 (Fig. 6A). When the single case with a very high TMB was excluded the linear regression slightly decreased, with an R2 correlation of 0.91 (Fig. 6B).

Fig. 6

A and B Linear regression demonstrating TMB correlation between OPXv5 and OPXv6.

Effect of DNA input and DNA extraction methods

Prior OncoPlex versions ideally required at least 500 ​ng of DNA to obtain sufficiently high-quality libraries to yield robust sequence data. In order to assess the minimum DNA input quantity for OPXv6, 4 samples were prepared using two different DNA input quantities for shearing; “standard” DNA input averaged 236 ​ng, while the comparative “low” DNA input averaged 119 ​ng (See Supplementary Table 2). The concordance between the different DNA inputs was 100% across all alteration types (Table 5). One low abundance SNV that was detected at a VAF below 5% was not detected at either input level by OPXv6.

Table 5

Effect of DNA input and extraction methods to sensitivity of OPXv6.

Mutation Type	Standard Input	Low Input	DNA extraction	TNA extraction
SNVs	96.9% (31/32)	96.9% (31/32)	85.7% (18/21)	90.5% (19/21)
Indels	85.7% (6/7)	85.7% (6/7)	60% (3/5)	80% (4/5)
SVs	100% (1/1)	100% (1/1)	75% (3/4)	100% (4/4)
CNVs	100% (11/11)	100% (11/11)	100% (3/3)	66.7% (2/3)

TNA = total nucleic acid.

Due to the variety of samples submitted for clinical testing in our laboratory, multiple DNA isolation kits are in clinical production, with the specific extraction method used depending on the clinical specimen being extracted. The most common extraction protocol uses the Qiagen GeneRead DNA FFPE Kit DNA extraction kit, which was compared with the total nucleic acid (TNA) extraction followed by DNA isolation, either as part of DNA/RNA AllPrep or via Qiagen GeneRead DNA FFPE Kit DNA extraction. Samples processed by TNA extraction followed by DNA isolation demonstrated improved performance for all alteration types except for low copy number alterations. A single SNV (SLX4 p.T84A) was identified only after TNA extraction, as the GeneRead DNA FFPE kit extraction from the same FFPE sample failed average coverage quality metrics (96X for the GeneRead DNA FFPE kit compared to 521X for the DNA/RNA AllPrep kit). An SQSTM1-NTRK1 fusion was also not detected using the standard DNA isolation on an intra-run replicate due to similarly failed coverage metrics likely due to poor DNA quality (114X compared to 574X). Higher quality scores by GRIDSS were observed for all structural variants detected by TNA extraction. For CNVs, findings were very similar between the extraction kits.

Discussion

Since the UW-OncoPlex assay was first clinically offered in August of 2012, there have been four major assay updates, each containing increasing numbers of clinically relevant genes and expanded biomarker assessment necessary for comprehensive tumor profiling. Though the sequencing technology and the supporting bioinformatics pipelines have evolved alongside each update, the method for target-gene enrichment has remained static. OPXv6 replaces the fixed content targeted hybridization-based capture design used in previous assays (Agilent SureSelect, Agilent Technologies, Santa Clara CA) with a modular design (IDT xGen Lockdown Probes, IDT, Coralville IA) capable of the more frequent updates necessary to maintain pace with clinical discovery. In addition to its modular design update, OPXv6 also expanded the number of targeted genes from 262 to 340 genes (including 24 genes with recurrent structural alterations), which were chosen for their clinical significance across a wide range of neoplasms, and updated the associated pipelines for optimal variant detection.

Ultimately, this most recent iteration of OncoPlex (OPXv6) achieved a sensitivity of 99% for SNVs, 98% for indels, 97% for SVs, 99% for CNVs, 100% for MSI, and 100% for TMB compared to prior orthogonal methods. Reproducibility and repeatability were likewise high for all variant types, and specificity of SNV and indel detection was 100%. We also investigated the limit of detection of SNVs and indels and determined that we could reliably detect 5% variant allele frequency, suggesting that although we have set a minimum tumor content of 20% neoplastic cell nuclei, samples with as little as 10% tumor content can be interpreted with discretion and a caveat statement indicating that false-negative results cannot be entirely excluded in the setting of low tumor content. DNA input requirements were also evaluated, and between the validation cohort and subsequent clinical runs, we determined that OPXv6 had the ability to detect SNVs, indels, SVs, and CNVs in the setting of extremely low DNA quantities (less than 1 ng/μL in 40 μL). This evaluation, however, highlighted an assay-specific limitation - that lower DNA input trended towards an increased mSINGS score, which could result in false-positive reporting of MSI-high status, if evaluated in isolation. This phenomenon did not result in any incorrect classifications, as our process requires director interpretation and review of the mSINGS score in the context of all other genomic findings (i.e. detection of pathogenic mismatch repair mutations, increased tumor burden, and/or presence/absence of MLH1 methylation) and the clinical context. Of note, the MSI-high samples evaluated to identify unstable loci for inclusion in the mSINGS algorithm consisted of colon, endometrial, and prostate cancer, while a broader range of tumor types was included in the cohort of MSS samples. Decreased sensitivity for the detection of low copy number changes (similar to prior versions) was also highlighted as a limitation, which is to be expected given that they are difficult to detect in the setting of FFPE artifact. Overall, the performance characteristics of OPXv6 are comparable to previously described assays [7,17,18].

Clinical assays used to assess genetic alterations in the setting of neoplastic processes are often presented with samples that are sub-optimal, which can challenge the analytical and interpretive capabilities of clinical assays. In addition, as our understanding of biology, medicine, and the natural history of diseases have progressed, new and expanded types of samples are potential substrates for genetic assessment. Thus, we challenged OPXv6 using a broad range of sample types (cell-free DNA, saliva, FFPE tissue, fresh-frozen tissue, peripheral blood, and bone marrow aspirates) that were orthogonally evaluated using multiple clinical methods, compared across two Illumina sequencing platforms, and extracted DNA with multiple nucleic acid isolation methods. We demonstrated that OPXv6 was capable of identifying all classes of relevant genomic findings in all sample types tested, including circulating tumor (ctDNA), and was concordant with prior methods. Additionally, we determined that DNA isolated following total nucleic acid (TNA) extraction demonstrated comparable results and showed a trend towards improved sequencing metrics and variant detection. More importantly, in the context of a negative DNA UW-OncoPlex test, RNA isolated as part of the TNA extraction was immediately available for potential reflex testing. Currently, this approach is standard for pediatric neoplasms, where fusions can be more common.

In addition to the advantages realized in analytical performance, scalability, and capture updates, the adoption of the modular IDT xGen lockdown probes, and their accompanying workflow, resulted in numerous time-saving process improvements. The new process resulted in a 40% reduction of OPXv6 library preparation turn-around time, requiring only 3 days from DNA shearing to sequencer loading compared to a 5-day workflow for prior UW-OncoPlex versions. Time savings arose from the ligation of adapters on day 1 of library preparation, which allowed for multiplex hybridization capture with associated decreased pipetting, reduced hybridization time of 4 ​h (compared to 24 ​h), and shorter wash steps (from 3 to 1.5 ​h). Another advantage is that the pooled specimen capture by hybridizing in a single Eppendorf tube early in the preparation process not only reduced pipetting, which limited potential for cross contamination or mix up of samples during the workflow, but also decreased required reagent amounts. In addition to the laboratory workflow optimization, the substitution of a new SV caller (GRIDSS) further decreased turn-around time, resulting in sequencing data becoming available in as little as 4–5 days after initiation of testing. The decreased time to data allows for more rapid reporting of clinically significant findings, but also will result in lower overall cost-per-result due to decreased effort from laboratory staff and more efficient reagent utilization.

The implementation of the IDT xGen chemistry resulted in significant increases in average overall coverage and percent on-target rate of OPXv6 compared to prior OncoPlex versions, with a 2.5-fold and 4-fold increase, respectively. These improved coverage metrics likely stemmed from improved efficiency of probe hybridization to target sequences (secondary to the use of xGen Blocking Oligos and Human Cot-1 DNA®, which resulted in reduced non-specific adapter binding and blockage of repetitive DNA sequences) and increased on-target sequencing rate [19,20]. This increased coverage has significant implications in clinical cases with low tumor content and/or poor sequencing quality, as was highlighted by several of the validation cohort samples which included clinical cases where prior UW-OncoPlex testing failed to detect clinically significant mutations that were subsequently detected by an alternative methodology. One such example detected by OPXv6 is a TPM3-NTRK1 fusion in a papillary thyroid carcinoma, which may be amenable to treatment with the FDA-approved TRK inhibitor larotrectinib [[21], [22], [23]]. Improved coverage and on-target rates will allow for improved detection of clinically-relevant findings in precious clinical samples, not only providing important precision medicine data, but also potentially reducing the need for additional, invasive procedures to obtain more tissue for additional laboratory studies.

As precision medicine has evolved from narrowly defined testing indications and therapies targeting specific alterations in characteristic tumor types to treatments that are tumor agnostic, the need for diagnostic platforms to accommodate more and varied targets has emerged. This evolution in understanding and applications drives technological advances, which in turn further inform our understanding of cancer etiology and progression. We believe that OPXv6, and other comprehensive somatic profiling assays, exemplifies the analytical and performance capabilities needed to keep pace with the rapidly expanding understanding of genetic markers important to oncology diagnostics and rapid pace of clinical care. Given the excellent analytical performance, comprehensive content, reduced turn-around time, and decreased cost, OPXv6 has become the single platform for clinician-directed tumor testing at the University of Washington. Additionally, implementation of OPXv6 allowed for replacement of multiple assays, including OPXv5 and a rapid, smaller NGS panel targeting a limited set of frequently mutated oncogenes. This powerful tool, when combined with expert interpretation and data assessment, will enable us to continue to offer choices instead of condolences to those diagnosed with cancer.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

CRediT authorship contribution statement

Ayako J. Kuo: Conceptualization, Validation, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review & editing, Visualization. Vera A. Paulson: Conceptualization, Validation, Formal analysis, Investigation, Data curation, Writing - original draft, Writing - review & editing, Visualization. Jennifer A. Hempelmann: Validation, Investigation, Data curation, Writing - review & editing. Mallory Beightol: Validation, Investigation. Sheena Todhunter: Software, Validation, Investigation, Data curation. Brice G. Colbert: Software, Data curation. Stephen J. Salipante: Conceptualization, Software, Investigation, Resources. Eric Q. Konnick: Conceptualization, Resources, Validation, Writing - review & editing. Colin C. Pritchard: Conceptualization, Validation, Resources, Writing - review & editing. Christina M. Lockwood: Conceptualization, Validation, Resources, Writing - original draft, Writing - review & editing, Supervision.

Declaration of competing interest

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: CC Pritchard and SJ Salipante: consult for Promega EQ Konnick: Travel & honoraria – Ventana, Medscape, Roche, Genentech CM Lockwood: spouse is employed by Bayer.