Eric J Duncavage1, Haley J Abel, Jason D Merker, John B Bodner, Qin Zhao, Karl V Voelkerding, John D Pfeifer. 1. From the Departments of Pathology (Drs Duncavage and Pfeifer) and Genetics (Dr Abel), Washington University School of Medicine, St Louis, Missouri; the Department of Pathology (Dr Merker), Stanford University School of Medicine, Stanford, California; Product Development, Laboratory Improvement Program (Mr Bodner), and the Surveys Department (Dr Zhao), College of American Pathologists, Northfield, Illinois; and the Department of Pathology and ARUP Laboratories, University of Utah, Salt Lake City (Dr Voelkerding).
Abstract
CONTEXT: -Most current proficiency testing challenges for next-generation sequencing assays are methods-based proficiency testing surveys that use DNA from characterized reference samples to test both the wet-bench and bioinformatics/dry-bench aspects of the tests. Methods-based proficiency testing surveys are limited by the number and types of mutations that either are naturally present or can be introduced into a single DNA sample. OBJECTIVE: -To address these limitations by exploring a model of in silico proficiency testing in which sequence data from a single well-characterized specimen are manipulated electronically. DESIGN: -DNA from the College of American Pathologists reference genome was enriched using the Illumina TruSeq and Life Technologies AmpliSeq panels and sequenced on the MiSeq and Ion Torrent platforms, respectively. The resulting data were mutagenized in silico and 26 variants, including single-nucleotide variants, deletions, and dinucleotide substitutions, were added at variant allele fractions (VAFs) from 10% to 50%. Participating clinical laboratories downloaded these files and analyzed them using their clinical bioinformatics pipelines. RESULTS: -Laboratories using the AmpliSeq/Ion Torrent and/or the TruSeq/MiSeq participated in the 2 surveys. On average, laboratories identified 24.6 of 26 variants (95%) overall and 21.4 of 22 variants (97%) with VAFs greater than 15%. No false-positive calls were reported. The most frequently missed variants were single-nucleotide variants with VAFs less than 15%. Across both challenges, reported VAF concordance was excellent, with less than 1% median absolute difference between the simulated VAF and mean reported VAF. CONCLUSIONS: -The results indicate that in silico proficiency testing is a feasible approach for methods-based proficiency testing, and demonstrate that the sensitivity and specificity of current next-generation sequencing bioinformatics across clinical laboratories are high.
CONTEXT: -Most current proficiency testing challenges for next-generation sequencing assays are methods-based proficiency testing surveys that use DNA from characterized reference samples to test both the wet-bench and bioinformatics/dry-bench aspects of the tests. Methods-based proficiency testing surveys are limited by the number and types of mutations that either are naturally present or can be introduced into a single DNA sample. OBJECTIVE: -To address these limitations by exploring a model of in silico proficiency testing in which sequence data from a single well-characterized specimen are manipulated electronically. DESIGN: -DNA from the College of American Pathologists reference genome was enriched using the Illumina TruSeq and Life Technologies AmpliSeq panels and sequenced on the MiSeq and Ion Torrent platforms, respectively. The resulting data were mutagenized in silico and 26 variants, including single-nucleotide variants, deletions, and dinucleotide substitutions, were added at variant allele fractions (VAFs) from 10% to 50%. Participating clinical laboratories downloaded these files and analyzed them using their clinical bioinformatics pipelines. RESULTS: -Laboratories using the AmpliSeq/Ion Torrent and/or the TruSeq/MiSeq participated in the 2 surveys. On average, laboratories identified 24.6 of 26 variants (95%) overall and 21.4 of 22 variants (97%) with VAFs greater than 15%. No false-positive calls were reported. The most frequently missed variants were single-nucleotide variants with VAFs less than 15%. Across both challenges, reported VAF concordance was excellent, with less than 1% median absolute difference between the simulated VAF and mean reported VAF. CONCLUSIONS: -The results indicate that in silico proficiency testing is a feasible approach for methods-based proficiency testing, and demonstrate that the sensitivity and specificity of current next-generation sequencing bioinformatics across clinical laboratories are high.
Authors: Lawrence J Jennings; Maria E Arcila; Christopher Corless; Suzanne Kamel-Reid; Ira M Lubin; John Pfeifer; Robyn L Temple-Smolkin; Karl V Voelkerding; Marina N Nikiforova Journal: J Mol Diagn Date: 2017-03-21 Impact factor: 5.568
Authors: Scott Quainoo; Jordy P M Coolen; Sacha A F T van Hijum; Martijn A Huynen; Willem J G Melchers; Willem van Schaik; Heiman F L Wertheim Journal: Clin Microbiol Rev Date: 2017-10 Impact factor: 26.132
Authors: Jason D Merker; Kelly Devereaux; A John Iafrate; Suzanne Kamel-Reid; Annette S Kim; Joel T Moncur; Stephen B Montgomery; Rakesh Nagarajan; Bryce P Portier; Mark J Routbort; Craig Smail; Lea F Surrey; Patricia Vasalos; Alexander J Lazar; Neal I Lindeman Journal: Arch Pathol Lab Med Date: 2018-10-30 Impact factor: 5.534
Authors: Emma Wilcox; Steven M Harrison; Edward Lockhart; Karl Voelkerding; Ira M Lubin; Heidi L Rehm; Lisa V Kalman; Birgit Funke Journal: J Mol Diagn Date: 2021-08-09 Impact factor: 5.568