Cody Glickman1,2, Jo Hendrix3,4, Michael Strong3,4. 1. Center for Genes, Environment, and Health, National Jewish Health, 1400 Jackson Street, Denver, CO, 80206, USA. cody.glickman@cuanschutz.edu. 2. Computational Bioscience, University of Colorado Anschutz, 12801 E 17th Avenue, Aurora, CO, 80045, USA. cody.glickman@cuanschutz.edu. 3. Center for Genes, Environment, and Health, National Jewish Health, 1400 Jackson Street, Denver, CO, 80206, USA. 4. Computational Bioscience, University of Colorado Anschutz, 12801 E 17th Avenue, Aurora, CO, 80045, USA.
Abstract
BACKGROUND: Viruses, including bacteriophages, are important components of environmental and human associated microbial communities. Viruses can act as extracellular reservoirs of bacterial genes, can mediate microbiome dynamics, and can influence the virulence of clinical pathogens. Various targeted metagenomic analysis techniques detect viral sequences, but these methods often exclude large and genome integrated viruses. In this study, we evaluate and compare the ability of nine state-of-the-art bioinformatic tools, including Vibrant, VirSorter, VirSorter2, VirFinder, DeepVirFinder, MetaPhinder, Kraken 2, Phybrid, and a BLAST search using identified proteins from the Earth Virome Pipeline to identify viral contiguous sequences (contigs) across simulated metagenomes with different read distributions, taxonomic compositions, and complexities. RESULTS: Of the tools tested in this study, VirSorter achieved the best F1 score while Vibrant had the highest average F1 score at predicting integrated prophages. Though less balanced in its precision and recall, Kraken2 had the highest average precision by a substantial margin. We introduced the machine learning tool, Phybrid, which demonstrated an improvement in average F1 score over tools such as MetaPhinder. The tool utilizes machine learning with both gene content and nucleotide features. The addition of nucleotide features improves the precision and recall compared to the gene content features alone.Viral identification by all tools was not impacted by underlying read distribution but did improve with contig length. Tool performance was inversely related to taxonomic complexity and varied by the phage host. For instance, Rhizobium and Enterococcus phages were identified consistently by the tools; whereas, Neisseria prophage sequences were commonly missed in this study. CONCLUSION: This study benchmarked the performance of nine state-of-the-art bioinformatic tools to identify viral contigs across different simulation conditions. This study explored the ability of the tools to identify integrated prophage elements traditionally excluded from targeted sequencing approaches. Our comprehensive analysis of viral identification tools to assess their performance in a variety of situations provides valuable insights to viral researchers looking to mine viral elements from publicly available metagenomic data.
BACKGROUND: Viruses, including bacteriophages, are important components of environmental and human associated microbial communities. Viruses can act as extracellular reservoirs of bacterial genes, can mediate microbiome dynamics, and can influence the virulence of clinical pathogens. Various targeted metagenomic analysis techniques detect viral sequences, but these methods often exclude large and genome integrated viruses. In this study, we evaluate and compare the ability of nine state-of-the-art bioinformatic tools, including Vibrant, VirSorter, VirSorter2, VirFinder, DeepVirFinder, MetaPhinder, Kraken 2, Phybrid, and a BLAST search using identified proteins from the Earth Virome Pipeline to identify viral contiguous sequences (contigs) across simulated metagenomes with different read distributions, taxonomic compositions, and complexities. RESULTS: Of the tools tested in this study, VirSorter achieved the best F1 score while Vibrant had the highest average F1 score at predicting integrated prophages. Though less balanced in its precision and recall, Kraken2 had the highest average precision by a substantial margin. We introduced the machine learning tool, Phybrid, which demonstrated an improvement in average F1 score over tools such as MetaPhinder. The tool utilizes machine learning with both gene content and nucleotide features. The addition of nucleotide features improves the precision and recall compared to the gene content features alone.Viral identification by all tools was not impacted by underlying read distribution but did improve with contig length. Tool performance was inversely related to taxonomic complexity and varied by the phage host. For instance, Rhizobium and Enterococcus phages were identified consistently by the tools; whereas, Neisseria prophage sequences were commonly missed in this study. CONCLUSION: This study benchmarked the performance of nine state-of-the-art bioinformatic tools to identify viral contigs across different simulation conditions. This study explored the ability of the tools to identify integrated prophage elements traditionally excluded from targeted sequencing approaches. Our comprehensive analysis of viral identification tools to assess their performance in a variety of situations provides valuable insights to viral researchers looking to mine viral elements from publicly available metagenomic data.
Authors: David Paez-Espino; Emiley A Eloe-Fadrosh; Georgios A Pavlopoulos; Alex D Thomas; Marcel Huntemann; Natalia Mikhailova; Edward Rubin; Natalia N Ivanova; Nikos C Kyrpides Journal: Nature Date: 2016-08-17 Impact factor: 49.962
Authors: Samuel Minot; Alexandra Bryson; Christel Chehoud; Gary D Wu; James D Lewis; Frederic D Bushman Journal: Proc Natl Acad Sci U S A Date: 2013-07-08 Impact factor: 11.205
Authors: Josef Wagner; Jovana Maksimovic; Gabriella Farries; Winnie H Sim; Ruth F Bishop; Donald J Cameron; Anthony G Catto-Smith; Carl D Kirkwood Journal: Inflamm Bowel Dis Date: 2013-07 Impact factor: 5.325
Authors: Jeremy J Barr; Rita Auro; Mike Furlan; Katrine L Whiteson; Marcella L Erb; Joe Pogliano; Aleksandr Stotland; Roland Wolkowicz; Andrew S Cutting; Kelly S Doran; Peter Salamon; Merry Youle; Forest Rohwer Journal: Proc Natl Acad Sci U S A Date: 2013-05-20 Impact factor: 11.205
Authors: Lasha Gogokhia; Kate Buhrke; Rickesha Bell; Brenden Hoffman; D Garrett Brown; Christin Hanke-Gogokhia; Nadim J Ajami; Matthew C Wong; Arevik Ghazaryan; John F Valentine; Nathan Porter; Eric Martens; Ryan O'Connell; Vinita Jacob; Ellen Scherl; Carl Crawford; W Zac Stephens; Sherwood R Casjens; Randy S Longman; June L Round Journal: Cell Host Microbe Date: 2019-02-13 Impact factor: 21.023