Jeremy Hyrkas1, Sophie Clayton2, Francois Ribalet2, Daniel Halperin3, E Virginia Armbrust4, Bill Howe3. 1. Department of Computer Science and Engineering. 2. School of Oceanography and. 3. Department of Computer Science and Engineering, eScience Institute, University of Washington, Seattle, WA 98195, USA. 4. School of Oceanography and eScience Institute, University of Washington, Seattle, WA 98195, USA.
Abstract
MOTIVATION: Recent technological innovations in flow cytometry now allow oceanographers to collect high-frequency flow cytometry data from particles in aquatic environments on a scale far surpassing conventional flow cytometers. The SeaFlow cytometer continuously profiles microbial phytoplankton populations across thousands of kilometers of the surface ocean. The data streams produced by instruments such as SeaFlow challenge the traditional sample-by-sample approach in cytometric analysis and highlight the need for scalable clustering algorithms to extract population information from these large-scale, high-frequency flow cytometers. RESULTS: We explore how available algorithms commonly used for medical applications perform at classification of such a large-scale, environmental flow cytometry data. We apply large-scale Gaussian mixture models to massive datasets using Hadoop. This approach outperforms current state-of-the-art cytometry classification algorithms in accuracy and can be coupled with manual or automatic partitioning of data into homogeneous sections for further classification gains. We propose the Gaussian mixture model with partitioning approach for classification of large-scale, high-frequency flow cytometry data. AVAILABILITY AND IMPLEMENTATION: Source code available for download at https://github.com/jhyrkas/seaflow_cluster, implemented in Java for use with Hadoop. CONTACT: hyrkas@cs.washington.edu SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.
MOTIVATION: Recent technological innovations in flow cytometry now allow oceanographers to collect high-frequency flow cytometry data from particles in aquatic environments on a scale far surpassing conventional flow cytometers. The SeaFlow cytometer continuously profiles microbial phytoplankton populations across thousands of kilometers of the surface ocean. The data streams produced by instruments such as SeaFlow challenge the traditional sample-by-sample approach in cytometric analysis and highlight the need for scalable clustering algorithms to extract population information from these large-scale, high-frequency flow cytometers. RESULTS: We explore how available algorithms commonly used for medical applications perform at classification of such a large-scale, environmental flow cytometry data. We apply large-scale Gaussian mixture models to massive datasets using Hadoop. This approach outperforms current state-of-the-art cytometry classification algorithms in accuracy and can be coupled with manual or automatic partitioning of data into homogeneous sections for further classification gains. We propose the Gaussian mixture model with partitioning approach for classification of large-scale, high-frequency flow cytometry data. AVAILABILITY AND IMPLEMENTATION: Source code available for download at https://github.com/jhyrkas/seaflow_cluster, implemented in Java for use with Hadoop. CONTACT: hyrkas@cs.washington.edu SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.