Hesham ElAbd1, Yana Bromberg2,3,4, Adrienne Hoarfrost2, Tobias Lenz5, Andre Franke6, Mareike Wendorff1. 1. Institute of Clinical Molecular Biology, Christian-Albrechts-University of Kiel, Kiel, Germany. 2. Department of Biochemistry and Microbiology, Rutgers University, New Brunswick, NJ, USA. 3. Department of Genetics, Rutgers University, New Brunswick, NJ, USA. 4. Technical University of Munich Institute for Advanced Study, (TUM-IAS), Lichtenbergstr. 2a, 85748, Garching/Munich, Germany. 5. Research Group for Evolutionary Immunogenomics, Max Planck Institute for Evolutionary Biology, 24306, Plön, Germany. 6. Institute of Clinical Molecular Biology, Christian-Albrechts-University of Kiel, Kiel, Germany. a.franke@mucosa.de.
Abstract
BACKGROUND: The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction - a process called 'end-to-end learning' - has led to state-of-the-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN. RESULTS: By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension. CONCLUSION: Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.
BACKGROUND: The number of applications of deep learning algorithms in bioinformatics is increasing as they usually achieve superior performance over classical approaches, especially, when bigger training datasets are available. In deep learning applications, discrete data, e.g. words or n-grams in language, or amino acids or nucleotides in bioinformatics, are generally represented as a continuous vector through an embedding matrix. Recently, learning this embedding matrix directly from the data as part of the continuous iteration of the model to optimize the target prediction - a process called 'end-to-end learning' - has led to state-of-the-art results in many fields. Although usage of embeddings is well described in the bioinformatics literature, the potential of end-to-end learning for single amino acids, as compared to more classical manually-curated encoding strategies, has not been systematically addressed. To this end, we compared classical encoding matrices, namely one-hot, VHSE8 and BLOSUM62, to end-to-end learning of amino acid embeddings for two different prediction tasks using three widely used architectures, namely recurrent neural networks (RNN), convolutional neural networks (CNN), and the hybrid CNN-RNN. RESULTS: By using different deep learning architectures, we show that end-to-end learning is on par with classical encodings for embeddings of the same dimension even when limited training data is available, and might allow for a reduction in the embedding dimension without performance loss, which is critical when deploying the models to devices with limited computational capacities. We found that the embedding dimension is a major factor in controlling the model performance. Surprisingly, we observed that deep learning models are capable of learning from random vectors of appropriate dimension. CONCLUSION: Our study shows that end-to-end learning is a flexible and powerful method for amino acid encoding. Further, due to the flexibility of deep learning systems, amino acid encoding schemes should be benchmarked against random vectors of the same dimension to disentangle the information content provided by the encoding scheme from the distinguishability effect provided by the scheme.
Authors: Muhammad Nabeel Asim; Muhammad Ali Ibrahim; Christoph Zehe; Johan Trygg; Andreas Dengel; Sheraz Ahmed Journal: Interdiscip Sci Date: 2022-08-10 Impact factor: 3.492
Authors: Faheem Ahmed; Jae Wook Lee; Anupama Samantasinghar; Young Su Kim; Kyung Hwan Kim; In Suk Kang; Fida Hussain Memon; Jong Hwan Lim; Kyung Hyun Choi Journal: Front Public Health Date: 2022-06-16
Authors: Roman Feldbauer; Lukas Gosch; Lukas Lüftinger; Patrick Hyden; Arthur Flexer; Thomas Rattei Journal: Bioinformatics Date: 2020-12-26 Impact factor: 6.937
Authors: Jefferson Daniel Suquilanda-Pesántez; Evelyn Dayana Aguiar Salazar; Diego Almeida-Galárraga; Graciela Salum; Fernando Villalba-Meneses; Marco Esteban Gudiño Gomezjurado Journal: F1000Res Date: 2022-02-09