Mathieu Hatt1, Baptiste Laurent2, Anouar Ouahabi2, Hadi Fayad2, Shan Tan3, Laquan Li3, Wei Lu4, Vincent Jaouen2, Clovis Tauber5, Jakub Czakon6, Filip Drapejkowski6, Witold Dyrka7, Sorina Camarasu-Pop8, Frédéric Cervenansky8, Pascal Girard8, Tristan Glatard9, Michael Kain10, Yao Yao10, Christian Barillot10, Assen Kirov4, Dimitris Visvikis2. 1. LaTIM, UMR 1101, INSERM, IBSAM, UBO, UBL, Brest, France. Electronic address: hatt@univ-brest.fr. 2. LaTIM, UMR 1101, INSERM, IBSAM, UBO, UBL, Brest, France. 3. Key Laboratory of Image Processing and Intelligent Control of Ministry of Education of China, School of Automation, Huazhong University of Science and Technology, Wuhan 430074, China. 4. Memorial Sloan-Kettering Cancer Center, New-York, USA. 5. INSERM, UMR 930, Imaging and brain, University of Tours, France. 6. Stermedia Sp. z o. o., ul. A. Ostrowskiego 13, Wroclaw, Poland. 7. Stermedia Sp. z o. o., ul. A. Ostrowskiego 13, Wroclaw, Poland; Wroclaw University of Science and Technology, Faculty of Fundamental Problems of Technology, Department of Biomedical Engineering, Poland. 8. Université de Lyon, CREATIS, CNRS UMR5220, INSERM UMR 1044, INSA-Lyon, Université Lyon 1, Lyon, France. 9. Department of Computer Science and Software Engineering, Concordia University, Montreal, Canada. 10. INRIA, Visages project-team, CNRS, IRISA 6074, INSERM, Visages, UMR 1228, University of Rennes I, Rennes Cx 35042, France.
Abstract
INTRODUCTION: Automatic functional volume segmentation in PET images is a challenge that has been addressed using a large array of methods. A major limitation for the field has been the lack of a benchmark dataset that would allow direct comparison of the results in the various publications. In the present work, we describe a comparison of recent methods on a large dataset following recommendations by the American Association of Physicists in Medicine (AAPM) task group (TG) 211, which was carried out within a MICCAI (Medical Image Computing and Computer Assisted Intervention) challenge. MATERIALS AND METHODS: Organization and funding was provided by France Life Imaging (FLI). A dataset of 176 images combining simulated, phantom and clinical images was assembled. A website allowed the participants to register and download training data (n = 19). Challengers then submitted encapsulated pipelines on an online platform that autonomously ran the algorithms on the testing data (n = 157) and evaluated the results. The methods were ranked according to the arithmetic mean of sensitivity and positive predictive value. RESULTS: Sixteen teams registered but only four provided manuscripts and pipeline(s) for a total of 10 methods. In addition, results using two thresholds and the Fuzzy Locally Adaptive Bayesian (FLAB) were generated. All competing methods except one performed with median accuracy above 0.8. The method with the highest score was the convolutional neural network-based segmentation, which significantly outperformed 9 out of 12 of the other methods, but not the improved K-Means, Gaussian Model Mixture and Fuzzy C-Means methods. CONCLUSION: The most rigorous comparative study of PET segmentation algorithms to date was carried out using a dataset that is the largest used in such studies so far. The hierarchy amongst the methods in terms of accuracy did not depend strongly on the subset of datasets or the metrics (or combination of metrics). All the methods submitted by the challengers except one demonstrated good performance with median accuracy scores above 0.8.
INTRODUCTION: Automatic functional volume segmentation in PET images is a challenge that has been addressed using a large array of methods. A major limitation for the field has been the lack of a benchmark dataset that would allow direct comparison of the results in the various publications. In the present work, we describe a comparison of recent methods on a large dataset following recommendations by the American Association of Physicists in Medicine (AAPM) task group (TG) 211, which was carried out within a MICCAI (Medical Image Computing and Computer Assisted Intervention) challenge. MATERIALS AND METHODS: Organization and funding was provided by France Life Imaging (FLI). A dataset of 176 images combining simulated, phantom and clinical images was assembled. A website allowed the participants to register and download training data (n = 19). Challengers then submitted encapsulated pipelines on an online platform that autonomously ran the algorithms on the testing data (n = 157) and evaluated the results. The methods were ranked according to the arithmetic mean of sensitivity and positive predictive value. RESULTS: Sixteen teams registered but only four provided manuscripts and pipeline(s) for a total of 10 methods. In addition, results using two thresholds and the Fuzzy Locally Adaptive Bayesian (FLAB) were generated. All competing methods except one performed with median accuracy above 0.8. The method with the highest score was the convolutional neural network-based segmentation, which significantly outperformed 9 out of 12 of the other methods, but not the improved K-Means, Gaussian Model Mixture and Fuzzy C-Means methods. CONCLUSION: The most rigorous comparative study of PET segmentation algorithms to date was carried out using a dataset that is the largest used in such studies so far. The hierarchy amongst the methods in terms of accuracy did not depend strongly on the subset of datasets or the metrics (or combination of metrics). All the methods submitted by the challengers except one demonstrated good performance with median accuracy scores above 0.8.
Authors: Kurt G Schilling; Alessandro Daducci; Klaus Maier-Hein; Cyril Poupon; Jean-Christophe Houde; Vishwesh Nath; Adam W Anderson; Bennett A Landman; Maxime Descoteaux Journal: Magn Reson Imaging Date: 2018-11-29 Impact factor: 2.546
Authors: Andrei Gafita; Marie Bieth; Markus Krönke; Giles Tetteh; Fernando Navarro; Hui Wang; Elisabeth Günther; Bjoern Menze; Wolfgang A Weber; Matthias Eiber Journal: J Nucl Med Date: 2019-03-08 Impact factor: 10.057
Authors: Tonghe Wang; Yang Lei; Yabo Fu; Walter J Curran; Tian Liu; Jonathon A Nye; Xiaofeng Yang Journal: Phys Med Date: 2020-07-29 Impact factor: 2.685
Authors: Albert Comelli; Samuel Bignardi; Alessandro Stefano; Giorgio Russo; Maria Gabriella Sabini; Massimo Ippolito; Anthony Yezzi Journal: Comput Biol Med Date: 2020-03-16 Impact factor: 4.589