H A Haenssle1, C Fink1, R Schneiderbauer1, F Toberer1, T Buhl2, A Blum3, A Kalloo4, A Ben Hadj Hassen5, L Thomas6, A Enk1, L Uhlmann7. 1. Department of Dermatology, University of Heidelberg, Heidelberg, Germany. 2. Department of Dermatology, University of Göttingen, Göttingen, Germany. 3. Office Based Clinic of Dermatology, Konstanz, Germany. 4. Dermatology Service, Department of Medicine, Memorial Sloan Kettering Cancer Center, New York, USA. 5. Faculty of Computer Science and Mathematics, University of Passau, Passau, Germany. 6. Department of Dermatology, Lyons Cancer Research Center, Lyon 1 University, Lyon, France. 7. Institute of Medical Biometry and Informatics, University of Heidelberg, Heidelberg, Germany.
Abstract
Background: Deep learning convolutional neural networks (CNN) may facilitate melanoma detection, but data comparing a CNN's diagnostic performance to larger groups of dermatologists are lacking. Methods: Google's Inception v4 CNN architecture was trained and validated using dermoscopic images and corresponding diagnoses. In a comparative cross-sectional reader study a 100-image test-set was used (level-I: dermoscopy only; level-II: dermoscopy plus clinical information and images). Main outcome measures were sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for diagnostic classification (dichotomous) of lesions by the CNN versus an international group of 58 dermatologists during level-I or -II of the reader study. Secondary end points included the dermatologists' diagnostic performance in their management decisions and differences in the diagnostic performance of dermatologists during level-I and -II of the reader study. Additionally, the CNN's performance was compared with the top-five algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge. Results: In level-I dermatologists achieved a mean (±standard deviation) sensitivity and specificity for lesion classification of 86.6% (±9.3%) and 71.3% (±11.2%), respectively. More clinical information (level-II) improved the sensitivity to 88.9% (±9.6%, P = 0.19) and specificity to 75.7% (±11.7%, P < 0.05). The CNN ROC curve revealed a higher specificity of 82.5% when compared with dermatologists in level-I (71.3%, P < 0.01) and level-II (75.7%, P < 0.01) at their sensitivities of 86.6% and 88.9%, respectively. The CNN ROC AUC was greater than the mean ROC area of dermatologists (0.86 versus 0.79, P < 0.01). The CNN scored results close to the top three algorithms of the ISBI 2016 challenge. Conclusions: For the first time we compared a CNN's diagnostic performance with a large international group of 58 dermatologists, including 30 experts. Most dermatologists were outperformed by the CNN. Irrespective of any physicians' experience, they may benefit from assistance by a CNN's image classification. Clinical trial number: This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; https://www.drks.de/drks_web/).
Background: Deep learning convolutional neural networks (CNN) may facilitate melanoma detection, but data comparing a CNN's diagnostic performance to larger groups of dermatologists are lacking. Methods: Google's Inception v4 CNN architecture was trained and validated using dermoscopic images and corresponding diagnoses. In a comparative cross-sectional reader study a 100-image test-set was used (level-I: dermoscopy only; level-II: dermoscopy plus clinical information and images). Main outcome measures were sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for diagnostic classification (dichotomous) of lesions by the CNN versus an international group of 58 dermatologists during level-I or -II of the reader study. Secondary end points included the dermatologists' diagnostic performance in their management decisions and differences in the diagnostic performance of dermatologists during level-I and -II of the reader study. Additionally, the CNN's performance was compared with the top-five algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge. Results: In level-I dermatologists achieved a mean (±standard deviation) sensitivity and specificity for lesion classification of 86.6% (±9.3%) and 71.3% (±11.2%), respectively. More clinical information (level-II) improved the sensitivity to 88.9% (±9.6%, P = 0.19) and specificity to 75.7% (±11.7%, P < 0.05). The CNN ROC curve revealed a higher specificity of 82.5% when compared with dermatologists in level-I (71.3%, P < 0.01) and level-II (75.7%, P < 0.01) at their sensitivities of 86.6% and 88.9%, respectively. The CNN ROC AUC was greater than the mean ROC area of dermatologists (0.86 versus 0.79, P < 0.01). The CNN scored results close to the top three algorithms of the ISBI 2016 challenge. Conclusions: For the first time we compared a CNN's diagnostic performance with a large international group of 58 dermatologists, including 30 experts. Most dermatologists were outperformed by the CNN. Irrespective of any physicians' experience, they may benefit from assistance by a CNN's image classification. Clinical trial number: This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; https://www.drks.de/drks_web/).
Authors: Niels Kvorning Ternov; T Vestergaard; L Rosenkrantz Hölmich; K Karmisholt; A L Wagenblast; H Klyver; M Hald; L Schøllhammer; L Konge; A H Chakera Journal: Arch Dermatol Res Date: 2020-06-28 Impact factor: 3.017
Authors: Bettina Zippel-Schultz; Carsten Schultz; Dirk Müller-Wieland; Andrew B Remppis; Martin Stockburger; Christian Perings; Thomas M Helms Journal: Herzschrittmacherther Elektrophysiol Date: 2021-01-15
Authors: Andreas Kleppe; Ole-Johan Skrede; Sepp De Raedt; Knut Liestøl; David J Kerr; Håvard E Danielsen Journal: Nat Rev Cancer Date: 2021-01-29 Impact factor: 60.716
Authors: Mikko J Huttunen; Radu Hristu; Adrian Dumitru; Iustin Floroiu; Mariana Costache; Stefan G Stanciu Journal: Biomed Opt Express Date: 2019-12-10 Impact factor: 3.732
Authors: Philipp Tschandl; Cliff Rosendahl; Bengu Nisa Akay; Giuseppe Argenziano; Andreas Blum; Ralph P Braun; Horacio Cabo; Jean-Yves Gourhant; Jürgen Kreusch; Aimilios Lallas; Jan Lapins; Ashfaq Marghoob; Scott Menzies; Nina Maria Neuber; John Paoli; Harold S Rabinovitz; Christoph Rinner; Alon Scope; H Peter Soyer; Christoph Sinz; Luc Thomas; Iris Zalaudek; Harald Kittler Journal: JAMA Dermatol Date: 2019-01-01 Impact factor: 10.282