A Udrea1,2, G D Mitra2, D Costea1,2, E C Noels3, M Wakkee3, D M Siegel4,5, T M de Carvalho3, T E C Nijsten3. 1. Department of Automatic Control and Systems Engineering, University Politehnica of Bucharest, Bucharest, Romania. 2. SkinVision BV, Amsterdam, The Netherlands. 3. Department of Dermatology, Erasmus Medical Center, Rotterdam, The Netherlands. 4. State University of New York Downstate Medical Center, Brooklyn, NY, USA. 5. Brooklyn Veterans Administration Medical Center, New York, NY, USA.
Abstract
BACKGROUND: Machine learning algorithms achieve expert-level accuracy in skin lesion classification based on clinical images. However, it is not yet shown whether these algorithms could have high accuracy when embedded in a smartphone app, where image quality is lower and there is high variability in image taking scenarios by users. In the past, these applications were criticized due to lack of accuracy. OBJECTIVE: In this study, we evaluate the accuracy of the newest version of a smartphone application (SA) for risk assessment of skin lesions. METHODS: This SA uses a machine learning algorithm to compute a risk rating. The algorithm is trained on 131 873 images taken by 31 449 users in multiple countries between January 2016 and August 2018 and rated for risk by dermatologists. To evaluate the sensitivity of the algorithm, we use 285 histopathologically validated skin cancer cases (including 138 malignant melanomas), from two previously published clinical studies (195 cases) and from the SA user database (90 cases). We calculate the specificity on a separate set from the SA user database containing 6000 clinically validated benign cases. RESULTS: The algorithm scored a 95.1% (95% CI, 91.9-97.3%) sensitivity in detecting (pre)malignant conditions (93% for malignant melanoma and 97% for keratinocyte carcinomas and precursors). This level of sensitivity was achieved with a 78.3% (95% CI, 77.2-79.3%) specificity. CONCLUSIONS: This SA provides a high sensitivity to detect skin cancer; however, there is still room for improvement in terms of specificity. Future studies are needed to assess the impact of this SA on the health systems and its users.
BACKGROUND: Machine learning algorithms achieve expert-level accuracy in skin lesion classification based on clinical images. However, it is not yet shown whether these algorithms could have high accuracy when embedded in a smartphone app, where image quality is lower and there is high variability in image taking scenarios by users. In the past, these applications were criticized due to lack of accuracy. OBJECTIVE: In this study, we evaluate the accuracy of the newest version of a smartphone application (SA) for risk assessment of skin lesions. METHODS: This SA uses a machine learning algorithm to compute a risk rating. The algorithm is trained on 131 873 images taken by 31 449 users in multiple countries between January 2016 and August 2018 and rated for risk by dermatologists. To evaluate the sensitivity of the algorithm, we use 285 histopathologically validated skin cancer cases (including 138 malignant melanomas), from two previously published clinical studies (195 cases) and from the SA user database (90 cases). We calculate the specificity on a separate set from the SA user database containing 6000 clinically validated benign cases. RESULTS: The algorithm scored a 95.1% (95% CI, 91.9-97.3%) sensitivity in detecting (pre)malignant conditions (93% for malignant melanoma and 97% for keratinocyte carcinomas and precursors). This level of sensitivity was achieved with a 78.3% (95% CI, 77.2-79.3%) specificity. CONCLUSIONS: This SA provides a high sensitivity to detect skin cancer; however, there is still room for improvement in terms of specificity. Future studies are needed to assess the impact of this SA on the health systems and its users.
Authors: Gaby N Moawad; Jad Elkhalil; Jordan S Klebanoff; Sara Rahman; Nassir Habib; Ibrahim Alkatout Journal: J Clin Med Date: 2020-11-25 Impact factor: 4.241
Authors: Marise J Kasteleyn; Anke Versluis; Petra van Peet; Ulrik Bak Kirk; Jens van Dalfsen; Eline Meijer; Persijn Honkoop; Kendall Ho; Niels H Chavannes; Esther P W A Talboom-Kamp Journal: Eur J Gen Pract Date: 2021-12 Impact factor: 1.904
Authors: Hao Wen; Wenjian Yu; Yuanqing Wu; Jun Zhao; Xiaolong Liu; Zhexiang Kuang; Rong Fan Journal: Technol Health Care Date: 2022 Impact factor: 1.205
Authors: Karoline Freeman; Jacqueline Dinnes; Naomi Chuchu; Yemisi Takwoingi; Sue E Bayliss; Rubeta N Matin; Abhilash Jain; Fiona M Walter; Hywel C Williams; Jonathan J Deeks Journal: BMJ Date: 2020-02-10
Authors: Andre G C Pacheco; Gustavo R Lima; Amanda S Salomão; Breno Krohling; Igor P Biral; Gabriel G de Angelo; Fábio C R Alves; José G M Esgario; Alana C Simora; Pedro B C Castro; Felipe B Rodrigues; Patricia H L Frasson; Renato A Krohling; Helder Knidel; Maria C S Santos; Rachel B do Espírito Santo; Telma L S G Macedo; Tania R P Canuto; Luíz F S de Barros Journal: Data Brief Date: 2020-08-25