Ralf H J M Kurvers1, Jens Krause2, Giuseppe Argenziano3, Iris Zalaudek4, Max Wolf5. 1. Department of Biology and Ecology of Fishes, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany2Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany. 2. Department of Biology and Ecology of Fishes, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany3Faculty of Life Sciences, Humboldt-University of Berlin, Berlin, Germany. 3. Department of Dermatology, Second University of Naples, Naples, Italy. 4. Department of Dermatology and Venereology, Medical University of Graz, Graz, Austria. 5. Department of Biology and Ecology of Fishes, Leibniz-Institute of Freshwater Ecology and Inland Fisheries, Berlin, Germany.
Abstract
IMPORTANCE: Incidence rates of skin cancer are increasing globally, and the correct classification of skin lesions (SLs) into benign and malignant tissue remains a continuous challenge. A collective intelligence approach to skin cancer detection may improve accuracy. OBJECTIVE: To evaluate the performance of 2 well-known collective intelligence rules (majority rule and quorum rule) that combine the independent conclusions of multiple decision makers into a single decision. DESIGN, SETTING, AND PARTICIPANTS: Evaluations were obtained from 2 large and independent data sets. The first data set consisted of 40 experienced dermoscopists, each of whom independently evaluated 108 images of SLs during the Consensus Net Meeting of 2000. The second data set consisted of 82 medical professionals with varying degrees of dermatology experience, each of whom evaluated a minimum of 110 SLs. All SLs were evaluated via the Internet. Image selection of SLs was based on high image quality and the presence of histopathologic information. Data were collected from July through October 2000 for study 1 and from February 2003 through January 2004 for study 2 and evaluated from January 5 through August 7, 2015. MAIN OUTCOMES AND MEASURES: For both collective intelligence rules, we determined the true-positive rate (ie, the hit rate or specificity) and the false-positive rate (ie, the false-alarm rate or 1 - sensitivity) and compared these rates with the performance of single decision makers. Furthermore, we evaluated the effect of group size on true- and false-positive rates. RESULTS: One hundred twenty-two medical professionals performed 16 029 evaluations. Use of either collective intelligence rule consistently outperformed single decision makers. The groups achieved an increased true-positive rate and a decreased false-positive rate. For example, individual decision makers in study 1, using the pattern analysis as diagnostic algorithm, achieved a true-positive rate of 0.83 and a false-positive rate of 0.17. Groups of 3 individuals achieved a true-positive rate of 0.91 and a false-positive rate of 0.14. These improvements increased with increasing group size. CONCLUSIONS AND RELEVANCE: Collective intelligence might be a viable approach to increase diagnostic accuracy in skin cancer and reduce skin cancer-related mortality.
IMPORTANCE: Incidence rates of skin cancer are increasing globally, and the correct classification of skin lesions (SLs) into benign and malignant tissue remains a continuous challenge. A collective intelligence approach to skin cancer detection may improve accuracy. OBJECTIVE: To evaluate the performance of 2 well-known collective intelligence rules (majority rule and quorum rule) that combine the independent conclusions of multiple decision makers into a single decision. DESIGN, SETTING, AND PARTICIPANTS: Evaluations were obtained from 2 large and independent data sets. The first data set consisted of 40 experienced dermoscopists, each of whom independently evaluated 108 images of SLs during the Consensus Net Meeting of 2000. The second data set consisted of 82 medical professionals with varying degrees of dermatology experience, each of whom evaluated a minimum of 110 SLs. All SLs were evaluated via the Internet. Image selection of SLs was based on high image quality and the presence of histopathologic information. Data were collected from July through October 2000 for study 1 and from February 2003 through January 2004 for study 2 and evaluated from January 5 through August 7, 2015. MAIN OUTCOMES AND MEASURES: For both collective intelligence rules, we determined the true-positive rate (ie, the hit rate or specificity) and the false-positive rate (ie, the false-alarm rate or 1 - sensitivity) and compared these rates with the performance of single decision makers. Furthermore, we evaluated the effect of group size on true- and false-positive rates. RESULTS: One hundred twenty-two medical professionals performed 16 029 evaluations. Use of either collective intelligence rule consistently outperformed single decision makers. The groups achieved an increased true-positive rate and a decreased false-positive rate. For example, individual decision makers in study 1, using the pattern analysis as diagnostic algorithm, achieved a true-positive rate of 0.83 and a false-positive rate of 0.17. Groups of 3 individuals achieved a true-positive rate of 0.91 and a false-positive rate of 0.14. These improvements increased with increasing group size. CONCLUSIONS AND RELEVANCE: Collective intelligence might be a viable approach to increase diagnostic accuracy in skin cancer and reduce skin cancer-related mortality.
Authors: Ralf H J M Kurvers; Stefan M Herzog; Ralph Hertwig; Jens Krause; Patricia A Carney; Andy Bogart; Giuseppe Argenziano; Iris Zalaudek; Max Wolf Journal: Proc Natl Acad Sci U S A Date: 2016-07-18 Impact factor: 11.205
Authors: Pavel Atanasov; Andreas Diamantaras; Amanda MacPherson; Esther Vinarov; Daniel M Benjamin; Ian Shrier; Friedemann Paul; Ulrich Dirnagl; Jonathan Kimmelman Journal: Neurology Date: 2020-06-16 Impact factor: 9.910
Authors: Elaine C Khoong; Sarah S Nouri; Delphine S Tuot; Shantanu Nundy; Valy Fontil; Urmimala Sarkar Journal: Med Decis Making Date: 2021-08-11 Impact factor: 2.583
Authors: Cristina Carrera; Michael A Marchetti; Stephen W Dusza; Giuseppe Argenziano; Ralph P Braun; Allan C Halpern; Natalia Jaimes; Harald J Kittler; Josep Malvehy; Scott W Menzies; Giovanni Pellacani; Susana Puig; Harold S Rabinovitz; Alon Scope; H Peter Soyer; Wilhelm Stolz; Rainer Hofmann-Wellenhof; Iris Zalaudek; Ashfaq A Marghoob Journal: JAMA Dermatol Date: 2016-07-01 Impact factor: 10.282
Authors: Jacqueline Dinnes; Jonathan J Deeks; Naomi Chuchu; Rubeta N Matin; Kai Yuen Wong; Roger Benjamin Aldridge; Alana Durack; Abha Gulati; Sue Ann Chan; Louise Johnston; Susan E Bayliss; Jo Leonardi-Bee; Yemisi Takwoingi; Clare Davenport; Colette O'Sullivan; Hamid Tehrani; Hywel C Williams Journal: Cochrane Database Syst Rev Date: 2018-12-04
Authors: Jacqueline Dinnes; Jonathan J Deeks; Matthew J Grainge; Naomi Chuchu; Lavinia Ferrante di Ruffano; Rubeta N Matin; David R Thomson; Kai Yuen Wong; Roger Benjamin Aldridge; Rachel Abbott; Monica Fawzy; Susan E Bayliss; Yemisi Takwoingi; Clare Davenport; Kathie Godfrey; Fiona M Walter; Hywel C Williams Journal: Cochrane Database Syst Rev Date: 2018-12-04