OBJECTIVE: Crowdsourcing research allows investigators to engage thousands of people to provide either data or data analysis. However, prior work has not documented the use of crowdsourcing in health and medical research. We sought to systematically review the literature to describe the scope of crowdsourcing in health research and to create a taxonomy to characterize past uses of this methodology for health and medical research. DATA SOURCES: PubMed, Embase, and CINAHL through March 2013. STUDY ELIGIBILITY CRITERIA: Primary peer-reviewed literature that used crowdsourcing for health research. STUDY APPRAISAL AND SYNTHESIS METHODS: Two authors independently screened studies and abstracted data, including demographics of the crowd engaged and approaches to crowdsourcing. RESULTS: Twenty-one health-related studies utilizing crowdsourcing met eligibility criteria. Four distinct types of crowdsourcing tasks were identified: problem solving, data processing, surveillance/monitoring, and surveying. These studies collectively engaged a crowd of >136,395 people, yet few studies reported demographics of the crowd. Only one (5 %) reported age, sex, and race statistics, and seven (33 %) reported at least one of these descriptors. Most reports included data on crowdsourcing logistics such as the length of crowdsourcing (n = 18, 86 %) and time to complete crowdsourcing task (n = 15, 71 %). All articles (n = 21, 100 %) reported employing some method for validating or improving the quality of data reported from the crowd. LIMITATIONS: Gray literature not searched and only a sample of online survey articles included. CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS: Utilizing crowdsourcing can improve the quality, cost, and speed of a research project while engaging large segments of the public and creating novel science. Standardized guidelines are needed on crowdsourcing metrics that should be collected and reported to provide clarity and comparability in methods.
OBJECTIVE: Crowdsourcing research allows investigators to engage thousands of people to provide either data or data analysis. However, prior work has not documented the use of crowdsourcing in health and medical research. We sought to systematically review the literature to describe the scope of crowdsourcing in health research and to create a taxonomy to characterize past uses of this methodology for health and medical research. DATA SOURCES: PubMed, Embase, and CINAHL through March 2013. STUDY ELIGIBILITY CRITERIA: Primary peer-reviewed literature that used crowdsourcing for health research. STUDY APPRAISAL AND SYNTHESIS METHODS: Two authors independently screened studies and abstracted data, including demographics of the crowd engaged and approaches to crowdsourcing. RESULTS: Twenty-one health-related studies utilizing crowdsourcing met eligibility criteria. Four distinct types of crowdsourcing tasks were identified: problem solving, data processing, surveillance/monitoring, and surveying. These studies collectively engaged a crowd of >136,395 people, yet few studies reported demographics of the crowd. Only one (5 %) reported age, sex, and race statistics, and seven (33 %) reported at least one of these descriptors. Most reports included data on crowdsourcing logistics such as the length of crowdsourcing (n = 18, 86 %) and time to complete crowdsourcing task (n = 15, 71 %). All articles (n = 21, 100 %) reported employing some method for validating or improving the quality of data reported from the crowd. LIMITATIONS: Gray literature not searched and only a sample of online survey articles included. CONCLUSIONS AND IMPLICATIONS OF KEY FINDINGS: Utilizing crowdsourcing can improve the quality, cost, and speed of a research project while engaging large segments of the public and creating novel science. Standardized guidelines are needed on crowdsourcing metrics that should be collected and reported to provide clarity and comparability in methods.
Authors: Clark C Freifeld; Rumi Chunara; Sumiko R Mekaru; Emily H Chan; Taha Kass-Hout; Anahi Ayala Iacucci; John S Brownstein Journal: PLoS Med Date: 2010-12-07 Impact factor: 11.069
Authors: Lena Maier-Hein; Daniel Kondermann; Tobias Roß; Sven Mersmann; Eric Heim; Sebastian Bodenstedt; Hannes Götz Kenngott; Alexandro Sanchez; Martin Wagner; Anas Preukschas; Anna-Laura Wekerle; Stefanie Helfert; Keno März; Arianeb Mehrabi; Stefanie Speidel; Christian Stock Journal: Int J Comput Assist Radiol Surg Date: 2015-04-18 Impact factor: 2.924
Authors: John A Naslund; Kelly A Aschbrenner; Lisa A Marsch; Gregory J McHugo; Stephen J Bartels Journal: Contemp Clin Trials Date: 2015-07-16 Impact factor: 2.226
Authors: Eric Heim; Tobias Roß; Alexander Seitel; Keno März; Bram Stieltjes; Matthias Eisenmann; Johannes Lebert; Jasmin Metzger; Gregor Sommer; Alexander W Sauter; Fides Regina Schwartz; Andreas Termer; Felix Wagner; Hannes Götz Kenngott; Lena Maier-Hein Journal: J Med Imaging (Bellingham) Date: 2018-09-08