Ivan R Diamond1, Robert C Grant2, Brian M Feldman3, Paul B Pencharz4, Simon C Ling5, Aideen M Moore6, Paul W Wales7. 1. Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Medical Imaging, The University of Toronto, 263 McCaul Street, 4th Floor, Toronto, ON M5T 1W7, Canada; Institute of Health Policy Management and Evaluation, The University of Toronto, 155 College Street, 4th floor, Toronto, ON M5T 3M7, Canada. 2. Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada. 3. Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Institute of Health Policy Management and Evaluation, The University of Toronto, 155 College Street, 4th floor, Toronto, ON M5T 3M7, Canada; Department of Paediatrics, The Hospital for Sick Children and The University of Toronto, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Dalla Lana School of Public Health, The University of Toronto, 155 College Street, 6th floor, Toronto, ON M5T 3M7, Canada. 4. Department of Paediatrics, The Hospital for Sick Children and The University of Toronto, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Department of Nutritional Sciences, The University of Toronto, 150 College Street, Room 316, Toronto, ON M5S 3E2, Canada. 5. Department of Paediatrics, The Hospital for Sick Children and The University of Toronto, 555 University Avenue, Toronto, ON M5G 1X8, Canada. 6. Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Institute of Health Policy Management and Evaluation, The University of Toronto, 155 College Street, 4th floor, Toronto, ON M5T 3M7, Canada; Department of Paediatrics, The Hospital for Sick Children and The University of Toronto, 555 University Avenue, Toronto, ON M5G 1X8, Canada. 7. Child Health Evaluative Sciences Program, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, Canada; Institute of Health Policy Management and Evaluation, The University of Toronto, 155 College Street, 4th floor, Toronto, ON M5T 3M7, Canada; Department of Surgery, The University of Toronto, 149 College Street, 5th Floor, Toronto, ON M5T 1P5, Canada. Electronic address: paul.wales@sickkids.ca.
Abstract
OBJECTIVE: To investigate how consensus is operationalized in Delphi studies and to explore the role of consensus in determining the results of these studies. STUDY DESIGN AND SETTINGS: Systematic review of a random sample of 100 English language Delphi studies, from two large multidisciplinary databases [ISI Web of Science (Thompson Reuters, New York, NY) and Scopus (Elsevier, Amsterdam, NL)], published between 2000 and 2009. RESULTS: About 98 of the Delphi studies purported to assess consensus, although a definition for consensus was only provided in 72 of the studies (64 a priori). The most common definition for consensus was percent agreement (25 studies), with 75% being the median threshold to define consensus. Although the authors concluded in 86 of the studies that consensus was achieved, consensus was only specified a priori (with a threshold value) in 42 of these studies. Achievement of consensus was related to the decision to stop the Delphi study in only 23 studies, with 70 studies terminating after a specified number of rounds. CONCLUSION: Although consensus generally is felt to be of primary importance to the Delphi process, definitions of consensus vary widely and are poorly reported. Improved criteria for reporting of methods of Delphi studies are required.
OBJECTIVE: To investigate how consensus is operationalized in Delphi studies and to explore the role of consensus in determining the results of these studies. STUDY DESIGN AND SETTINGS: Systematic review of a random sample of 100 English language Delphi studies, from two large multidisciplinary databases [ISI Web of Science (Thompson Reuters, New York, NY) and Scopus (Elsevier, Amsterdam, NL)], published between 2000 and 2009. RESULTS: About 98 of the Delphi studies purported to assess consensus, although a definition for consensus was only provided in 72 of the studies (64 a priori). The most common definition for consensus was percent agreement (25 studies), with 75% being the median threshold to define consensus. Although the authors concluded in 86 of the studies that consensus was achieved, consensus was only specified a priori (with a threshold value) in 42 of these studies. Achievement of consensus was related to the decision to stop the Delphi study in only 23 studies, with 70 studies terminating after a specified number of rounds. CONCLUSION: Although consensus generally is felt to be of primary importance to the Delphi process, definitions of consensus vary widely and are poorly reported. Improved criteria for reporting of methods of Delphi studies are required.
Authors: Christopher J Dy; Alison L Antes; Daniel A Osei; Charles A Goldfarb; James M DuBois Journal: J Bone Joint Surg Am Date: 2018-12-05 Impact factor: 5.284
Authors: Paula R Williamson; Douglas G Altman; Heather Bagley; Karen L Barnes; Jane M Blazeby; Sara T Brookes; Mike Clarke; Elizabeth Gargon; Sarah Gorst; Nicola Harman; Jamie J Kirkham; Angus McNair; Cecilia A C Prinsen; Jochen Schmitt; Caroline B Terwee; Bridget Young Journal: Trials Date: 2017-06-20 Impact factor: 2.279
Authors: Rebecca L Sudore; Daren K Heyland; Hillary D Lum; Judith A C Rietjens; Ida J Korfage; Christine S Ritchie; Laura C Hanson; Diane E Meier; Steven Z Pantilat; Karl Lorenz; Michelle Howard; Michael J Green; Jessica E Simon; Mariko A Feuz; John J You Journal: J Pain Symptom Manage Date: 2017-09-01 Impact factor: 3.612
Authors: Erik E Folch; Amit K Mahajan; Catherine L Oberg; Fabien Maldonado; Eric Toloza; William S Krimsky; Scott Oh; Mark R Bowling; Sadia Benzaquen; Charles M Kinsey; Atul C Mehta; Sebastian Fernandez-Bussy; Javier Flandes; Kelvin Lau; Ganesh Krishna; Michael A Nead; Felix Herth; Alejandro A Aragaki-Nakahodo; Emanuela Barisione; Sandeep Bansal; Dragos Zanchi; Michael Zgoda; Peter O Lutz; Robert J Lentz; Christopher Parks; Mario Salio; Kenneth Perret; Colleen Keyes; Gregory P LeMense; John D Hinze; Adnan Majid; Merete Christensen; Jordan Kazakov; Gonzalo Labarca; Ernest Waller; Michael Studnicka; Catalina V Teba; Sandeep J Khandhar Journal: Chest Date: 2020-02-14 Impact factor: 9.410