Steven MacLennan1, Jamie Kirkham2, Thomas B L Lam3, Paula R Williamson2. 1. Academic Urology Unit, University of Aberdeen, Aberdeen, UK. Electronic address: steven.maclennan@abdn.ac.uk. 2. Department of Biostatistics, University of Liverpool, Liverpool, UK. 3. Academic Urology Unit, University of Aberdeen, Aberdeen, UK; Department of Urology, Aberdeen Royal Infirmary, Aberdeen, UK.
Abstract
OBJECTIVES: The objective of the study was to explore the impact of different feedback strategies on (1) subsequent agreement and (2) variability in Delphi studies. STUDY DESIGN AND SETTING: A two-round Delphi survey, with a list of outcomes generated from the results of a systematic review and interviews, was undertaken while developing a core outcomes set for prostate cancer including two stakeholder groups (health professionals and patients). Seventy-nine outcomes were scored on a scale of one (not important) to nine (critically important). Participants were randomized in round 2 to receive round 1 feedback from peers only, multiple stakeholders separately, or multiple stakeholders combined. RESULTS: Agreement on outcomes retained for all feedback groups was high (peer: 92%, multiple separate: 90%, multiple combined: 84%). There were no statistically significant reduction in variability for peer vs. multiple separate (0.016 [-0.035, 0.067]; P = 0.529), or multiple separate vs. multiple combined feedback (0.063 [-0.003, 0.129]; P = 0.062). Peer feedback statistically significantly reduced variability compared with multiple combined feedback (0.079 [0.001, 0.157]; P = 0.046). CONCLUSIONS: We found no evidence of a difference between different feedback strategies in terms of the number of outcomes retained or reduction in variability of opinion. However, this may be explained by the high level of existing agreement in round 1. Further methodological studies nested within Delphi surveys will help clarify the best strategy.
RCT Entities:
OBJECTIVES: The objective of the study was to explore the impact of different feedback strategies on (1) subsequent agreement and (2) variability in Delphi studies. STUDY DESIGN AND SETTING: A two-round Delphi survey, with a list of outcomes generated from the results of a systematic review and interviews, was undertaken while developing a core outcomes set for prostate cancer including two stakeholder groups (health professionals and patients). Seventy-nine outcomes were scored on a scale of one (not important) to nine (critically important). Participants were randomized in round 2 to receive round 1 feedback from peers only, multiple stakeholders separately, or multiple stakeholders combined. RESULTS: Agreement on outcomes retained for all feedback groups was high (peer: 92%, multiple separate: 90%, multiple combined: 84%). There were no statistically significant reduction in variability for peer vs. multiple separate (0.016 [-0.035, 0.067]; P = 0.529), or multiple separate vs. multiple combined feedback (0.063 [-0.003, 0.129]; P = 0.062). Peer feedback statistically significantly reduced variability compared with multiple combined feedback (0.079 [0.001, 0.157]; P = 0.046). CONCLUSIONS: We found no evidence of a difference between different feedback strategies in terms of the number of outcomes retained or reduction in variability of opinion. However, this may be explained by the high level of existing agreement in round 1. Further methodological studies nested within Delphi surveys will help clarify the best strategy.
Authors: Howard Ryland; Jonathan Cook; Rob Ferris; Sarah Markham; Christian Sales; Raymond Fitzpatrick; Seena Fazel Journal: Psychol Crime Law Date: 2021-08-23
Authors: Jan W van der Scheer; Matthew Woodward; Akbar Ansari; Tim Draycott; Cathy Winter; Graham Martin; Karolina Kuberska; Natalie Richards; Ruth Kern; Mary Dixon-Woods Journal: BMC Med Res Methodol Date: 2021-05-11 Impact factor: 4.615
Authors: Ben A Marson; Joseph C Manning; Marilyn James; Simon Craxford; Sandeep R Deshmukh; Daniel C Perry; Benjamin J Ollivere Journal: Bone Joint J Date: 2021-08-19 Impact factor: 5.082
Authors: Toni Lange; Christian Kopkow; Jörg Lützner; Klaus-Peter Günther; Sascha Gravius; Hanns-Peter Scharf; Johannes Stöve; Richard Wagner; Jochen Schmitt Journal: BMC Med Res Methodol Date: 2020-02-10 Impact factor: 4.615
Authors: Ben Arthur Marson; Joseph C Manning; Marilyn James; Simon Craxford; Sandeep R Deshmukh; Benjamin J Ollivere Journal: BMJ Open Date: 2020-02-28 Impact factor: 2.692