Bogdan Grigore1, Jaime Peters, Christopher Hyde, Ken Stein. 1. Peninsula Technology Assessment Group (PenTAG), Institute of Health Research, University of Exeter Medical School, University of Exeter, Exeter, UK, bogdan.grigore@pcmd.ac.uk.
Abstract
BACKGROUND: Elicitation is a technique that can be used to obtain probability distribution from experts about unknown quantities. We conducted a methodology review of reports where probability distributions had been elicited from experts to be used in model-based health technology assessments. METHODS: Databases including MEDLINE, EMBASE and the CRD database were searched from inception to April 2013. Reference lists were checked and citation mapping was also used. Studies describing their approach to the elicitation of probability distributions were included. Data was abstracted on pre-defined aspects of the elicitation technique. Reports were critically appraised on their consideration of the validity, reliability and feasibility of the elicitation exercise. RESULTS: Fourteen articles were included. Across these studies, the most marked features were heterogeneity in elicitation approach and failure to report key aspects of the elicitation method. The most frequently used approaches to elicitation were the histogram technique and the bisection method. Only three papers explicitly considered the validity, reliability and feasibility of the elicitation exercises. CONCLUSION: Judged by the studies identified in the review, reports of expert elicitation are insufficient in detail and this impacts on the perceived usability of expert-elicited probability distributions. In this context, the wider credibility of elicitation will only be improved by better reporting and greater standardisation of approach. Until then, the advantage of eliciting probability distributions from experts may be lost.
BACKGROUND: Elicitation is a technique that can be used to obtain probability distribution from experts about unknown quantities. We conducted a methodology review of reports where probability distributions had been elicited from experts to be used in model-based health technology assessments. METHODS: Databases including MEDLINE, EMBASE and the CRD database were searched from inception to April 2013. Reference lists were checked and citation mapping was also used. Studies describing their approach to the elicitation of probability distributions were included. Data was abstracted on pre-defined aspects of the elicitation technique. Reports were critically appraised on their consideration of the validity, reliability and feasibility of the elicitation exercise. RESULTS: Fourteen articles were included. Across these studies, the most marked features were heterogeneity in elicitation approach and failure to report key aspects of the elicitation method. The most frequently used approaches to elicitation were the histogram technique and the bisection method. Only three papers explicitly considered the validity, reliability and feasibility of the elicitation exercises. CONCLUSION: Judged by the studies identified in the review, reports of expert elicitation are insufficient in detail and this impacts on the perceived usability of expert-elicited probability distributions. In this context, the wider credibility of elicitation will only be improved by better reporting and greater standardisation of approach. Until then, the advantage of eliciting probability distributions from experts may be lost.
Authors: Z Philips; L Ginnelly; M Sculpher; K Claxton; S Golder; R Riemsma; N Woolacoot; J Glanville Journal: Health Technol Assess Date: 2004-09 Impact factor: 4.014
Authors: Karl Claxton; Mark Sculpher; Chris McCabe; Andrew Briggs; Ron Akehurst; Martin Buxton; John Brazier; Tony O'Hagan Journal: Health Econ Date: 2005-04 Impact factor: 3.046
Authors: P M Speight; S Palmer; D R Moles; M C Downer; D H Smith; M Henriksson; F Augustovski Journal: Health Technol Assess Date: 2006-04 Impact factor: 4.014
Authors: Alan J Girling; Guy Freeman; Jason P Gordon; Philip Poole-Wilson; David A Scott; Richard J Lilford Journal: Int J Technol Assess Health Care Date: 2007 Impact factor: 2.188
Authors: T Colbourn; C Asseburg; L Bojke; Z Philips; K Claxton; A E Ades; R E Gilbert Journal: Health Technol Assess Date: 2007-08 Impact factor: 4.014
Authors: Matt D Stevenson; Jeremy E Oakley; Myfawny Lloyd Jones; Alan Brennan; Juliet E Compston; Eugene V McCloskey; Peter L Selby Journal: Med Decis Making Date: 2009-06-09 Impact factor: 2.583
Authors: C Meads; P Auguste; C Davenport; S Małysiak; S Sundar; M Kowalska; A Zapalska; P Guest; S Thangaratinam; P Martin-Hirsch; E Borowiack; P Barton; T Roberts; K Khan Journal: Health Technol Assess Date: 2013-03 Impact factor: 4.014
Authors: Laura Bojke; Bogdan Grigore; Dina Jankovic; Jaime Peters; Marta Soares; Ken Stein Journal: Pharmacoeconomics Date: 2017-09 Impact factor: 4.981
Authors: Christopher J Cadham; Marie Knoll; Luz María Sánchez-Romero; K Michael Cummings; Clifford E Douglas; Alex Liber; David Mendez; Rafael Meza; Ritesh Mistry; Aylin Sertkaya; Nargiz Travis; David T Levy Journal: Med Decis Making Date: 2021-10-25 Impact factor: 2.749
Authors: Laura Bojke; Marta Soares; Karl Claxton; Abigail Colson; Aimée Fox; Christopher Jackson; Dina Jankovic; Alec Morton; Linda Sharples; Andrea Taylor Journal: Health Technol Assess Date: 2021-06 Impact factor: 4.014
Authors: Antoine Poncet; Baris Gencer; Marc Blondon; Marianne Gex-Fabry; Christophe Combescure; Dipen Shah; Peter J Schwartz; Marie Besson; François R Girardin Journal: PLoS One Date: 2015-06-12 Impact factor: 3.240
Authors: Laura Bojke; Marta O Soares; Karl Claxton; Abigail Colson; Aimée Fox; Chris Jackson; Dina Jankovic; Alec Morton; Linda D Sharples; Andrea Taylor Journal: Med Decis Making Date: 2021-07-16 Impact factor: 2.583