Christopher J Cadham1, Marie Knoll2, Luz María Sánchez-Romero2, K Michael Cummings3, Clifford E Douglas1,4, Alex Liber2, David Mendez1, Rafael Meza5, Ritesh Mistry6, Aylin Sertkaya7, Nargiz Travis1,2, David T Levy2. 1. Department of Health Management and Policy, University of Michigan, School of Public Health, Ann Arbor, MI, USA. 2. Georgetown University, Lombardi Comprehensive Cancer Center, Washington, DC, USA. 3. Department of Psychiatry & Behavioral Sciences, Medical University of South Carolina, Charleston, SC, USA. 4. University of Michigan, Tobacco Research Network, Ann Arbor, MI, USA. 5. Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, MI, USA. 6. Department of Health Behavior and Health Education, University of Michigan School of Public Health, Ann Arbor, MI, USA. 7. Eastern Research Group, Inc., Lexington, MA, USA.
Abstract
BACKGROUND: Expert elicitation (EE) has been used across disciplines to estimate input parameters for computational modeling research when information is sparse or conflictual. OBJECTIVES: We conducted a systematic review to compare EE methods used to generate model input parameters in health research. DATA SOURCES: PubMed and Web of Science. STUDY ELIGIBILITY: Modeling studies that reported the use of EE as the source for model input probabilities were included if they were published in English before June 2021 and reported health outcomes. DATA ABSTRACTION AND SYNTHESIS: Studies were classified as "formal" EE methods if they explicitly reported details of their elicitation process. Those that stated use of expert opinion but provided limited information were classified as "indeterminate" methods. In both groups, we abstracted citation details, study design, modeling methodology, a description of elicited parameters, and elicitation methods. Comparisons were made between elicitation methods. STUDY APPRAISAL: Studies that conducted a formal EE were appraised on the reporting quality of the EE. Quality appraisal was not conducted for studies of indeterminate methods. RESULTS: The search identified 1520 articles, of which 152 were included. Of the included studies, 40 were classified as formal EE and 112 as indeterminate methods. Most studies were cost-effectiveness analyses (77.6%). Forty-seven indeterminate method studies provided no information on methods for generating estimates. Among formal EEs, the average reporting quality score was 9 out of 16. LIMITATIONS: Elicitations on nonhealth topics and those reported in the gray literature were not included. CONCLUSIONS: We found poor reporting of EE methods used in modeling studies, making it difficult to discern meaningful differences in approaches. Improved quality standards for EEs would improve the validity and replicability of computational models. HIGHLIGHTS: We find extensive use of expert elicitation for the development of model input parameters, but most studies do not provide adequate details of their elicitation methods.Lack of reporting hinders greater discussion of the merits and challenges of using expert elicitation for model input parameter development.There is a need to establish expert elicitation best practices and reporting guidelines.
BACKGROUND: Expert elicitation (EE) has been used across disciplines to estimate input parameters for computational modeling research when information is sparse or conflictual. OBJECTIVES: We conducted a systematic review to compare EE methods used to generate model input parameters in health research. DATA SOURCES: PubMed and Web of Science. STUDY ELIGIBILITY: Modeling studies that reported the use of EE as the source for model input probabilities were included if they were published in English before June 2021 and reported health outcomes. DATA ABSTRACTION AND SYNTHESIS: Studies were classified as "formal" EE methods if they explicitly reported details of their elicitation process. Those that stated use of expert opinion but provided limited information were classified as "indeterminate" methods. In both groups, we abstracted citation details, study design, modeling methodology, a description of elicited parameters, and elicitation methods. Comparisons were made between elicitation methods. STUDY APPRAISAL: Studies that conducted a formal EE were appraised on the reporting quality of the EE. Quality appraisal was not conducted for studies of indeterminate methods. RESULTS: The search identified 1520 articles, of which 152 were included. Of the included studies, 40 were classified as formal EE and 112 as indeterminate methods. Most studies were cost-effectiveness analyses (77.6%). Forty-seven indeterminate method studies provided no information on methods for generating estimates. Among formal EEs, the average reporting quality score was 9 out of 16. LIMITATIONS: Elicitations on nonhealth topics and those reported in the gray literature were not included. CONCLUSIONS: We found poor reporting of EE methods used in modeling studies, making it difficult to discern meaningful differences in approaches. Improved quality standards for EEs would improve the validity and replicability of computational models. HIGHLIGHTS: We find extensive use of expert elicitation for the development of model input parameters, but most studies do not provide adequate details of their elicitation methods.Lack of reporting hinders greater discussion of the merits and challenges of using expert elicitation for model input parameter development.There is a need to establish expert elicitation best practices and reporting guidelines.
Authors: Steven B Duff; Elizabeth A Scott; Michael S Mafilios; Ewen C Todd; Leonard R Krilov; Alasdair M Geddes; Stacey J Ackerman Journal: J Food Prot Date: 2003-11 Impact factor: 2.077
Authors: Gabriel Rogers; Ruth Garside; Stuart Mealing; Martin Pitt; Rob Anderson; Matthew Dyer; Ken Stein; Margaret Somerville Journal: Pharmacoeconomics Date: 2008 Impact factor: 4.981
Authors: Yoel Lubell; Sarah G Staedke; Brian M Greenwood; Moses R Kamya; Malcolm Molyneux; Paul N Newton; Hugh Reyburn; Robert W Snow; Umberto D'Alessandro; Mike English; Nick Day; Peter Kremsner; Arjen Dondorp; Wilfred Mbacham; Grant Dorsey; Seth Owusu-Agyei; Kathryn Maitland; Sanjeev Krishna; Charles Newton; Geoffrey Pasvol; Terrie Taylor; Lorenz von Seidlein; Nicholas J White; Fred Binka; Anne Mills; Christopher J M Whitty Journal: PLoS One Date: 2011-02-24 Impact factor: 3.240
Authors: Alison Peel; Michelle Jenks; Moni Choudhury; Rosemary Lovett; Juan Carlos Rejon-Parrilla; Andrew Sims; Joyce Craig Journal: Appl Health Econ Health Policy Date: 2018-12 Impact factor: 2.561