Stuart J Wright1, Caroline M Vass1, Gene Sim1, Michael Burton2, Denzil G Fiebig3, Katherine Payne4. 1. Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, The University of Manchester, Jean McFarlane Building, Oxford Road, Manchester, M13 9PL, UK. 2. UWA School of Agriculture and Environment, University of Western Australia, Perth, WA, Australia. 3. School of Economics, University of New South Wales Business School Building, University of New South Wales, Sydney, NSW, Australia. 4. Manchester Centre for Health Economics, Division of Population Health, Health Services Research and Primary Care, The University of Manchester, Jean McFarlane Building, Oxford Road, Manchester, M13 9PL, UK. katherine.payne@manchester.ac.uk.
Abstract
BACKGROUND: Scale heterogeneity, or differences in the error variance of choices, may account for a significant amount of the observed variation in the results of discrete choice experiments (DCEs) when comparing preferences between different groups of respondents. OBJECTIVE: The aim of this study was to identify if, and how, scale heterogeneity has been addressed in healthcare DCEs that compare the preferences of different groups. METHODS: A systematic review identified all healthcare DCEs published between 1990 and February 2016. The full-text of each DCE was then screened to identify studies that compared preferences using data generated from multiple groups. Data were extracted and tabulated on year of publication, samples compared, tests for scale heterogeneity, and analytical methods to account for scale heterogeneity. Narrative analysis was used to describe if, and how, scale heterogeneity was accounted for when preferences were compared. RESULTS: A total of 626 healthcare DCEs were identified. Of these 199 (32%) aimed to compare the preferences of different groups specified at the design stage, while 79 (13%) compared the preferences of groups identified at the analysis stage. Of the 278 included papers, 49 (18%) discussed potential scale issues, 18 (7%) used a formal method of analysis to account for scale between groups, and 2 (1%) accounted for scale differences between preference groups at the analysis stage. Scale heterogeneity was present in 65% (n = 13) of studies that tested for it. Analytical methods to test for scale heterogeneity included coefficient plots (n = 5, 2%), heteroscedastic conditional logit models (n = 6, 2%), Swait and Louviere tests (n = 4, 1%), generalised multinomial logit models (n = 5, 2%), and scale-adjusted latent class analysis (n = 2, 1%). CONCLUSIONS: Scale heterogeneity is a prevalent issue in healthcare DCEs. Despite this, few published DCEs have discussed such issues, and fewer still have used formal methods to identify and account for the impact of scale heterogeneity. The use of formal methods to test for scale heterogeneity should be used, otherwise the results of DCEs potentially risk producing biased and potentially misleading conclusions regarding preferences for aspects of healthcare.
BACKGROUND: Scale heterogeneity, or differences in the error variance of choices, may account for a significant amount of the observed variation in the results of discrete choice experiments (DCEs) when comparing preferences between different groups of respondents. OBJECTIVE: The aim of this study was to identify if, and how, scale heterogeneity has been addressed in healthcare DCEs that compare the preferences of different groups. METHODS: A systematic review identified all healthcare DCEs published between 1990 and February 2016. The full-text of each DCE was then screened to identify studies that compared preferences using data generated from multiple groups. Data were extracted and tabulated on year of publication, samples compared, tests for scale heterogeneity, and analytical methods to account for scale heterogeneity. Narrative analysis was used to describe if, and how, scale heterogeneity was accounted for when preferences were compared. RESULTS: A total of 626 healthcare DCEs were identified. Of these 199 (32%) aimed to compare the preferences of different groups specified at the design stage, while 79 (13%) compared the preferences of groups identified at the analysis stage. Of the 278 included papers, 49 (18%) discussed potential scale issues, 18 (7%) used a formal method of analysis to account for scale between groups, and 2 (1%) accounted for scale differences between preference groups at the analysis stage. Scale heterogeneity was present in 65% (n = 13) of studies that tested for it. Analytical methods to test for scale heterogeneity included coefficient plots (n = 5, 2%), heteroscedastic conditional logit models (n = 6, 2%), Swait and Louviere tests (n = 4, 1%), generalised multinomial logit models (n = 5, 2%), and scale-adjusted latent class analysis (n = 2, 1%). CONCLUSIONS: Scale heterogeneity is a prevalent issue in healthcare DCEs. Despite this, few published DCEs have discussed such issues, and fewer still have used formal methods to identify and account for the impact of scale heterogeneity. The use of formal methods to test for scale heterogeneity should be used, otherwise the results of DCEs potentially risk producing biased and potentially misleading conclusions regarding preferences for aspects of healthcare.
Authors: D Blaauw; E Erasmus; N Pagaiya; V Tangcharoensathein; K Mullei; S Mudhune; C Goodman; M English; M Lagarde Journal: Bull World Health Organ Date: 2010-05 Impact factor: 9.408
Authors: Andrew Mirelman; Emmanouil Mentzakis; Elizabeth Kinter; Francesco Paolucci; Richard Fordham; Sachiko Ozawa; Marcos Ferraz; Rob Baltussen; Louis W Niessen Journal: Value Health Date: 2012-05 Impact factor: 5.725
Authors: Nancy L Schoenborn; Norah L Crossnohere; Ellen M Janssen; Craig E Pollack; Cynthia M Boyd; Antonio C Wolff; Qian-Li Xue; Jacqueline Massare; Marcela Blinka; John F P Bridges Journal: J Gen Intern Med Date: 2019-08-26 Impact factor: 5.128
Authors: Suzana Karim; Benjamin M Craig; Caroline Vass; Catharina G M Groothuis-Oudshoorn Journal: Pharmacoeconomics Date: 2022-08-12 Impact factor: 4.558
Authors: David J Mott; Koonal K Shah; Juan Manuel Ramos-Goñi; Nancy J Devlin; Oliver Rivero-Arias Journal: Med Decis Making Date: 2021-03-18 Impact factor: 2.583