Michał Jakubczyk1, Benjamin M Craig2, Mathias Barra3, Catharina G M Groothuis-Oudshoorn4, John D Hartman5, Elisabeth Huynh6, Juan M Ramos-Goñi7, Elly A Stolk7, Kim Rand8. 1. Decision Analysis and Support Unit, SGH Warsaw School of Economics, Warsaw, Poland. 2. Department of Economics, University of South Florida, Tampa, FL, USA. Electronic address: bcraig@usf.edu. 3. Health Services Research Centre, Akershus University Hospital, Lørenskog, Norway. 4. Department of Health Technology and Services Research, University of Twente, Twente, The Netherlands. 5. Haas Center for Business Research and Economic Development, University of West Florida, Pensacola, FL, USA. 6. Institute for Choice, University of South Australia, Adelaide, Australia. 7. EuroQol Research Foundation, Rotterdam, The Netherlands. 8. Health Services Research Centre, Akershus University Hospital, Lørenskog, Norway; Department of Health Management and Health Economics, University of Oslo, Oslo, Norway.
Abstract
OBJECTIVE: To identify which specifications and approaches to model selection better predict health preferences, the International Academy of Health Preference Research (IAHPR) hosted a predictive modeling competition including 18 teams from around the world. METHODS: In April 2016, an exploratory survey was fielded: 4074 US respondents completed 20 out of 1560 paired comparisons by choosing between two health descriptions (e.g., longer life span vs. better health). The exploratory data were distributed to all teams. By July, eight teams had submitted their predictions for 1600 additional pairs and described their analytical approach. After these predictions had been posted online, a confirmatory survey was fielded (4148 additional respondents). RESULTS: The victorious team, "Discreetly Charming Econometricians," led by Michał Jakubczyk, achieved the smallest χ2, 4391.54 (a predefined criterion). Its primary scientific findings were that different models performed better with different pairs, that the value of life span is not constant proportional, and that logit models have poor predictive validity in health valuation. CONCLUSIONS: The results demonstrated the diversity and potential of new analytical approaches in health preference research and highlighted the importance of predictive validity in health valuation.
OBJECTIVE: To identify which specifications and approaches to model selection better predict health preferences, the International Academy of Health Preference Research (IAHPR) hosted a predictive modeling competition including 18 teams from around the world. METHODS: In April 2016, an exploratory survey was fielded: 4074 US respondents completed 20 out of 1560 paired comparisons by choosing between two health descriptions (e.g., longer life span vs. better health). The exploratory data were distributed to all teams. By July, eight teams had submitted their predictions for 1600 additional pairs and described their analytical approach. After these predictions had been posted online, a confirmatory survey was fielded (4148 additional respondents). RESULTS: The victorious team, "Discreetly Charming Econometricians," led by Michał Jakubczyk, achieved the smallest χ2, 4391.54 (a predefined criterion). Its primary scientific findings were that different models performed better with different pairs, that the value of life span is not constant proportional, and that logit models have poor predictive validity in health valuation. CONCLUSIONS: The results demonstrated the diversity and potential of new analytical approaches in health preference research and highlighted the importance of predictive validity in health valuation.