Ruixuan Jiang1, Thomas Kohlmann2, Todd A Lee3, Axel Mühlbacher4, James Shaw5, Surrey Walton3, A Simon Pickard6. 1. Center for Observational and Real-world Evidence, Merck & Co., 2000 Galloping Hill Rd., Kenilworth, NJ, 07033, USA. 2. Institute for Community Medicine, Medical University Greifswald, Ellernholzstraße 1/2, 17489, Greifswald, Germany. 3. Department of Pharmacy Systems, Outcomes, and Policy, University of Illinois at Chicago College of Pharmacy, 833 S Wood St, Chicago, IL, 60612, USA. 4. Health Economics and Healthcare Management, Hochschule Neubrandenburg, Brodaer Str. 2, 17033, Neubrandenburg, Germany. 5. Patient-Reported Outcomes Assessment, Bristol-Myers Squibb, 3551 Lawrenceville Rd, Princeton, NJ, 08540, USA. 6. Department of Pharmacy Systems, Outcomes, and Policy, University of Illinois at Chicago College of Pharmacy, 833 S Wood St, Chicago, IL, 60612, USA. Pickard1@uic.edu.
Abstract
BACKGROUND: Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. METHODS: The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). RESULTS: The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. CONCLUSIONS: Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.
BACKGROUND: Web-based surveys are increasingly utilized for health valuation studies but may be more prone to lack of engagement and, therefore, poor data validity. The objective of this study was to evaluate the effect of imposed engagement (i.e., at least three trade-offs) in the composite time trade-off (cTTO) task. METHODS: The EQ-5D-5L valuation study protocol and study design were adapted for online, unsupervised completion in two arms: base case and engagement. Validity of preferences was assessed using the prevalence of inconsistent valuations and expected patterns of TTO values. Respondent task engagement was measured using time per task. Value sets were generated using linear regression with a random intercept (RILR). RESULTS: The base case (n = 501) and engagement arms (n = 504) clustered at different TTO values: [base case] 0, 1; [engagement] -0.5, 0.45, 0.6. Mean TTO values were lower for the engagement arm. Engagement respondents did not spend more time per TTO task: [base case] 63.3 s (SD 77.9 s); [engagement] 64.7 s (SD 73.3 s); p = 0.36. No significant difference was found between arms for prevalence of respondents with at least one inconsistent TTO value: [base case] 61.1%; [engagement] 63.5%; p = 0.43. Both value sets had significant intercepts far from 1: [base case] 0.846; [engagement] 0.783. The relative importance of the EQ-5D dimensions also differed between arms. CONCLUSIONS: Both online arms had poor quality data. A minimum trade-off threshold did not improve engagement nor face validity of the data, indicating that modifications to the number of iterations are insufficient alone to improve data quality/validity of online TTO studies.
Authors: Richard Norman; Madeleine T King; Dushyant Clarke; Rosalie Viney; Paula Cronin; Deborah Street Journal: Qual Life Res Date: 2010-02-22 Impact factor: 4.147
Authors: Feng Xie; Eleanor Pullenayegum; A Simon Pickard; Juan Manuel Ramos Goñi; Min-Woo Jo; Ataru Igarashi Journal: Health Econ Date: 2016-10-16 Impact factor: 3.046
Authors: Kristina S Boye; Louis S Matza; David H Feeny; Joseph A Johnston; Lee Bowman; Jessica B Jordan Journal: Expert Rev Pharmacoecon Outcomes Res Date: 2014-06 Impact factor: 2.217
Authors: Manraj N Kaur; Richard L Skolasky; Philip A Powell; Feng Xie; I-Chan Huang; Ayse Kuspinar; John L O'Dwyer; Amy M Cizik; Donna Rowen Journal: Qual Life Res Date: 2021-10-18 Impact factor: 3.440