Melanie Hawkins1, Gerald R Elsworth2, Sandra Nolte3, Richard H Osborne2. 1. Swinburne University of Technology, Centre for Global Health and Equity, School of Health Sciences, PO Box 218, Hawthorn, Melbourne, Victoria, 3122, Australia. melaniehawkins@swin.edu.au. 2. Swinburne University of Technology, Centre for Global Health and Equity, School of Health Sciences, PO Box 218, Hawthorn, Melbourne, Victoria, 3122, Australia. 3. Charité-Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Medical Department, Division of Psychosomatic Medicine, Berlin, Germany.
Abstract
BACKGROUND: Contrary to common usage in the health sciences, the term "valid" refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane's argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. METHODS: The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane's five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. RESULTS: The general HLQ interpretive argument was built to demonstrate Kane's five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. CONCLUSIONS: Kane's five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.
BACKGROUND: Contrary to common usage in the health sciences, the term "valid" refers not to the properties of a measurement instrument but to the extent to which data-derived inferences are appropriate, meaningful, and useful for intended decision making. The aim of this study was to determine how validity testing theory (the Standards for Educational and Psychological Testing) and methodology (Kane's argument-based approach to validation) from education and psychology can be applied to validation practices for patient-reported outcomes that are measured by instruments that assess theoretical constructs in health. METHODS: The Health Literacy Questionnaire (HLQ) was used as an example of a theory-based self-report assessment for the purposes of this study. Kane's five inferences (scoring, generalisation, extrapolation, theory-based interpretation, and implications) for theoretical constructs were applied to the general interpretive argument for the HLQ. Existing validity evidence for the HLQ was identified and collated (as per the Standards recommendation) through a literature review and mapped to the five inferences. Evaluation of the evidence was not within the scope of this study. RESULTS: The general HLQ interpretive argument was built to demonstrate Kane's five inferences (and associated warrants and assumptions) for theoretical constructs, and which connect raw data to the intended interpretation and use of the data. The literature review identified 11 HLQ articles from which 57 sources of validity evidence were extracted and mapped to the general interpretive argument. CONCLUSIONS: Kane's five inferences and associated warrants and assumptions were demonstrated in relation to the HLQ. However, the process developed in this study is likely to be suitable for validation planning for other measurement instruments. Systematic and transparent validation planning and the generation (or, as in this study, collation) of relevant validity evidence supports developers and users of PRO instruments to determine the extent to which inferences about data are appropriate, meaningful and useful (i.e., valid) for intended decisions about the health and care of individuals, groups and populations.
Authors: Alison Beauchamp; Roy W Batterham; Sarity Dodson; Brad Astbury; Gerald R Elsworth; Crystal McPhee; Jeanine Jacobson; Rachelle Buchbinder; Richard H Osborne Journal: BMC Public Health Date: 2017-03-03 Impact factor: 3.295
Authors: Roy W Batterham; Rachelle Buchbinder; Alison Beauchamp; Sarity Dodson; Gerald R Elsworth; Richard H Osborne Journal: BMC Public Health Date: 2014-07-07 Impact factor: 3.295
Authors: Yu-Chi Chen; Christina Cheng; Richard H Osborne; Lars Kayser; Chieh-Yu Liu; Li-Chun Chang Journal: J Med Internet Res Date: 2022-01-19 Impact factor: 5.428
Authors: Shyam Sundar Budhathoki; Melanie Hawkins; Gerald Elsworth; Michael T Fahey; Jeevan Thapa; Sandeepa Karki; Lila Bahadur Basnet; Paras K Pokharel; Richard H Osborne Journal: Int J Environ Res Public Health Date: 2022-03-09 Impact factor: 3.390