Dana Garbarski1, Nora Cate Schaeffer2,3, Jennifer Dykema3. 1. Department of Sociology, Loyola University Chicago, Coffey Hall 440, 1032 W. Sheridan Rd., Chicago, IL, 60660, USA. dgarbarski@luc.edu. 2. Department of Sociology, University of Wisconsin-Madison, 475 N. Charter Street, Madison, WI, 53706, USA. 3. University of Wisconsin Survey Center, University of Wisconsin-Madison, 475 N. Charter Street, Madison, WI, 53706, USA.
Abstract
PURPOSE: Following calls for replication of research studies, this study documents the results of two studies that experimentally examine the impact of response option order on self-rated health (SRH). METHODS: Two studies from an online panel survey examined how the order of response options (positive to negative versus negative to positive) influences the distribution of SRH answers. RESULTS: The results of both studies indicate that the distribution of SRH varies across the experimental treatments, and mean SRH is lower (worse) when the response options start with "poor" rather than "excellent." In addition, there are differences across the two studies in the distribution of SRH and mean SRH when the response options begin with "excellent," but not when the response options begin with "poor." CONCLUSION: The similarities in the general findings across the two studies strengthen the claim that SRH will be lower (worse) when the response options are ordered beginning with "poor" rather than "excellent" in online self-administered questionnaires, with implications for the validity of SRH. The slight differences in the administration of the seemingly identical studies further strengthen the claim and also serve as a reminder of the inherent variability of a single permutation of any given study.
PURPOSE: Following calls for replication of research studies, this study documents the results of two studies that experimentally examine the impact of response option order on self-rated health (SRH). METHODS: Two studies from an online panel survey examined how the order of response options (positive to negative versus negative to positive) influences the distribution of SRH answers. RESULTS: The results of both studies indicate that the distribution of SRH varies across the experimental treatments, and mean SRH is lower (worse) when the response options start with "poor" rather than "excellent." In addition, there are differences across the two studies in the distribution of SRH and mean SRH when the response options begin with "excellent," but not when the response options begin with "poor." CONCLUSION: The similarities in the general findings across the two studies strengthen the claim that SRH will be lower (worse) when the response options are ordered beginning with "poor" rather than "excellent" in online self-administered questionnaires, with implications for the validity of SRH. The slight differences in the administration of the seemingly identical studies further strengthen the claim and also serve as a reminder of the inherent variability of a single permutation of any given study.
Authors: Andreas Hinz; Ines Conrad; Matthias L Schroeter; Heide Glaesmer; Elmar Brähler; Markus Zenger; Rüya-Daniela Kocalevent; Philipp Y Herzberg Journal: Qual Life Res Date: 2018-03-27 Impact factor: 4.147