Reliability shifts in measurement reactivity: driven by content engagement or self-engagement?

Self-report measures require respondents to comprehend the inquiry and then engage the self. Two studies investigated how these 2 processes affect the answers produced. In Study 1,480 participants completed a locus-of-control scale describing themselves, their best friend, or Bill Cosby. Item answers became more reliable as the items moved from the beginning to the end of the measure. The similar increase for self, friend, and Cosby suggested that exposure to the content, rather than self-engagement, was driving the reliability shift. Self-engagement did activate an actor-observer difference in scale means. Study 2 focused on the content engagement process. With more item experience, respondents were better able to distinguish that prototypic items belonged to the locus-of-control scale and that distractor items did not. These studies imply that early questions clarify the meaning of a measure and improve the reliability of later answers.

RCT Entities:   Population Interventions Outcomes