Assessing hearing and cognition challenges in consumer processing of televised risk information: Validation of self-reported measures using performance indicators.

Public health researchers face important challenges if they wish to include measures of hearing or cognitive ability in risk communication studies. We sought validity evidence for self-report measures of hearing and cognitive ability by comparing those measures to performance-based measures and risk information recall. We measured hearing ability (with audiologist-assisted assessment and self report), cognitive ability (with an established performance task and self report), and reactions to direct-to-consumer prescription drug promotion with adults 18 and older (n = 1064) in North Carolina, USA, in 2017. We found moderate correspondence between self-reported hearing loss and audiologist-assessed hearing loss. Both measures also showed a small negative association with recall of presented risk information. Cognitive ability results suggested less substantial correspondence between self report and performance task and the measures differed in predicting risk recall. Our results suggested a moderately efficient measure for hearing ability for research on risk information exposure and retention, and yet also suggested the need for caution regarding future use of self-reported cognitive ability as a substitute for a performance-based measure.

Introduction

Various mass media channels provide consumers with risk information. In the United States, direct-to-consumer (DTC) prescription drug advertising on television offers a prime example. Despite the prevalence of this type of promotion and requirements for inclusion of drug risk and benefit information, people may not experience equal opportunity to process and retain such risk information. Various aspects of DTC media content affect consumer engagement (e.g., Aikin et al., 2017; O'Donoghue et al., 2014; Sullivan et al., 2016; West et al., 2013) and personal characteristics can play a role. For example, people vary in hearing ability and cognitive processing ability. As people get older, they are more likely to experience hearing loss and cognitive challenges (Martin and Jerger, 2005; Salthouse, 1996; Wingfield and Tun, 2001). Older adults in the United States also watch more television (Depp et al., 2010) and typically have ample opportunity for exposure to risk information through ads despite sometimes having constrained hearing and lessened cognitive ability.

Social scientists face important obstacles if they wish to include measures of hearing or cognitive ability, however. Clinical indicators of hearing ability, administered by audiologists, are time- and resource-intensive. Cognitive ability similarly poses resource demands if researchers opt to assess ability on a multi-item processing task. At the same time, self-reported measures of hearing or cognitive ability, while less burdensome, may not be sufficiently valid to be useful. Kimberlin and Winterstein (2008) have warned against disjuncture between theoretical description and construct measurement, which is especially important because different types of hearing and cognitive ability measures sometimes perform differently in models, e.g., Polku et al. (2016). We need to identify valid self-report indicators of hearing and cognitive ability to improve future research.

We generated validity evidence for two types of self-report measures in a sample spanning young to older adulthood. We investigated correspondence between a hearing performance measure administered by audiology staff and a self-reported hearing challenge measure. We also analyzed correspondence between a cognitive ability task-based measure and self-reported cognitive ability.

Hypotheses

We assessed four hypotheses involving construct validity of self-reported measures and nomological validity of all hearing and cognitive ability measures in our study. First, we tested the hypothesis that audiologist-measured hearing performance (measured as hearing loss) positively predicts self-reported hearing inability (Hypothesis 1). Second, we investigated whether performance on a cognitive ability task positively predicts self-reported cognitive ability (Hypothesis 2). Our nomological validation hypotheses focus on the effect of hearing and cognitive ability on information processing; we expected hearing inability would dampen information encoding (and thus retention) and cognitive ability would facilitate encoding. Our third tested hypothesis (Hypothesis 3) was that both audiologist-assessed hearing inability and self-reported hearing inability will negatively predict recall of risk information from an advertisement. Fourth, we tested Hypothesis 4: the proposal that both performance on a cognitive ability task and self-reported cognitive ability will positively predict recall of risk information.

Method

We collected data on hearing ability (both assessed by audiology professionals using World Health Organization (WHO) standards and self-reported), cognitive ability (both measured with an established task and self-reported), and reactions to presented prescription drug promotion from adults aged 18 and older in North Carolina distributed across four age categories: 18 to 25, 40 to 49, 60 to 74, and 75 years and older. We then enacted a two-step approach for measure assessment. First, we analyzed correspondence between self-reported hearing and hearing professionally-assessed by WHO standards as well as between self-reported cognitive ability and task-based cognitive ability, in each case judging association strength as an indicator of construct validity. Second, we sought nomological validation evidence linking hearing or cognitive ability measures with a measure of a theoretically-related construct, namely recall of prescription drug risks presented verbally in a direct-to-consumer television ad that should be at least modestly limited by hearing or cognitive ability.

Data collection and measures

We collected data in person (n = 1064) at an interview facility in Raleigh, North Carolina, and asked hearing, cognitive ability, and television ad perception questions as a part of a study on responses to television ads. For both hearing and cognitive ability, we used a self-reported measure and a performance-based measure.

Hearing

We included a single-item self-reported hearing measure: the question “Do you feel you have a hearing loss?” (including yes or no options) used previously by Sindhusake et al. (2001). We asked the question before audiology professionals assessed hearing and before participants saw the television ad and answered other questions.

For our hearing performance measure, audiologists performed an otoscopy and visual inspection before pure-tone testing. Participants were disqualified from participating if the audiologist observed conditions likely to affect testing (e.g., excessive cerumen) or indicative of active disease (e.g., drainage). Staff measured hearing for everyone who could have at least one ear tested, conducting pure-tone testing through assessment of pure tone air conduction audiometric thresholds in decibels hearing level (dB HL) for each ear at 500, 1000, 2000, 3000, 4000, 6000, and 8000 Hz, an ascending method using 5 dB steps (modified Hughson-Westlake technique, Carhart and Jerger, 1959). Participants who wore hearing devices were asked to remove the devices for the hearing test, however after test completion they were reminded to again use their hearing device before entering the survey room. The purpose of this procedure was to allow standard audiologist assessment of actual hearing but also to observe video viewing as it would occur at home, assuming many of those with hearing challenges have less than perfect hearing even when using a device. We calculated summary pure-tone average (PTA) as the average of air conduction pure-tone thresholds at 500, 1000, 2000, and 4000 Hz (Nash et al., 2011) and a high-frequency PTA (HPTA) as the average of air conduction pure-tone thresholds at 3000, 4000, 6000, and 8000 Hz, frequencies more affected by aging and noise. Using the PTA and HPTA scores, we calculated a final hearing ability measure to capture levels of hearing loss based on gradients from the World Health Organization (2015), with PTA and HPTA ≤25 dB in the better ear coded as no hearing loss, PTA or HPTA 26 to 40 dB in the better ear coded as mild hearing loss, PTA or HPTA 41 to 60 dB in the better ear coded as moderate hearing loss, and PTA or HPTA 61 or more dB in the better ear coded as severe. We had relatively few respondents with profound hearing loss level (>81 dB) and collapsed severe and profound categories.

Cognitive ability

We included two items measuring self-reported cognitive ability with five options ranging from excellent to poor: how participants rate their ability to think quickly in everyday life and how participants rate their memory in everyday life.

For our performance measure of cognitive ability, we implemented a letter digit substitution test (LDST) (Van der Elst et al., 2008). Specifically, using two separate items, we gave participants 60 s to replace randomized letters with appropriate digits as quickly and accurately as possible. The cognitive task included 125 substitution tasks. The number of correct substitutions served as the outcome variable for cognitive ability; a higher score indicated greater cognitive ability.

Risk recall

We measured risk recall with an open-ended question asking for risks associated with a fictitious but realistic prescription drug promoted on television as a treatment for high cholesterol. After establishing intercoder reliability (with Krippendorff's alpha = 1.0), a team of two coders coded responses to create a score for number of risks correctly recalled from the ad, which could have ranged from 0 to 12.

Analysis

We assessed correspondence of task-based and self-reported measures using indicators of association appropriate to the level of measurement: Pearson's r for interval measures and Spearman's rho for relationships involving at least one ordinal measure. To judge evidence for construct validity, we looked for association strength of 0.5 or stronger (rho or correlation coefficient) between self-reported and performance measures. We also assessed the predictive relationship between each measure and risk recall.

Results

Our sample included robust representation of age groups, hearing ability, and cognitive ability. Participants were evenly distributed across age group categories: 270 were 18 to 25, 270 were 40 to 49, 269 were 60 to 74, and 255 were 75 or older. The majority of participants had no hearing loss (61%), 15% had mild hearing loss, and 24% had at least moderate hearing loss; approximately 9% reported wearing a hearing device (of which two-thirds wore the device to complete our study). About half (45.6%) of participants had a below average cognitive ability score.

Regarding Hypothesis 1, we found statistically significant correspondence between self-reported hearing loss and audiologist-assessed hearing loss, with Spearman's rho = 0.55, p < .0001, n = 1064, suggesting modest construct validity for the self-reported measure. We then examined the correspondence of self-reported hearing loss and risk recall after viewing a television ad, per Hypothesis 3: for that relationship, Pearson's r = −0.06, p = .04, n = 1064, which suggested limited predictive validity for the self-reported measure of hearing as well (we would only anticipate a small negative relationship given that, unlike natural viewing situations, participants were instructed to watch the ad on a screen immediately in front of them). We also assessed the correspondence of audiologist-assessed hearing loss and risk recall after viewing a television ad: for that relationship, Spearman's rho = −0.09, p = .005, n = 1064. Self-reported hearing and audiologist-assessed hearing positively corresponded, in other words, and both negatively predicted recall of presented risk information in prescription drug promotion, albeit in a limited way (with coefficients significantly different from but also close to zero).

Regarding Hypothesis 2, results for cognitive ability measures suggested less substantial correspondence between a self-reported indicator and a performance-based one (the LDST), although each self-reported item was significantly associated with the LDST score, with the relationship between self-reported ability to think quickly and LDST, r = 0.25, p < .0001, n = 1061, and self-reported memory and LDST, r = 0.18, p < .0001, n = 1057. Regarding Hypothesis 4, LDST score predicted recall of presented risk information in promotion, r = 0.20, p < .0001, n = 1064, as did each of the self-reported items, with the relationship between self-reported ability to think quickly and risk recall, r = 0.08, p = .01, n = 1061, and self-reported memory and risk recall, r = 0.13, p < .0001, n = 1057. Again, these were small predictive relationships. Nonetheless, LDST score was a stronger predictor of risk recall than either of the other two self-reported measures, with a one-tailed Fisher Z-score test suggesting significant difference between coefficients, p = .049 for the smaller coefficient difference.

Conclusion

Our results support several observations, despite limitations of the study such as the laboratory setting which may have differed from natural television viewing. Researchers who want to assess hearing or cognitive ability using self-reported measures can find more evidence of construct validity for the self-reported hearing measure reported here than for self-reported cognitive ability. Only the self-reported hearing measure achieved a moderate relationship to a performance measure. That suggests people are more likely to accurately report their own hearing ability (or loss) – perhaps based on regular cues from their everyday environment – than to accurately report their own cognitive ability. Self-reported hearing and performance-based hearing performed comparably in predicting risk information recall after viewing a television ad. In both cases prediction of risk information recall was weak, although aggregated small effects across a population still may be noteworthy (Cortina and Landis, 2009). In the case of cognitive ability, the LDST (as adapted here) showed ability to predict recall of risk information from a presented ad and outperformed self-reported cognitive ability measures in that prediction, underscoring the utility of a performance measure despite being time-intensive.