Reliability but not bias: Developing a scale to measure preferred channels for risk information during the COVID pandemic.

To develop a new measure of preferred sources for risk information, two studies asked respondents to indicate what channels they were reliant on for information about COVID-19, from 25 news channels ranging across the political spectrum. Unexpectedly, dependencies clustered around level of reliability rather than the political orientation of the news channel. In other words, each cluster included media channels from both the left and right side of the political spectrum, while dependencies clustered into sources that varied by the degree to which their content is reliable. Participants who turned to lower reliability channels indicated lower risk perceptions, less accurate probability estimations, reduced vaccination intentions, and lower protective behavioral intentions. Those inclined to use higher reliability channels indicated higher risk perceptions, more accurate probability estimations, increased vaccination intentions, and higher protective behavioral intentions. These relationships are discussed in terms of implications for our understanding of source reliance and risk perception, information sufficiency, and implications for both future research and public health interventions.

INTRODUCTION

COVID‐19 has created unprecedented challenges in terms of getting timely, accurate, and actionable information to the public such that they can make informed public health decisions. As the crisis has progressed, the public has been called upon to respond to complex health information that has changed over time, and to make nuanced estimations of risk as conditions shift regionally and nationally. Further complicating this matter, there is ample evidence that news outlets have presented politicized interpretations of this information to the public, and that conservative leaning media outlets have largely downplayed the severity of the crisis and the need for a unified public health response. Coupled with the notion of “echo chambers,” concerns have been raised that audiences will gravitate toward programming that supports their existing beliefs about COVID, and that heavily politicized content may attenuate both risk perceptions and desire to protect oneself and others.

This begs the question of how to identify the channels audiences are most reliant upon for information, and to evaluate their risk perceptions and protective motivations in the context of these dependencies. To that end, the current project attempted to develop an instrument gauging risk information preference across the political spectrum. Two separate data collections asked respondents to indicate what channels they were most reliant on for information about COVID‐19, from 25 news channels ranging across the political spectrum. The factor analyses in both studies produced surprising results. Rather than clustering around political orientation, source reliance seems to cluster around level of information reliability (high versus low), and reliance on legacy media. Relationships are then described between these clusters of source reliance and risk perception, preventative behaviors, and vaccine intention, along with implications for future research.

Political orientation and vaccine intentions

The COVID‐19 epidemic, while the most recent, is far from the only example of the politicization of health and risk information in the United States. For example, there is ample evidence to suggest that parental vaccination behavior is at least partially explained by political ideology. Rabinowitz et al. (2016) found that parents identifying as conservatives and moderates were significantly less likely to have fully vaccinated their children prior to the age of 2. Similarly, Estep (2018) found that personal belief exemptions (PBE) to routine childhood vaccinations were significantly higher in majority conservative neighborhoods. Researchers have also found that adolescent vaccination adherence for at least one dose of the human papillomavirus (HPV), tetanus, and meningococcal vaccine respectively, was significantly lower in states with a historical tendency to elect conservatives to office (Bernstein et al., 2016).

Beyond childhood and adolescent vaccination, researchers have also identified political disparities in overall vaccination trust and flu vaccination adherence. State survey data from Oregon and New Hampshire revealed that progressives were significantly more likely than conservatives to trust scientists for information about vaccines (Hamilton et al., 2015). These differences were revealed to widen with education; while they likely reflect underlying ideological differences rather than specific attitudes toward vaccines, they suggest that existing political opinions impact health information processing. Similarly, in a recent poll of a representative sample of United States adults, conservatives indicated lower vaccination rates than did progressives (Kannan & Veazie, 2018).

Belief‐consistency and information seeking

More broadly, while not focused on political attitudes exclusively, Meppelink et al. (2019) identified confirmation bias in online health information seeking. Participants were more likely to select belief‐consistent information than belief‐inconsistent information and found the belief‐consistent information to be “more credible, useful, and convincing” (p. 129). It should be noted that while participants were citizens of the Netherlands, a systematic review of the literature related to parental vaccine hesitancy found that interventions in the United States that incorporated the target populations’ religious or political affiliations were more likely to be successful (Olson et al., 2020). This suggests that within the United States, the processing of health risk information is likely related to existing belief structures, whether they be spiritual or political in nature. The data imply that rather than risk information being processed on its own merits, existing belief structures have an impact on both the interpretation of the information, and the likelihood of behavioral adoption (Olson et al., 2020).

In the current climate of COVID‐19, multiple studies have identified the impact of political ideology on attitudes, opinions, and behaviors (Dhanani & Franz, 2020; Hart et al., 2020; Rothgerber et al., 2020). Rothgerber et al. (2020) found evidence that conservatives were less likely to comply with COVID‐19 mitigation behaviors such as social distancing. Conservatives were also less likely to be informed about the virus, believe the accuracy of the media's coverage about the virus, and believe that COVID‐19 posed a health risk to themselves or someone they knew. The researchers suggested that it might be the politicization of the pandemic that led to conservatives being more likely to downplay the health risks associated with COVID‐19, and ultimately reducing the likelihood they would follow recommended public health behaviors.

A content analysis performed on COVID‐19 coverage from March 2020 to May 2020 found overall polarization in newspaper and television network broadcasts during this time, with newspapers having more politicized coverage including more politicians than scientists in their coverage (Hart et al., 2020). While newspapers and television broadcasts are not an exhaustive analysis of media coverage, the early politicization, especially in the newspaper coverage, likely had ramifications for the public's understanding of COVID‐19. Data also indicates that reliance on certain media formats and channels is associated with COVID‐19 knowledge, belief in misinformation, and prejudice, and that those who relied more heavily on television and radio news channels were less likely to have accurate information about COVID‐19 (Dhanani & Franz, 2020). Given the algorithms used on social media sites, political bias in COVID‐19 information may be more likely resulting in not only inaccurate understanding of health risks, but also potentially dangerous behavior (Bakshy et al., 2015). Additionally, prior research has suggested that simply correcting misinformation, once believed, is more complicated when attached to political topics (Krause et al., 2020).

What seems clear is that, at least in the United States, lack of knowledge is not the only communication challenge that needs to be faced during a public health crisis. Existing attitudes, particularly political ideology, appear to impact the processing of health risk information and the subsequent adoption of behavior. This is especially concerning in the context of a long history of research suggesting that people will gravitate toward content that confirms their existing attitudes, beliefs, and biases. Research in psychology, communication, and political science consistently indicates that individuals are inclined to avoid information challenging their attitudes, beliefs, and behaviors, and more inclined to engage with information that confirms them (Eagly & Chaiken, 1998; 2005; Hart et al., 2009; Olson & Stone, 2005).

Selective exposure and polarization

The politicizing of health risk information is especially problematic considering evidence that that selective exposure to confirming information will drive further attitudinal polarization (Garrett et al., 2019; Jones, 2002; Stroud, 2007, 2008, 2010). Scholars have noted that on social media platforms, individual choice to avoid ideology inconsistent information played a stronger role in limiting exposure than algorithmic ranking (Bakshy et al., 2015). In fact, in a recent longitudinal study found that not only did political affiliation and ideology predict media exposure, but also that partisan selective media exposure increased between 2000 and 2012(Rodriguez et al., 2017). Most recently, scholars have noted that these ideological differences may also result in individuals discounting ideological divergent media coverage related to the COVID‐19 outbreak (Rothgerber et al., 2020). In other words, exposure itself is not sufficient to change attitudes or beliefs (Conway et al., 2021). Other research has focused specifically on online “echo chambers,” given the range of tailored content now available through social media and web pages. The term “echo chamber” typically refers to a mediated environment in which a narrow range of attitudes and perspectives is offered, such that those drawn to the environment are only receiving information with which they already agree (Jamieson & Cappella, 2008; Sunstein, 2009). While these environments may exist in traditional media or online, social media in particular has driven increased consideration of “echo chambers,” due to its capacity to bring users content driven by algorithmic filters (Dubois & Blank, 2018; Pariser, 2011).

In summary, extant research suggests that not only is inaccurate information problematic in terms of influencing public health outcomes, but the politicizing of health information may lead audiences to selectively consume problematic content. To explore attitudinal polarization and its connection to politicized health risk information, it seems necessary to develop a measure of information reliance that is based on news outlet, as opposed to medium. It should be noted that the key concept here is reliance, as opposed to exposure, dependency, or some other phrase. A substantive body of research suggests that exposure and reliance are related but distinct concepts (Moy et al., 2005). For example, while exposure may serve to define what people come across in their media landscape, reliance has been operationalized as the extent to which a given source is important to an individual in their efforts to acquire information about a given topic (Pierce et al., 1990). While it may be the case that media exposure is shaping attitudes, it may also be the case that poor information literacy is driving both exposure and related health decision making; both possibilities are accounted for when considering reliance as a key predictor of risk perceptions and outcomes.

In order to examine the degree to which risk information reliance clusters around different outlets, an instrument was developed examining these reliance patterns across a range of political orientation. The scale was deployed in two separate data collections during the summer of 2020, one examining mitigation behaviors related to COVID and the other exploring vaccine intentions and perceived norms. Items asked participants to evaluate their degree of reliance on specific for information about COVID19, with instructions to consider their use of these channels across all media platforms. The response options included 25 different news outlets drawn from a broad range of political inclination and reliability (see adfontesmedia.com). Ad Fontes Media rates the reliability and bias of media channels through the review of multiple pieces of content (e.g., a newspaper article; podcast episode) by multiple expert analysts.

It should be noted that from a conceptual standpoint, we chose the Ad Fontes typology over others (such as Pew) since it focuses entirely on the nature of the content, as opposed to the anticipated or real characteristics of the audience; we went into the study wishing to examine how audiences gravitated toward information about COVID, and the ways in which source preference clustered within these audiences. Exploratory factor analysis was used to reduce these 25 channels to clusters, which were then verified using Confirmatory Factor Analysis. These factors were then evaluated in terms of their tendency to predict risk perception, protective action, and vaccine intentions.

STUDY 1

Study 1 methods

Survey data was collected from a representative sample of US residents during summer 2020. Data collection was performed by Qualtrics through their online panel. Qualtrics was also responsible for establishing sampling quotas to ensure representativeness across ethnicity, age, and income. Responses were screened for straight‐lining and irregular response times. The data collection was funded by a grant from the National Science Foundation, award number 2029258. In total, 5019 participants responded, and were compensated $2.25 for their participation. In terms of demographics, Whites comprised 61.5% of the sample, followed by Latinx (17.1%), Blacks (13.6%), Asians (6.0%), American Indian or Alaska Native (1.1%), Native Hawaiian or Pacific Islander (0.1%), and some other race or ethnicity (0.5%). Males accounted for 48.5% of the sample, females for 51.1 %. For annual income, 18.7% reported making less than $24,999 per year, 24.4% reported making between $25,000 and 49,999, 19.4% between $50,000 and $74,999, 13.2% between $75,000 and $99,999, and 23.7% reported making over $100,000. The average respondent age was 45.56(SD = 17.81).

Instrumentation

Information channel reliance

A series of items asked participants to evaluate their degree of reliance on specific for information about COVID, with instructions to consider their use of these channels across all media platforms. Respondents were asked to indicate their use of 25 different news outlets drawn from a broad range of political inclination and reliability for information about COVID (see adfontesmedia.com). For each source, responses were scored on a seven‐point Likert scale ranging from 1 (“not at all reliant”) to 7 (“very reliant”), with a response of 4 indicating “neutral.” Factor analysis and reliability estimates can be found in the results section. See osf.io/65gcb for copies of the instrumentation used.

General risk perception

For general risk perception, participants responded to the event hazard‐outrage scale (Lachlan & Spence, 2010). This 32‐item scale measures both perceptions of the generalized risk posed by a given hazard and negative affect associated with these risk perceptions. Confirmatory factor analysis supported a two‐factor model, CFI = 0.91, RMSEA = 0.09. Only the hazard factor was used in the current analysis as an indicator of general risk perception (ω = 0.91).

Probability estimation

A series of single‐item indicators were adapted from previous research to evaluate perceived likelihood of COVID19 transmission and consequences (see Spence et al., 2017). Participants were asked to estimate the percentage of the US population that will become infected (M = 48.47, SD = 26.50), the percentage of those that will develop a serious illness (M = 37.39, SD = 26.56), and the percentage that will die as a result (M = 30.80, SD = 28.46).

Protective action

Respondents then yes or no to whether they had engaged in a series of protective actions recommended by the CDC (2020). These included including cleaning and disinfecting a home more frequently, covering one's mouth when sneezing or coughing, staying home from work or school, keeping a distance of 6 feet from others, avoiding touching one's face, using hand sanitizer, and washing hands more regularly. Affirmative responses were summed to produce an additive index of protective actions, (M = 6.11, SD = 1.36).

Demographics

Participants were asked a series of demographic questions (see above).

Study 1 results

A number of steps were taken in evaluating the factor structure, internal consistency and reliability of the scale under development. First, responses to the 25 source reliance items were subject to an exploratory factor analysis (EFA), using a maximum likelihood extraction and varimax rotation. The obtained solution was then re‐analyzed through a confirmatory factor analysis approach (CFA) using AMOS. An alternate CFA was conducted to check for second‐order unidimensionality, and these two solutions are compared. Finally, a series of exploratory analyses examined relationships between the obtained factors and key variables of interest.

Exploratory factor analysis

The 25 items measuring reliance on specific news channels were subject to an EFA, using a maximum likelihood extraction and varimax rotation. The Kaiser–Meyer–Olkin measure of sampling adequacy was found to be 0.97, while Bartlett's test of sphericity was significant at χ 2(300) = 120,971.83, p < 0.001. Thus, ample evidence supported the appropriateness of an EFA procedure.

Using the criteria of eigenvalues > 1, the EFA procedure produced a three‐factor solution, accounting for 71.07% of the variance when rotated. The factors included eigenvalues of 14.80, 2.53, and 1.29, and when rotated accounted for variance of 33.33, 23.82, and 13.92%, respectively. The rotated factor matrix revealed primary factor loadings greater than 0.50 for almost all indicators (see Table 1). The items represented in the three factors seem to represent legacy television news (CNN, NBC, CBS, ABC), content with some degree of reliability (Washington Post, New York Times, Wall Street Journal, etc.), and content of lower reliability and elevated political bias (both liberal and conservative), much of which is primarily found online and through social media (Buzzfeed, Breitbart, Mother Jones, etc.).

TABLE 1

Study 1—Factor analysis of media source dependency items exploratory factor analysis with MLM and varimax rotation

Channel	Factor 1	Factor 2	Factor 3
CBS	0.18	0.24	0.85
ABC	0.16	0.24	0.80
NBC	0.15	0.28	0.80
CNN	0.16	0.49	0.50
NPR	0.37	0.55	0.32
BBC	0.43	0.61	0.28
AP	0.39	0.64	0.32
Reuters	0.47	0.64	0.25
NYT	0.27	0.81	0.24
WashPo	0.29	0.81	0.26
WSJ	0.41	0.70	0.22
MSNBC	0.26	0.52	0.51
HuffPo	0.53	0.61	0.24
Fox	0.52	−0.16	0.42
Infowars	0.84	0.30	0.16
Blaze	0.85	0.28	0.13
OANN	0.85	0.28	0.12
Breitbart	0.85	0.26	0.10
NewsMax	0.84	0.32	0.15
Daily Caller	0.82	0.36	0.18
Daily Kos	0.77	0.43	0.20
Buzzfeed	0.57	0.49	0.23
Mother Jones	0.71	0.47	0.21
Slate	0.67	0.48	0.23
New Republic	0.77	0.43	0.20

Confirmatory factor analysis

Given the somewhat unexpected findings in the exploratory analysis, a confirmatory procedure was conducted using AMOS to further test the obtained three‐factor model and evaluate overall model fit. Modification indices were examined, and the model respecified accordingly (see Buchholz & Hartig, 2020). The internal consistency of the proposed three factor model was tested using maximum likelihood modeling, and reliability for each factor was assessed by calculating McDonald's Omega using the phantom variable approach (see Goodboy & Martin, 2020; Hayes & Coutts, 2020; Raykov, 1997).

Overall model fit statistics indicated adequate fit for the three‐factor model, CFI = 0.93, RMSEA = 0.08 (see Hu & Bentler, 1999; Marsh et al., 2004). Individual factor loadings were quite high, with most exceeding 0.70 (see Table 2). The item measuring reliance on Fox News was an outlier; while achieving adequate strength, it was substantially lower than the other loadings in the model (0.41). All three subfactors demonstrated a high level of reliability, at ω   = 0.96, 0.92, and 0.90. In order to test for second‐order unidimensionality, a second CFA was performed to test a one factor model of channel dependence. This model did not meet adequate fit criteria, CFI = 0.74, RMSEA = 0.15, and individual factor loadings were substantially lower than those found in the three‐factor model. Thus, the confirmatory analyses support the three‐factor model.

TABLE 2

Study 1: Factor analysis of media source dependency items confirmatory factor analysis with standardized regression coefficients

Channel	Factor	Coefficient
CBS	Legacy	0.91
ABC	Legacy	0.83
NBC	Legacy	0.84
CNN	Legacy	0.70	ω = 0.90
NPR	High reliability	0.73
BBC	High reliability	0.81
AP	High reliability	0.82
Reuters	High reliability	0.83
NYT	High reliability	0.81
WashPo	High reliability	0.83
WSJ	High reliability	0.82
MSNBC	High reliability	0.72
HuffPo	High reliability	0.82	ω = 0.92
Fox	Low reliability	0.41
Infowars	Low reliability	0.87
Blaze	Low reliability	0.87
OANN	Low reliability	0.87
Breitbart	Low reliability	0.85
NewsMax	Low reliability	0.89
Daily Caller	Low reliability	0.91
Daily Kos	Low reliability	0.91
Buzzfeed	Low reliability	0.76
Mother Jones	Low reliability	0.88
Slate	Low reliability	0.84
New Republic	Low reliability	0.91	ω = 0.96

Risk outcomes

In order to explore the predictive power of these factors on outcomes associated with probability estimation and risk response, a series of hierarchical regression models examined the impact of the high reliability and low bias channels (referred to as “high reliability” for the remainder of the manuscript; Wall Street Journal, New York Times, NPR, etc.), low reliability and high bias media (referred to as “low reliability” for the remainder of the manuscript; Breitbart, Buzzfeed, OANN, etc.), and legacy television factors on general risk perception, probability estimation, and protective actions against COVID. Composite variables for each of the three media factors were created by taking the mean item score within factor. For each regression model, demographic variables were entered on the first step, followed by both demographics and the channel reliance factors (see Table 3). Collinearity diagnostics did not suggest collinearity issues across any of the analyses. Given the sample size and degree of statistical power, findings accounting for trivial amounts of variance (e.g., < 2%) are not reported.

TABLE 3

Study 1 regression analyses

	F	P	R 2	ΔR 2
General risk perception
Block 1	26.52	0.001	0.03
Block 2	82.96	0.001	0.12	0.09
Infection rate estimate
Block 1	65.70	0.001	0.06
Block 2	63.09	0.001	0.09	0.03
Serious illness estimate
Block 1	96.62	0.001	0.09
Block 2	133.16	0.001	0.18	0.09
Fatality estimate
Block 1	119.56	0.001	0.11
Block 2	179.47	0.001	0.29	0.18
Mitigation behaviors
Block 1	16.25	0.001	0.02
Block 2	35.60	0.001	0.06	0.04

Note: Dependent variables are in Italics.

Block 1: Sex, age, education, income, and ethnicity.

Block 2: Sex, age, education, income, ethnicity, low reliability, high reliability, and legacy media.

For generalized risk perception, a significant model was detected for the demographic predictors, though it accounted for a small amount of variance, F (5, 4971) = 26.52, p < 0.001, R 2 = 0.03. Adding the media reliance factors led to significant improvement in the model, F (8, 4968) = 82.96, p < 0.001, ΔR 2 = 0.09. Reliance on low reliability channels negatively predicted general risk perception, β = −0.22, p < 0.001, while both high depth (β = 0.22, p < 0.001) and legacy (β = 0.25, p < 0.001) channel reliance positively predicted general risk perception.

Analyses then went on to examine the impact of these variables on the three indicators of risk estimation. For estimates of the percentage of the US population that would become infected, demographic variables significantly were significantly predictive, F (5, 4958) = 65.70, p < 0.001, R 2 = 0.06. Adding the channel reliance factors led to a small but statistically significant improvement in the model, F (8, 4955) = 63.09, p < 0.001, ΔR 2 = 0.03. Low reliability (β = 0.05, p < 0.05) and legacy channel reliance (β = 0.12, p < 0.001) were predictive of risk estimates, while reliance on low bias channels was not. A similar pattern emerged in estimates of those infected who would become seriously ill, while accounting for a larger proportion of the variance. The initial model was found significant, F (5, 4943) = 96.62, p < 0.001, R 2 = 0.09. Adding the reliance measures significantly improved the model, F (8, 4940) = 133.16, p < 0.001, ΔR 2 = 0.09. Once again, low reliability (β = 0.24 p < 0.001) and legacy channel reliance (β = 0.15, p < 0.001) emerged as predictive, while high reliability channel reliance did not.

Regression analyses indicated stronger predictive power on the third risk estimate variable, the percentage of those infected who would die from COVID. The initial model was found significant, F (5, 4945) = 119.56, p < 0.001, R 2 = 0.11. Adding the reliance factors strongly and significantly improved the model, F (8, 4942) = 179.47, p < 0.001, ΔR 2 = 0.18. Low reliability (β = 0.33, p < 0.001) and legacy channel reliance (β = 0.15 p < 0.001) positively predicted fatality estimation, while reliance on high reliability channels was negatively predictive (β = −0.09 p < 0.001).

Finally, this analytic strategy was repeated once again for the index of mitigation behaviors. A significant model was detected for the demographic predictors, F (5, 4770) = 16.25, p < 0.001, R 2 = 0.02. Adding the reliance factors once again significantly improved the model, while accounting for a modest amount of variance, F (8, 4767) = 35.60, p < 0.001, ΔR 2 = 0.04. Like general risk perception, reliance on low reliability channels negatively predicted mitigation behaviors (β = −0.11, p < 0.001), while both high reliability (β = 0.11, p < 0.001) and legacy channels (β = 0.17, p < 0.001) were positively predictive.

STUDY 2

Study 2 methods

Data were again collected by Qualtrics during the summer of 2020. Funding for the study was received from (home institution redacted). This second sample was comprised of 510 Connecticut residents. Once again, participants were compensated with $2.25 for their responses. Participants were screened to ensure that they were at least 18 years of age, and stratified sampling was used once again to generate a representative sample of adult state residents.

Risk perception

A reduced five‐item scale was used to evaluate general risk perception, adopted from the longer survey used in Study 1(Lachlan & Spence, 2010). Questions were asked on a five‐point Likert scales ranging from “strongly disagree (1)” to “strongly agree (5).” The statements were, “I have been worried over the course of the COVID‐19 outbreak,” “At the beginning of the COVID‐19 outbreak, I thought the risk was serious,” “As the COVID‐19 outbreak has progressed, I've been scared I would be badly affected,” “As the COVID‐19 outbreak has progressed, I thought my life was in danger,” and “As the COVID‐19 has progressed, I've worried my friends or loved ones might be in danger.” The scale had sufficient reliability, ω = .84.

COVID‐19 behavioral intentions

Measures of both behavioral intentions and perceived norms were adapted from prior research (Myers & Goodwin, 2011). Once again, these questions were asked on a five‐point Likert scale, ranging from “strongly disagree (1)” to “strongly agree (5).” For behavioral intentions a single item indicator was used: “Do you intend to receive a vaccination for COVID‐19 when it becomes available?” three‐item scale was used to measure perceived vaccine norms. These items included “People who are important to me would approve of me receiving a COVID‐19 vaccination,” “My family would approve of my receiving a COVID‐19 vaccination,” and “My friends would approve of me receiving a COVID‐19 vaccination.” The scale had strong reliability, ω = 0.90

Channel reliance measure

The measure of channel reliance was used once again, with the omission of Fox News as an option due to a low initial factor loading (0.35), and CBS due to data completion issues. Once again, a confirmatory procedure was conducted using AMOS to test the revised three‐factor model using maximum likelihood modeling. Overall model fit statistics indicated adequate fit for the revised three‐factor model, CFI = 0.94, RMSEA = 0.07. Individual factor loadings were again quite high, with all factor loadings exceeding 0.66 and most much stronger (see Table 4). All three subfactors demonstrated high level of reliability, at ω = 95, 0.91, and 0.74. Thus, the confirmatory analyses once again support a three‐factor model.

TABLE 4

Study 2: Factor analysis of media source dependency items confirmatory factor analysis with standardized regression coefficients

Channel	Factor	Coefficient
ABC	Legacy	0.69
NBC	Legacy	0.71
CNN	Legacy	0.77	ω = 0.74
NPR	High reliability	0.70
BBC	High reliability	0.73
AP	High reliability	0.72
Reuters	High reliability	0.76
NYT	High reliability	0.72
WashPo	High reliability	0.78
WSJ	High reliability	0.74
MSNBC	High reliability	0.74
HuffPo	High reliability	0.79	ω = 0.91
Infowars	Low reliability	0.84
Blaze	Low reliability	0.91
OANN	Low reliability	0.81
Breitbart	Low reliability	0.85
NewsMax	Low reliability	0.87
Daily Caller	Low reliability	0.89
Daily Kos	Low reliability	0.88
Buzzfeed	Low reliability	0.67
Mother Jones	Low reliability	0.80
Slate	Low reliability	0.72
New Republic	Low reliability	0.84	ω = 0.95

Study 2 results

A similar analytic strategy was used as in Study 1, using hierarchical regression with demographics entered on the first block and the three‐factor channel reliance measure added to the second (see Table 5). Similar findings were detected for general risk perception. The initial model was found significant, F (3, 506) = 5.03, p < 0.001, R 2 = 0.04. Adding the reliance factors strongly and significantly improved the model, F (6, 503) = 15.20, p < 0.001, ΔR 2 = 0.12. High reliability (β = 0.17, p < 0.01) and legacy channel reliance (β = 0.24, p < 0.001) positively predicted risk estimation, while reliance on low reliability channels was negatively predictive (β = −0.09, p < 0.001).

TABLE 5

Study 2 regression analyses

	F	P	R 2	ΔR2
General risk perception
Block 1	5.03	0.001	0.04
Block 2	15.20	0.001	0.16	0.12
Willingness to get vaccinated
Block 1	12.43	0.001	0.05
Block 2	11.57	0.001	0.12	0.07
Perception of vaccine norms
Block 1	12.58	0.001	0.07
Block 2	15.47	0.001	0.16	0.09

Note: Dependent variables are in italics.

Block 1: Sex, age, and income.

Block 2: Sex, age, income, low reliability, high reliability, and legacy media.

For the single item concerning willingness to get vaccinated, the initial model was found significant at F (3, 506) = 12.43, p < 0.001, R 2 = 0.05. The addition of the reliance measures significantly improved the model, F (6, 503) = 11.57, p < 0.001, ΔR 2 = 0.07. Once again, reliance on low reliability coverage negatively predicted vaccine intentions (β = −0.17, p < 0.01), while high reliability coverage (β = 0.17, p < 0.02) and reliance on legacy media (β = 0.18, p < 0.001) positively predicted vaccine intention.

Similar findings were also detected for perception of vaccine norms. The initial model was found significant at F (3, 506) = 12.58, p < .001, R 2 = 0.07. Adding the reliance measures in the second predictor block significantly improved the model (F (6, 503) = 15.47, p < 0.001, ΔR 2 = 0.09). Like the prior two analyses, reliance on low reliability coverage negatively predicted perceived vaccine norms (β = −0.29, p < 0.001), while high reliability coverage (β = 0.20, p < 0.01) and reliance on legacy media (β = 0.16, p < 0.01) positively predicted vaccine norms.

Post hoc analyses for Fox

Given the unexpected failure of Fox to load with other factors in the second study, and its comparatively low factor loading in the first study, a series of post hoc analyses examined the impact of reliance on Fox as a news source on the dependent variables of interest. Across the board, Fox accounted for a statistically significant but extremely small proportion of the variance when controlling for audience demographics. In the first study, adding Fox to the demographic block marginally improved the model for predicting risk perception, ΔR 2 = 0.01, β = 0.09, p < 0.001. Similar results were detected for infection estimates (ΔR 2 = 0.003, β = 0.06, p < 0.001), estimates of who would become seriously ill (ΔR 2 = 0.02, β = 0.16, p < 0.001), fatality estimates (ΔR 2 = 0.04, β = 0.19, p < 0.001), and behavioral intentions (ΔR 2 = 0.01, β = 0.09, p < 0.001).

Similar findings were uncovered when looking at the impact of Fox on vaccine‐related outcomes. The addition of Fox news reliance to demographic predictors did not account for a significant amount of variance in general risk perception, ΔR 2 = 0.000, n.s. Adding Fox to the predictor block accounted for small but significant changes in intention to vaccinate (ΔR 2 = 0.02, β = −0.15, p < 0.001) and perceived vaccination norms (ΔR 2 = 0.03, β = −.017, p < 0.001).

DISCUSSION

This research, utilizing two unique datasets, identified a robust three‐factor media reliance scale that subsequently explained risk perceptions, probability estimates, and a number of protective behavioral intentions. Participants in both studies who relied on legacy media channels (ABC, NBC, CNN, and CBS for study one) indicated higher risk perceptions and stronger intentions to vaccinate and perform other protective behaviors. Participants in both studies who relied on coverage from low reliability channels (e.g., Buzzfeed, Blaze) indicated lower risk perceptions, less accurate probability estimations, reduced vaccination intentions, and lower protective behavioral intentions. Participants in both studies who relied on high reliability channel coverage (e.g., NYT, Reuters) had higher risk perceptions, more accurate probability estimations, increased vaccination intentions, and higher protective behavioral intentions. While demographics did occasionally explain a significant amount of variance in risk perception and behavioral intentions, the proportion of this variance was rather small. Perhaps of greater importance, the factor analyses suggest that channel reliance may be more a product of willingness to engage than of political orientation.

Contrary to a growing body of literature that suggests that COVID‐19 is understood through a political lens (Chu et al., 2021; Hart et al., 2020; Ju & You, 2021; Rothgerber et al., 2020), the three‐factor solution identified in both datasets provides evidence that media reliance in health risk contexts may be more complex than just political identity. While existing literature suggests that health information is often viewed through a political lens, the results of this study suggest that a more detailed understanding of media reliance beyond political ideology may be useful in explaining additional variance. The three‐factor media reliance measure did not differentiate between political affiliation, with channels like Mother Jones (liberal) and Breitbart (conservative) loading highly onto the low reliability media channel factor in both studies. This does not negate the role of politics in health information news seeking, but it does suggest that the political component alone may be insufficient. The results from both studies support an important relationship between type of media reliance and COVID‐19 related understanding, attitudes, and behavioral intentions.

Second, while reliance on legacy news and high reliability media both positively predicted risk perceptions of COVID‐19, participants differed in their estimations related to COVID‐19 probabilities: the percentage of United States residents who would become infected with COVID‐19, become seriously ill, and who would ultimately die from COVID‐19. Participants who relied on legacy news, as well as low reliability media, had higher estimations for all three COVID‐19 probabilities, while individuals who relied on high reliability media had lower and more accurate estimations. This suggests that while general risk perceptions were high for both legacy news and high reliability media consumers (i.e., COVID‐19 should be taken seriously), individuals who relied on high reliability news may have had a more nuanced understanding of the reality of COVID‐19 in the United States.

This may be the result of a few different factors. One, the legacy media channels are all broadcast news networks, which have been criticized in the past for sensationalized reporting and needlessly elevating fear (Kilgo et al., 2019; Slattery et al., 2001). Further, most of the high reliability media channels included in the measure are generally considered elite publications, which may result in journalistic decisions that prioritize tempered language (e.g., NYT, WSJ, BBC) (Carpenter, 2007). It may also be the case that these publications prioritize thorough and reasonably objective reporting, given the perceived expectations of their audiences.

Next, the low estimation of general COVID‐19 risk, but overestimation of specific risk for participants who relied more heavily on low reliability media channels is interesting to consider. Individuals who rely on low reliability channels may be incorrectly underestimating the general threat posed by COVID‐19. While not directly tested by the current data, this could be a function of level of a number of different factors related to interest and to baseline concern. It is likely that those not already concerned about the pandemic would not devote a great deal of time to learning more about it, thus the low reliability content meets some threshold of information sufficiency. What is puzzling, however, is that this direction reverses for specific threats, such as infection rates and fatality estimates, which are overestimated amongst those reliant on low reliability content. Both are incorrect, but in different directions. It could be the case that those already prone to greater interest are doing more nuanced information seeking, as this would be consistent with the findings that those reliant on legacy media and high reliability content have more realistic expectations. It also suggests that there may be factors beyond the scope of the current data that are influencing media selection.

There are a number of potential sources of variability in interest and information seeking. A number of studies suggest that those high in need for cognition, or who are higher in motivation, may be inclined to seek out a wider range of media on a given subject, may be more inclined to seek detail, and may acquire a more accurate understanding of risk (see Chen, 2018; Dvir‐Gvirsman, 2015; Jung et al., 2020; Wei et al., 2007, 2010, 2015). Similarly, research on dual process models of information processing, such as the heuristic systematic model (Eagly & Chaiken, 1993), would suggest that people will continue to seek information until they reach some level of information sufficiency, whereby they feel that they can make successful decisions (Kahlor, 2007; ter Huurne, Griffin, & Gutteling, 2009; Yang et al., 2011). This may also drive scanning and information consumption among those most interested, while those less interested are satisfied with less sophisticated content. Again, the connection between these processes and the information seeking patterns found in the current study is speculative, and future research should attempt to directly establish the relationship between level of involvement and reliance on these high reliability, low reliability, and legacy channels.

Fourth, this research also suggests that media reliance is related to other COVID‐19 attitudes and intentions. Likely related to the higher perceived general risk, individuals who relied more heavily on legacy and high reliability media had higher intentions to vaccinate and perform protective behaviors and elevated vaccine norms; individuals who relied more heavily on low reliability media channels had reduced intentions and norms. There is existing research to suggest that when individuals do not feel a risk is genuine, they are unlikely to respond in a protective manner (Lachlan & Spence, 2007). It is not clear what is driving these attitudes, given that a number of low reliability media channels are considered left‐leaning (e.g., Mother Jones, New Republic, Daily Kos) and it is unlikely that they underplayed the threat of COVID‐19. Again, cognitive involvement may be playing a role, as research surrounding other health crises has shown that increased cognitive involvement results in more accurate understanding (Jung et al., 2020; Wei et al., 2007, 2010, 2015).

Finally, the somewhat unique role that Fox News played in both studies should be noted. In the first data set, Fox News had a lower factor loading than any other channel, and in the second data set, Fox News did not have a sufficient factor loading to be included in the analysis. The post hoc analyses examining the impact of reliance on Fox alone suggests that it may not play much of a role in general risk perception at all, but may be associated with vaccine norms and intentions. If anything, reliance on Fox was a slight positive predictor of risk perception and the probability of negative outcomes. At the same time, it negatively predicted both vaccine norms and vaccine intentions. Once again, there may be more nuanced factors accounting for interest and/or health decision making that are drawing viewers toward particular sources, and Fox News may be explaining variance in a unique way disparate from the other media channels on both the right and the left. Politics is not the only lens through which to understand media reliance and COVID‐19 related attitudes and behavioral intentions, but it remains an important one to consider.

Limitations and future directions

While the models presented in this research differentiate between predictor and outcome variables, the data is cross‐sectional meaning that causality is not able to be determined. Additionally, while most variables were consistent across both study one and study two, there were discrepancies that hinder our ability to truly compare across the two datasets. A replication of the current findings with a larger and broader group of news channels may provide more insight on the extent to which these decisions are related to willingness to engage (as opposed to political alignment). It is also the case that self‐reports of “reliance” are not the same thing as use; it may be worthwhile to try to capture passive processing of politicized health information, and/or compare self‐reports of reliance with reports of time spent using a given channel. Future studies should consider including variables related to both volume of use and cognitive processing in order to shed further light on these initial findings. The scale developed herein may also be adaptable to assessing information use and responses to information concerning future public health risks.