Adolescent risk perceptions of ENDS use: Room for change in tobacco education.

Electronic Nicotine Delivery Systems (ENDS) have surpassed combustible cigarettes as the most commonly used tobacco product among U.S. middle and high school students, and research shows that youth do not perceive great risk of harm from regular ENDS use. FDA's public education campaigns help educate youth about the potential risks of using tobacco products and three separate experimental copy testing/ad testing studies (N = 1907) were conducted in support of the FDAs "The Real Cost" (TRC) Cigarette and ENDS Campaigns. These studies provided data for the current investigation which used harm perception items to assess perceived risks of cigarette or ENDS use among adolescents after viewing a public health education advertisement. Eligible youth aged 13-17 who were susceptible, or experimenting, with cigarettes or vaping products were recruited online and randomized into either an ad viewing exposure group, or a non-ad viewing control group. The ads focused on health effects, addiction, or both. Effect sizes on key harm perception measures between groups were computed and standardized to allow for comparisons. Both TRC Cigarette and TRC ENDS ads were able to change harm and addiction perceptions (p < .05); however, effect sizes were significantly larger for items related to health effects for ENDS vs cigarettes (p < .05). When designing youth focused ENDS education campaigns, practitioners should present novel facts in order to take advantage of large effect sizes. Evaluators of early campaign efforts to educate youth about these products may anticipate significant increases in health-related risk perceptions.

Introduction

E-cigarettes, or Electronic Nicotine Delivery Systems (ENDS) use among adolescents has increased steadily and thus is a growing public health concern (Cullen, 2018, U.S. Food and Drug Administration, 2021). ENDS are the most commonly used tobacco products among U.S. middle and high school students with 19.6% of high school students (3.02 million) and 4.7% of middle school students (550 thousand) reporting using ENDS in 2020 (Gentzke et al., 2020). As evidenced by the success of “The Real Cost” campaign and other youth prevention ENDS efforts, mass media campaigns can be effective in reducing tobacco use among youth (Duke, 2019, Schar et al., 2006). In fact, the U.S. has seen a dramatic shift in perceptions about smoking due to decades of sustained tobacco control efforts that have included large mass media campaigns to educate the public about the risks of smoking (Cummings and Proctor, 2014). As a relatively new product, information about the health risks of ENDS is still emerging; however, the extreme popularity of these products among youth accompanied by the risks of nicotine exposure to the developing adolescent brain have made combating youth ENDS use a core priority for the FDA (U.S. Food and Drug Administration, 2018).

The Family Smoking Prevention and Tobacco Control Act grants the U.S. Food and Drug Administration (FDA) the authority to regulate tobacco products and educate the public about the dangers of tobacco use (Family Smoking Prevention and Tobacco Control, 2009). As part of this authority, FDA launched “The Real Cost” campaign in 2014 to educate young people ages 12–17 about the dangers of cigarette smoking. “The Real Cost” campaign ads focused on the risks of tobacco use by highlighting consequences youth are concerned about, such as loss of control due to addiction, dangerous chemicals in tobacco products, and cosmetic health effects like tooth loss and skin damage. An evaluation of “The Real Cost” demonstrated that, in its first two years, the campaign has prevented up to 587,000 youth ages 11 to 19 from initiating smoking, half of whom might have gone on to become established adult smokers (Duke, 2019).

While there have been multiple studies testing messages communicating the potential harms of e-cigarettes to adults (Yang et al., 2019, Banerjee et al., 2016, Owusu et al., 2020), there has been a lack of information about communicating the potential harms of e-cigarettes to adolescents. A significant body of research has demonstrated the importance of “novel and unique information” in public health education, including tobacco education (Brennan et al., 2017, Sangalang et al., 2016). This indicates the importance of messaging on beliefs of which a large proportion of the population do not currently hold and are therefore available to be moved through campaign messaging. In other words, messaging on facts that are already known and agreed upon may have diminishing returns. In addition, beliefs where mean awareness is dramatically above the midpoint are limited in regard to messaging as they do not allow for movement on those beliefs, due to ceiling effects (Hornik and Woolf, 1999).

While there is an extensive body of literature regarding the creation of effective cigarette ads; currently there is limited information available about different ENDS messaging strategies and their comparisons with cigarette messaging on perceived harms. The goal of this study is to see if there is more opportunity to influence ENDS beliefs as there has been less messaging, and less negative public perceptions, regarding ENDS beliefs than those of cigarettes. This manuscript addresses this gap by presenting findings from three separate copy testing studies (Study 1 – four cigarette ad viewing groups and one control group, and Studies 2 & 3 – each with one ad viewing group and one control group). The current study uses data from these copy testing studies, a form of message testing, to assess differences in key ENDS perceptions between those who saw an ENDS prevention ad for “The Real Cost” campaign and those who did not. Effect sizes were then compared between the “The Real Cost”: Cigarette ad study and the “The Real Cost”: ENDS ad studies that use similar messaging strategies.

Current study

The current research data are from three copy testing studies: the first study (N = 1292) was conducted as part of “The Real Cost: Cigarettes” campaign (referred to for the remainder of this paper as TRC:C) and took place between July and August of 2018. The second (N = 300) and third (N = 315) studies were conducted as part of “The Real Cost: ENDS” campaign (referred to for the remainder of this paper as TRC:E) and took place between June of 2018 and March of 2019 (see Table 1 for a description of ads). In-market ads can be viewed at https://www.youtube.com/c/TheRealCost/videos. This research is a secondary data analysis of these studies.

Table 1

Ad. Descriptions.

The Real Cost: Cigarettes (TRC:C) Campaign (Study 1)
Ad Title	Image of Produced Ad	Description
Gift		A teen is alone at a park and has just finished smoking a cigarette when a delivery man appears out of nowhere giving them a gift. Inside the box is a pair of ugly, disgusting, yellow-stained teeth with a note from his cigarettes, thanking him for smoking.
Hacked		A teenage girl with a mouth that resembles a USB port pulls out a device and plugs into the port. Her brain is “hacked” by the nicotine in the device. As she drops the device on the ground and steps on it, we see that she is putting out a cigarette.
Straw City		The big bad wolf is shown walking through a town of pigs. At first the pigs are fearful of him, but when he tries to huff and puff he is unable to do so. The pigs are not scared, and it is revealed that he smokes.
Little Lungs		The main character “Little Lungs” smoked as a teen, so his lungs never developed to their full size.
The Real Cost: ENDS (TRC:E) Campaign
Epidemic (Study 2)		The voiceover states that there is an epidemic spreading that can change your brain, release dangerous chemicals into your bloodstream, and expose your body to acrolein. The epidemic is revealed to be vaping.
Magic (Study 3)		Famous magician Julius Dein walks up to teens on the street and turns their vapes into cigarettes via a magic trick to depict the fact that youth who vape are more likely to start smoking cigarettes.

Methods

The data used in the analyses detailed in this manuscript come from three separate studies. Study 1 tested four cigarette focused ads (Gift, Hacked, Straw City, Little Lungs). Study 2 tested one ENDS focused ad (Epidemic) and Study 3 tested one ENDS focused ad (Magic), see Table 1 for more details about the tested ads.

Sample

To be eligible to participate in the TRC:C study, individuals needed to qualify as a susceptible never-trier by self-reporting never smoking cigarettes (even 1 puff) but being susceptible to smoking as defined by Pierce’s susceptibility scale (Pierce et al., 1995), or identified as an experimenter of cigarettes (having smoked between 1 puff but less than 100 cigarettes). In order to be eligible for each of the TRC:E studies, individuals needed to qualify as a susceptible never-trier by self-reporting never using ENDS (even 1 puff) but being susceptible to using ENDS according to the Pierce Susceptibility Scale, or as an experimenter of ENDS (having used ENDS at least once but less than 100 times in their lifetime). The Pierce susceptibility scale is a 3-item scale that assesses susceptibility by asking the participant if they will try the product soon, in the next year, or if one of their best friends offered the product. If respondents answer anything other than definitely not, they are labeled as susceptible. In the second TRC:E study, a measure was included to exclude frequent users of ENDS (having used more than 21 of last 30 days). For all studies, individuals needed to be between the ages of 12–17, see Table 2 for a breakdown of age and race/ethnicity for each study.

Table 2

Demographic Characteristics.

	The Real Cost: Cigarettes					The Real Cost: ENDS
	Study 1					Study 2		Study 3
	The Gift	Hacked	Straw City	Little Lungs	No Ad	Epidemic	No Ad	Magic	No Ad
	N = 258	N = 259	N = 258	N = 258	N = 259	N = 153	N = 147	N = 158	N = 157
Age
13	15%	17%	16%	17%	17%	10%	12%	17.1%	20.4%
14	16%	14%	17%	14%	15%	16%	22%	24.1%	16.6%
15	21%	18%	19%	27%	22%	22%	22%	20.3%	19.7%
16	22%	23%	24%	22%	21%	27%	22%	20.3%	21.0%
17	26%	28%	24%	21%	25%	25%	22%	18.4%	22.3%
Gender
Male	48%	55%	48%	50%	49%	51%	49%	49.4%	49.0%
Female	52%	45%	52%	50%	51%	49%	51%	50.6%	51.0%
Race/Ethnicity
White	75%	75%	75%	72%	73%	84%	90%	79.7%	82.2%
Hispanic	25%	28%	30%	33% A	29%	13%	11%	16.5%	14.0%
Black or African American	18%	20%	19%	20%	24%	14%	8%	18.4%	15.3%
American Indian or Alaska Native	3%	4%	5%	5%	5%	3%	3%	4.4%	3.8%
Asian	3%	2%	3%	3%	2%	1%	4%	3.2%	2.5%
Native Hawaiian or Other Pacific Islander	2%	2%	0%	2%	0%	1%	1%	1.9%	0%

Procedures

For the TRC:C study, participants were recruited using the mall intercept technique which entails recruiting participants that appear to fit the study criteria at a mall (Rice and Hancock, 2005). For the TRC:E studies, participants were recruited using a preexisting panel database. In both instances parental consent and youth assent were collected and the studies themselves were online surveys. Data collection was approved by FDA’s Institutional Review Board, and the Center for Tobacco Products Research Involving Human Subjects Committee.

Online surveys used an experimental design in which participants were randomly assigned to an ad viewing group or a control group that saw no ad. Participants in the ad viewing groups were exposed to a single ad and were asked several outcome measures to understand various components of the ad. Participants in both the ad viewing and control groups were asked questions about their perceptions of harm related to tobacco use. The harm perception items are the focus of this study. The following questions regarding perceptions of harm of using cigarettes and perceptions of harm of using ENDS were asked in both the TRC:C and TRC:E studies respectively: “If I smoke/vape I will damage my body,” “If I smoke/vape I will damage my lungs” and “If I smoke I will be controlled by cigarettes/If I vape I will become addicted to vaping”. There were four items in Study 1 and 12 items in Study 2 & 3 that were not used in the current analysis as the ads did not focus on these perceived harms. These items were never intended as a scale as each item focuses on an isolated harm that does not apply to every ad. The items that were not analyzed focused on long term consequence such as shortening of life and developing cancer. All items were rated on a 5-point Likert scale from “Strongly Disagree” (1) to “Strongly Agree” (5). As mentioned previously, the current analyses are derived from three different study data sets, one study testing ads for TRC:C, and two studies testing ads for TRC:E. Each of the three studies had a control group for comparisons. This manuscript focuses on these comparable harm perceptions questions by assessing differences in these perceptions between control and exposure for both TRC:C and TRC:E and the effect size associated with these differences. This paper also compares effect sizes between TRC:C and TRC:E to better assess an increase in harm perceptions with these campaign ads.

Statistical analyses

This study assesses differences in perceptions of harm between participants exposed to an ad from the either of the two campaigns (TRC:C or TRC:E). The analyses were conducted using Stata Version 15. There were no control variables in the model. Each analysis compared the exposure condition, with one specific ad, to the control condition. In addition, this study explores a unique methodological approach of assessing differences in effect size as a means for determining the levels of effect on the harm perception items. Effect sizes were computed for the mean comparisons of the ad compared to the control group. Once the effect sizes were standardized into r statistics, the strength of the effect sizes were compared using z-tests.

Results

See Table 3 for means, significance testing statistics, and effect sizes for each of the ads compared to the control group. Effect sizes are reported in Cohen’s d and r. Effect sizes were converted into r as to allow for z-tests comparing correlations, a measure of effect size, from different samples. Missing data for all analyses were removed using listwise deletion.

Table 3

Means and standard deviations of ad viewing and control groups.

		Exposure	Control	Statistic	Effect Size Statistics
KABs	Ad	M (SD)	M (SD)	t	Cohen’s d	r
	TRC: CIGARETTE (Study 1)
Damage my body	Gift	4.46 (0.80)	4.15 (1.09)	3.62***	0.32	0.16
	Hacked	4.40 (0.95)	4.15 (1.09)	2.75**	0.24	0.12
	Straw City	4.40 (0.88)	4.15 (1.09)	2.82**	0.25	0.13
	Little Lungs	4.40 (0.86)	4.15 (1.09)	2.86**	0.25	0.13
	TRC: ENDS
	Epidemic (Study 2)	4.36 (0.85)	3.57 (1.09)	6.91***	0.81	0.38
	Magic (Study 3)	4.13 (0.92)	3.72 (0.99)	3.81***	0.43	0.21
	TRC: CIGARETTE (Study 1)
Damage my lungs	Gift	4.50 (0.70)	4.34 (0.86)	2.29*	0.20	0.10
	Hacked	4.50 (0.68)	4.34 (0.86)	2.32*	0.21	0.10
	Straw City	4.47 (0.77)	4.34 (0.86)	1.78	0.16	0.08
	Little Lungs	4.52 (0.71)	4.34 (0.86)	2.54*	0.23	0.11
	TRC: ENDS
	Epidemic (Study 2)	4.40 (0.83)	3.75 (1.04)	5.89***	0.69	0.33
	Magic (Study 3)	4.13 (0.92)	3.72 (0.99)	2.67**	0.30	0.15
	TRC: CIGARETTE (Study 1)
Be controlled by smoking/Be addicted to vaping	Gift	3.97 (1.10)	3.64 (1.17)	3.24**	0.29	0.14
	Hacked	4.09 (1.05)	3.64 (1.17)	4.54***	0.41	0.20
	Straw City	3.96 (1.10)	3.64 (1.17)	3.15**	0.28	0.14
	Little Lungs	3.95 (1.06)	3.64 (1.17)	3.11**	0.28	0.14
	TRC: ENDS
	Epidemic (Study 2)	3.91 (1.05)	3.49 (1.15)	3.25**	0.38	0.19
	Magic (Study 3)	4.13 (0.90)	3.75 (1.03)	3.49***	0.39	0.19

*p < .05, **p < .01, ***p < .001. The TRC: Cigarette ads were tested in the same sample. The TRCS: ENDS ads were tested in two different samples.

Across all tests of mean differences but one, there were significant differences between the ad viewing condition and the control condition such that harm perceptions increased in the ad viewing conditions. The exception to this was the comparison of the “damage my lungs” item for the Straw City ad viewing condition, compared to the control condition, which was only marginally significant (p = .08) but in the same direction as the other comparisons. Following these tests, z-tests were conducted comparing the effect size measure of r for the cigarette ad viewing condition vs. control condition and the ENDS ad viewing condition vs. control condition. These comparisons were conducted by each cigarette ad comparison by each ENDS ad comparison. See Table 4 for the z-scores.

Table 4

Z-scores for effect size comparisons.

Variable	ENDS Ad	Cigarette Ad	z-score
Damage my body	Epidemic	Gift	3.22*
		Hacked	3.77**
		Straw City	3.63**
		Little Lungs	3.64**
	Magic	Gift	0.72
		Hacked	1.28
		Straw City	1.14
		Little Lungs	1.14
Damage my lungs	Epidemic	Gift	3.27*
		Hacked	3.27*
		Straw City	3.54**
		Little Lungs	3.14*
	Magic	Gift	0.70
		Hacked	0.70
		Straw City	0.98
		Little Lungs	0.56
Be controlled by smoking/Be addicted to vaping	Epidemic	Gift	0.69
		Hacked	-0.14
		Straw City	0.69
		Little Lungs	0.69
	Magic	Gift	0.71
		Hacked	-0.14
		Straw City	0.71
		Little Lungs	0.71

*p < .05, **p < .01

For z-score comparisons, the effect sizes of the Epidemic mean comparisons, compared to the effect sizes of the cigarette ad mean comparisons, were all significant for the “Damage my body” and “Damage my lungs”. None of the comparisons of effect size between the Epidemic viewing condition and the Control condition, compared to the difference between the cigarette ad conditions and the Control group, were significant for “Become addicted to vaping/Be controlled by smoking”. In addition, across harm perception variables, none of the effect size comparisons between the Magic mean comparisons, compared to the cigarette ads’ mean comparisons, were significant.

Discussion

Results from this study show that both TRC:C and TRC:E ad viewing groups had significantly higher levels of agreement of harm perceptions than the control groups, one for each study collection, that did not see an ad. This is in-line with published research from previous TRC:C pretesting studies, which also showed that tested ads significantly changed perceptions (Zhao et al., 2016), suggesting that this campaign also has the potential to positively change youth beliefs about the risks of ENDS use.

Effect sizes, however, did vary across the campaigns. Effect sizes generally varied by campaign and to a certain degree by ad type within the campaign and by belief being asked about. The largest effect sizes were seen for the TRC:E ad Epidemic for the beliefs “Damage my Body” and “Damage my Lungs”. As shown in Table 4, the effect size of based on exposure to this ad was significantly larger than the effect size for exposure to any of the TRC:C ads for the same beliefs. Magic, the other TRC:E ad tested, had an effect size that was comparable to the effect sizes of the TRC:C ads for the beliefs “Damage my Body” and “Damage my Lungs”. It is important to note that the content of the two TRC:E ads are substantively different from each other. Epidemic, the first TRC:E ad, substantiates a variety of health risks related to using ENDS while Magic, the second ad, solely focuses on the fact that youth who use ENDS are more likely to also use cigarettes. It’s possible that the Epidemic ad gave a more powerful message and thus had a stronger impact. Conversely, while Magic did not message directly on physical harms of ENDS use individuals who viewed the ad still had higher perceptions of risks to one’s body and lungs than those in the control group.

All ads tested impacted perceptions related to addiction as assessed by the measure “If I smoke I will be controlled by cigarettes/If I vape I will become addicted to vaping,” however, the effect sizes for this measure were generally middling and did not differ significantly by campaign.

It is also important to assess the nature of the difference in how the TRC:C ads vs the TRC:E ads performed. Generally, the TRC:C ads had small to medium effect sizes whereas the TRC:E ads had medium to large effect sizes using Sawilowsky’s new effect size standards (Sawilowsky, 2009) (d = 0.01, very small; d = 0.02, small; d = 0.5, medium; d = 0.8, large; d = 1.2 , very large; d = 2.0, huge). When looking at differences in how ads performed between those who viewed an ad and the control condition, it becomes clear that this was primarily due to how low negative perceptions were among the control group for the ENDS ads as opposed to being a result of the ENDS ad viewing group having a much higher perception of risk than the cigarette ad viewing group. Perceptions of risks of ENDS use among the TRC:E control group are very low, suggesting that beliefs about ENDS have more room for change than beliefs for cigarettes. Given these findings as well as findings from literature previously cited, it is the assumption of the authors that the larger effects sizes seen in the ENDS studies are primarily due to differences in familiarity of knowledge about smoking and vaping. The harms associated with using cigarettes have been known for decades and there have already been effective multimedia campaigns aimed at educating youth on the harms of cigarette use (US Department of Health and Human Services, 2014, Lantz et al., 2000). That being said, results from this paper do suggest that both TRC:C and TRC:E ads which were tested have the potential to change perceptions among the target audience.

Limitations

While the data collected for the current study came from geographically and ethnically diverse samples, the data was not nationally representative. Further, each of the studies were slightly different in both sample composition and methods. While they were all experiments, with a control group, the screening criteria were different. These differences in the sample may have affected the results. Additionally, in the time since this data was collected, an outbreak of lung injury associated with ENDS use occurred and received wide media attention (Navon et al., 2019).

Additionally, the world experienced a global pandemic whose main feature was lung damage and which emerging research indicates impacted ENDS perceptions and behaviors (Majmundar et al., 2020, Soule et al., 2020). Both events may have dramatically impacted individuals’ perceptions of risks related to ENDS use. Finally, not all the belief statements were semantically identical between products. For example, the items “be controlled by smoking” and “be addicted to vaping” may not always be interpreted as interchangeable by the target audience.

Conclusions

This paper converted effect sizes to r and z-scores to allow for the comparison of levels of perceived harm between ads. Findings from this study suggest that it may be useful to have different expectations for ads that message on a well-known risk such as cigarette use versus a less well-known risk from ENDS. This lack of information about ENDS products relative to cigarettes presents an opportunity to change youth perceptions about these products through focused public health education. As the risks of cigarettes are already well known and generally believed by the public, ads messaging on the risks of cigarette use may only result in modest changes in beliefs among those who view them. Conversely, for ads that message on the risks of new and novel tobacco products – for which the risks are likely less well known – it may make sense to expect to see larger changes in beliefs.

Funding

This work was supported by the US Food & Drug Administration Center for Tobacco Products.

CRediT authorship contribution statement

Matthew W. Walker: Supervision, Conceptualization, Methodology, Writing – original draft, Writing – review & editing. Mario Navarro: Data curation, Writing – original draft, Writing – review & editing. Maria Roditis: Investigation, Writing – original draft. Atanaska (Nasi) Dineva: Investigation, Writing – original draft.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.