Assessing digital advertising exposure using a virtual experimental protocol.

Introduction: The rapid increase in online public education campaigns underscores the need for a better understanding of the effects of exposure to digital advertising and targeted individual-level outcomes. The goal of this study is to develop a virtual experimental protocol to evaluate the dose-response effects of individual-level exposure to digital video ads on campaign outcomes in a naturalistic online browsing context.
Methods: Young adults aged 18-24 years (n = 221) completed three 5 min viewing sessions on a realistic mock-up of the YouTube mobile app over a period of 2 weeks, followed by a 10-min survey after the third session. Participants were randomized to view between 0 and 6 exposures of ads from an e-cigarette prevention campaign; respondents viewed a total of 2 ads per session, with 0 to 2 of those ads being non-skippable digital video ads from the campaign and/or a dummy ad. The video ads played prior to short YouTube videos. Outcomes measured were self-reported ad recognition, frequency of ad exposure, and main message knowledge.
Results: This study demonstrates a rapidly accessible virtual experimental protocol for evaluating the dose-response effects of digital advertising and individual-level outcomes. Five digital exposures of non-skippable video ads delivered via this platform over a 2-week period generated the highest ad recognition when there were up to six exposures. Higher exposure levels may be needed for message knowledge and ad-content-related effects.
Conclusion: This protocol can be extended to investigate dose-response effects and mechanisms of action of individual-level exposure to digital advertising for multiple campaign outcomes, including changes in knowledge, attitudes, and beliefs. Findings can inform evidence for adequate levels of digital exposure in public education campaigns.

Introduction

Anti-tobacco mass media campaigns have been successful in changing tobacco-related attitudes, intentions, and behaviors among youth and young adults. Evaluation research has consistently shown that these campaigns can mitigate the influence of pro-tobacco marketing and promotional efforts, prevent tobacco use initiation, and reduce tobacco use prevalence.

The evolution of the media landscape over the past two decades has altered the delivery of many public education campaigns. An increasing proportion of campaign advertising is now delivered partially or fully online to adolescent and young adult audiences, among whom the internet and smartphone use is nearly ubiquitous.[2-4] Use of social media is increasing among all age groups but is still most prevalent among youth and young adults, especially for platforms that stream digital video.[2,5,6] Online videos currently comprise more than 82% of all consumer internet traffic, an increasing proportion of which is coming from social video platforms that are most popular among younger users. Among young adults, 95% use YouTube, three-quarters use Instagram and Snapchat, 55% use TikTok, and use is similar among teens.[9,10] Younger users are also more likely than older adults to use these sites daily and often multiple times per day.[2,6]

Similar to other industries, the tobacco industry has leveraged the growth in social media and video platforms to reach and communicate with younger audiences.[11-20] The growing prevalence of exposure to and engagement with tobacco advertising online among youth and young adults[11,21-28] underscores the need for effective anti-tobacco digital campaigns. Public education campaigns at the local, state, and national levels have dramatically shifted media delivery online, including national campaigns such as those from Truth Initiative focused on e-cigarette use prevention among older teens and young adults.[29,30]

Despite the promise of digital advertising, research attributing changes in targeted individual-level campaign outcomes to digital advertising exposure is sparse. A recent systematic review of mass media messaging for public health purposes found little evidence of effectiveness for campaigns utilizing digital media channels. One factor potentially limiting the success of digital campaigns is the lack of evidence on appropriate levels of exposure for digital advertising. Sufficient exposure to digital campaign ads is critical for achieving campaign targeted outcomes. The Centers for Disease Control recommends a benchmark of 800 to 1200 gross rating points per quarter for a traditional anti-tobacco television advertising campaign. This benchmark can be roughly translated to one ad exposure per week or two exposures every two weeks. These benchmarks are based on evaluation studies of media effects on behavior that either compare those exposed to the unexposed or examine a dose-response curve among varying levels of exposure. Unlike television advertising, there is limited research on effective digital campaigns for tobacco prevention and cessation and no rigorous evidence-based benchmarks associated with adequate levels of digital media exposure for the purpose of ad recognition or impacts on other targeted campaign outcomes.

A small number of campaign evaluation studies have been designed to isolate the effect of a digital campaign component on individual-level campaign outcomes or to assess exposure levels sufficient to impact these outcomes. The majority of these studies have utilized self-reported awareness measures to capture digital exposure, which raises concerns regarding endogeneity (i.e. lack of independence between the exposure and the outcome). Endogeneity can occur as a result of reverse causality (i.e. an individual who uses e-cigarettes is more likely to attend to e-cigarette ads), confounding (i.e. an unobserved variable such as an individual propensity for substance use can be associated with both selective attention to any e-cigarette advertising and a higher likelihood of using e-cigarettes),[35,36] or measurement error such as recall bias (i.e. an individual who uses e-cigarettes is more likely to recall e-cigarette advertising).

More recent campaign evaluation studies have utilized alternative approaches to measure digital ad exposure exogenously. Exogenous exposure variables reduce concerns regarding the independence of the exposure and the outcome as they are generally measured separately from the outcome assessment and objectively such that individuals are unaware of the measurement.[35,36] Digital metrics of exogenous ad exposure aggregated by geography or time have been used in campaign evaluation studies but these measures cannot capture the effects of digital ad exposure at the individual level. Understanding the effects of exogenous individual-level digital ad exposure is especially important given that digital media is delivered via precise targeting of individuals’ sociodemographic characteristics and prior behavior rather than solely based on geographic areas, such as with television advertising.

Four studies have examined exogenous individual-level digital ad exposure and campaign outcomes. Of these, three studies utilized quasi-experimental designs and digital ad tracking technology in online panels of consumers to link exogenous individual-level ad exposure during panelists’ natural day-to-day browsing with individual-level campaign outcomes. Kim et al.[39,40] found positive associations between exposure to digital display ads for smoking cessation campaigns and related information-seeking behavior. However, these studies did not examine digital video and only captured a single instance of exposure so were not able to evaluate dose-response effects. Romberg et al's. quasi-experimental study demonstrated that exposure to digital video e-cigarette prevention ads was positively associated with self-reported frequency of ad recognition in a dose-response manner but some evidence indicated selection bias in the construction of the exposed and unexposed groups. There is only one study utilizing an experimental design to assess individual-level digital campaign ad exposure and individual outcomes. In a lab-based setting, Romberg et al. randomized participants to increasing levels of exposure to a mix of forced and skippable digital video ads for e-cigarette use prevention in a naturalistic online browsing environment. The authors found that ad recognition did not vary in a dose-response manner by randomized exposure levels when participants were able to skip ads but ad recognition was sensitive to non-randomized completed ad views. Although Romberg utilized a digital environment that mimicked real-world online browsing, the research occurred in a lab setting and required participants to physically attend multiple lab visits over time. This type of lab-based protocol can be costly and slow with respect to recruitment and retention efforts, is constrained to local convenience samples and may have limited generalizability to a browsing experience in a person's natural environment.

The limited research on digital anti-tobacco campaign advertising and targeted individual-level outcomes undermines campaign development and evaluation[31,34] as well as our understanding of digital advertising effects on attitude and behavior change. Further, the lack of evidence to inform benchmarks for adequate levels of exposure in digital anti-tobacco campaigns reduces campaign efficiency and effectiveness. With the rapid shift from broadcast to digital media, researchers have called for additional studies focused on developing and evaluating digital campaign advertising exposure, exposure measures, and dose-response effects in a controlled digital environment. The overall goal of the current study, which is part of a larger project to examine the dose-response effects of digital media on tobacco use behavior (R01CA253013), is to develop a protocol to examine exogenous digital campaign advertising on targeted campaign outcomes in a naturalistic online browsing context among individuals in their real-world environment; and to evaluate dose-response effects of varying levels of digital exposure on outcomes. In this paper, we will describe the protocol and utilize advertising from Truth Initiative's e-cigarette use prevention campaign for older teens and young adults to examine varying levels of randomized digital advertising exposure to non-skippable ads on self-report measures of ad recognition, frequency of ad exposure and main message knowledge, and evaluate dose-response effects on these outcomes. We hypothesize the following:

Hypothesis 1: Digital exposure to non-skippable digital video ads will be positively associated with increased odds of self-reported ad recognition.

Hypothesis 2: Digital exposure to non-skippable digital video ads will be positively associated with self-reported frequency of ad exposure.

Hypothesis 3: Digital exposure to non-skippable digital video ads will be positively associated with knowledge of the main messages of the ads.

Methods

This research was conducted on a study platform designed and implemented by Ipsos, a global market research firm. The sample was drawn from the KnowledgePanel®, a probability-based nationally representative online panel of adults in the U.S. who are recruited via address-based sampling and provided with computer/internet access if needed. The age of panel respondents is validated through a series of checks during recruitment with corroboration from multiple respondents and is updated each week for each panelist.[43,44] From this panel, we identified eligible respondents as English speakers residing in the U.S. aged 18 to 24 years. The experimental ads presented in this study were designed for an audience aged 15 to 24 years but including individuals younger than 18 years was cost-prohibitive for this study. The study was conducted from 18 February through 16 April 2021 and was approved by the George Washington University Institutional Review Board (IRB).

Study procedure

Potential participants from the panel were recruited and consented online. An email was sent that asked respondents to participate in a study that involved completing three separate five-minute YouTube viewing sessions on their mobile device over a period of 2 weeks, followed by a 10-min follow-up survey within 2 weeks after the third session. Respondents were also told they would receive panel reward points equivalent to $25 for participation. Receipt of reward points required completion of all three sessions and the follow-up survey. A link to an online informed consent was provided in the email. Once consented and enrolled, participants received an email with links to the viewing sessions and up to two email reminders for each session and for the follow-up survey. Participants had up to five days to complete each session and up to two weeks after the third session to complete the follow-up survey, although the majority completed the follow-up survey within one to three days after their final session.

The Ipsos study platform consisted of a realistic mock-up of a YouTube app that participants could browse via their mobile device and watch videos as they normally would within the app. Respondents were asked to browse videos of interest posted on the platform for a total of 5 min per session, to complete the viewing all in one sitting, and to watch videos until they were automatically exited from the session. All videos available to watch were approximately 2.5 to 3 min, allowing time to watch two videos. As participants selected videos during a session, a single non-skippable ad played before the video, simulating a typical pre-roll ad exposure that occurs on YouTube. After 5 min, the platform automatically re-directed participants to the exit screen.

Digital advertising exposure

The experimental exposure consisted of two 15 s test ads selected from different e-cigarette campaigns from Truth Initiative designed for older teens and young adults. One ad (Doug the Pug) was from a campaign that featured a talking animal who was concerned about the lack of data on the long-term effects of using e-cigarettes. The second ad (Ditch Juul) was from a campaign focused on quitting e-cigarettes, offering viewers information on accessing Truth's text-to-quit program, “This is Quitting.” Both ads were extensively tested prior to airing, with evaluations showing high levels of perceived ad effectiveness and knowledge of ad messages (unpublished data, Truth Initiative). A dummy ad promoting a sneaker brand popular among young adults was also shown to some participants.

Randomization

Participants were randomized to one of seven groups of digital advertising exposure to the e-cigarette use prevention ads or a dummy ad (see Table 1). Across three separate viewing sessions, participants were randomized to view between 0 and 6 exposures of ads from the e-cigarette prevention campaign. Participants viewed two advertisements per session, with 0 to 2 of those ads being non-skippable digital video ads from the campaign and/or a dummy ad. The video ads played prior to the short standard YouTube videos on topics unrelated to the ads. Each viewing session was 5 min, each short video was 2.5 to 3 min (including the 15 s ad at the beginning of the video), and respondents were required to watch two videos in each session, ensuring exposure to two ads. The group that was exposed to 0 campaign ads was exposed to the dummy ad twice before each video within each session for all three viewing sessions. Groups exposed to 1 to 5 campaign ads were also exposed to the sneaker ads in some sessions depending on their group assignment. Exposures to the e-cigarette use prevention ads were randomized across the three viewing sessions within groups, with the exception of the treatment groups that received only one or only two campaign ad exposures. For these groups, we ensured that all participants received an e-cigarette use prevention ad on the third and final session to maintain consistent time to the follow-up survey with the other treatment groups, which viewed at least one e-cigarette use prevention ad in their final session.

Table 1.

Randomization of participants into a no exposure group and 6 experimental groups (N = 221).

	No exposuregroup	Experimental group 1	Experimental group 2	Experimental group 3	Experimental group 4	Experimental group 5	Experimental group 6
				Randomize order of sessions	Randomize order of sessions	Randomize order of sessions
Session 1	2 dummy ads	2 dummy ads	Doug the Pugand1 dummy ad(randomized)	Doug the Pug and1 dummy ad (randomized)	Doug the Pug AND Ditch Juul (randomized)	Doug the Pug AND Ditch Juul (randomized)	Doug the Pug AND Ditch Juul (randomized)
Session 2	2 dummy ads	2 dummy ads	2 dummy ads	Ditch Juuland1 dummy ad(randomized)	Doug the Pug and1 dummy ad(randomized)	Doug the Pug AND Ditch Juul (randomized)	Doug the Pug AND Ditch Juul (randomized)
Session 3	2 dummy ads	Doug the Pug OR Ditch Juulmust be in the third sessionand 1 dummy ad(randomized)	Ditch Juul(must be in the third session)and1 dummy ad(randomized)	Doug the Pug OR Ditch Juuland1 dummy ad(randomized)	Ditch Juuland1 dummy ad(randomized)	Doug the Pug OR Ditch Juuland1 dummy ad(randomized)	Doug the Pug AND Ditch Juul (randomized)
Total truth campaign ad exposure	0	1	2	3	4	5	6

The random assignment of eligible participants was independent of any demographic characteristics or study-related behaviors of any sample member to mitigate bias. For the three groups that included a randomized order of treatments (groups 3, 4, and 5), the order of treatment was further randomized for each member of the sample and also was independent of any demographic characteristics or behaviors of any sample members. The allocation sequence was not known to participants until they completed their participation (meaning after they received the treatments randomly assigned), and respondents were not aware of the other treatment arms included in this study.

Prior to recruitment, we conducted a power analysis by utilizing simulated datasets to fit a dose-response curve for digital ad exposure and ad recognition across seven equally-spaced exposure groups and determined that we could recover a dose-response curve with n = 20 to 25 participants per group. Our final sample was N = 221, with a range of 27 to 35 participants per group.

Measures

Self-reported ad recognition. Self-reported ad recognition involved two different screenshot collages, one for each of the possible ads seen followed by the question, “Have you ever seen this ad before now?” with the response options, “Yes” or “No.” The order in which the ads were assessed in the survey was randomized.

Self-reported frequency of ad exposure. Participants who reported recognizing the Ditch Juul or Doug the Pug ad were asked how many times they had seen the ad, with responses options of 1 time, 3 to 5 times, and more than 5 times. Those who did not recognize the ad were categorized as 0 for no self-reported exposure in all analyses. Self-reported frequency of exposure was constructed as an index ranging from 0 (no self-reported exposure) to 3 for each individual ad (with the responses of “1 time” counted as 1, “3–5 times” as 2, and “more than 5 times” as 3) and a summed index across both ads ranging from 0 to 6 for either ad.

Main message knowledge. We measured knowledge of the main message of each ad among all participants with the question “Does the ad communicate each of the following messages?” after showing an ad screenshot. Response options were: “(a) No one knows the long-term effects of vaping; (b) People who vape are being tested on; (c) You can text to join a quit vaping program; (d) Get rid of your vape/JUUL to stop vaping; (e) Do not vape.” Participants read each item and responded yes or no as to whether the ad conveyed the message. Correct responses for Doug the Pug were a and b and correct responses for Ditch Juul were c and d. We created a 0 to 2 index for correct responses for each ad, as well as a 0 to 4 index for total correct responses for both ads. Response e for “do not vape” was also a message in both ads but did not distinguish between the ads so we did not include it in the index here.

Analysis

We calculated descriptives of the sociodemographic characteristics of the sample and compared participant characteristics across the randomized conditions. To assess the hypotheses, we conducted separate logistic or ordinal regression models to analyze the outcomes of self-reported ad recognition, self-reported frequency of ad exposure, and main message knowledge, respectively, as a function of randomized digital exposure for each individual experimental ad (i.e. Ditch Juul and Doug the Pug) and total digital exposure to either experimental ad. We report percentages or means and 95% confidence intervals (CIs) for each outcome by digital exposure for each individual ad and total digital exposure for either ad. We also calculated pairwise comparisons for the outcomes across levels of digital exposure using the Wald statistical test and assessed differences. Statistically significant differences were assessed at the more stringent p < .01 level due to multiple comparisons. We used Stata (version 15 SE) for all analyses.

Results

Table 2 shows the sociodemographics of the sample. Over three-quarters were aged 21 years or older and 65.6% were female. The large majority were white (65.2%), approximately one-quarter were married or living with a partner and over one-third lived in the South. Over three-quarters of the sample had some college education or a bachelor‘s degree or higher. Approximately 10% had used e-cigarettes in the past 30 days and 8.1% had used cigarettes in the past 30 days. We compared these demographics across randomized conditions and found no significant differences between conditions.

Table 2.

Demographic characteristics of the sample (N = 221).

Variable	N	%
Age
18	13	5.9
19	22	10.0
20	19	8.6
21	26	11.8
22	37	16.7
23	50	22.6
24	54	24.4
Education
Less than high school	11	5.0
High school	37	16.7
Some college	104	47.1
Bachelor‘s degree or higher	69	31.2
Race/ethnicity
White non-Hispanic	144	65.2
Black non-Hispanic	9	4.1
Other non-Hispanic	7	3.2
Hispanic	50	22.6
2 + races, non-Hispanic	11	5.0
Gender
Male	76	34.4
Female	145	65.6
Marital status
Married	20	9.1
Divorced/separated	2	0.9
Never married	163	73.8
Living with partner	36	16.3
Region
Northeast	42	19.0
Midwest	54	24.4
South	77	34.8
West	48	21.7
E-cigarette use (past 30 days)
No	199	90.1
Yes	22	9.9
Cigarette use (past 30 days)
No	203	91.9
Yes	18	8.1

Hypothesis 1: Digital exposure to non-skippable digital video ads will be positively associated with increased odds of self-reported ad recognition.

Table 3 shows separate logistic regression models with the outcome of the self-reported ad recognition as a function of digital exposure to each separate ad and total digital exposure to either ad. Participants were randomly exposed to each individual ad from 0 to 3 times, and to either ad from 0 to 6 times. For each separate ad, the odds of self-reported recognition approximately doubled with each one-unit increase in digital exposure to the ad, with odds ratios significant at the p < .01 level. For total exposure, the odds of self-reported recognition to either ad were 1.36 times higher, 95% CI (1.16, 1.58) with each one-unit increase in digital exposure and were significant.

Table 3.

Logistic regression models of self-reported ad recognition as a function of exogenous digital exposure to Ditch Juul, Doug the Pug, and total exposure to either Ditch Juul or Doug the Pug (N = 221).

Digital exposure	Odds ratio	p-value	95% CI
Ditch Juul	2.06	<0.001	1.55, 2.72
Doug the Pug	1.98	<0.001	1.50, 2.63
Total exposure to either Juul or Doug	1.36	<0.001	1.16, 1.58

(a) Ad-specific digital exposure and self-reported ad recognition; (b) total digital exposure and self-reported ad recognition.

(a) Ad-specific digital exposure and self-reported frequency of exposure; (b) total digital exposure and self-reported frequency of exposure.

(a) Ad-specific digital exposure and message knowledge; (b) total digital exposure and message knowledge.

Figure 1(A) shows percentages for self-reported ad recognition across digital exposure for each ad. For the Ditch Juul ad, self-reported ad recognition among respondents ranged from 28.3%, 95% CI (17.8, 41.9) for those with 0 digital exposures to 72.7% (57.8, 83.9) for those with 3 digital exposures. For Doug the Pug, recognition was 30.6% (19.3, 44.8) for 0 digital exposures to 75% (60.2, 85.6) for three digital exposures. Pairwise comparisons for each individual ad showed that ad recognition was significantly higher for two digital exposures compared with 1 or 0 digital exposure but there was no incremental benefit for 3 digital exposures compared with 2 digital exposures as well as no significant difference between 1 and 0 digital exposures.

Figure 1(B) shows percentages for ad recognition of either ad across total digital exposure conditions. Self-reported ad recognition of either ad ranged from 54.3% (37.8, 69.9) for those with 0 digital exposures to 85.2% (66.4, 94.4) for those with six digital exposures. Analyses of pairwise comparisons showed a significant improvement in ad recognition for 5 or more digital exposures compared with fewer. There was no benefit for 6 versus 5 digital exposures.

Hypothesis 2: Digital exposure to non-skippable digital video ads will be positively associated with self-reported frequency of ad exposure.

Table 4 shows separate ordinal regression models for self-reported frequency of ad exposure as a function of digital exposure to each individual ad and total digital exposure to either ad. Results demonstrate that each unit increase in digital exposure significantly increased the odds of reporting a higher frequency of exposure by 1.77 times for Ditch Juul, 1.99 times for Doug the Pug, and 1.40 times for Either ad. All odds ratios were highly significant.

Table 4.

Ordinal logistic regression models of self-reported frequency of ad exposure as a function of exogenous digital exposure to Ditch Juul, Doug the Pug, and total exposure to either Ditch Juul or Doug the Pug (N = 221).

Digital exposure	Odds ratio	p-value	95% CI
Ditch Juul	1.77	<0.001	1.38, 2.27
Doug the Pug	1.99	<0.001	1.54, 2.57
Total exposure to either Juul or Doug	1.40	<0.001	1.23, 1.58

Figure 2(A) shows the mean self-reported frequency of ad exposure across digital exposure conditions for each individual ad. For Ditch Juul, the mean self-reported frequency of exposure ranged from 0.57, 95% CI (0.30, 0.83) to 1.32 (1.02, 1.62) for those with 0 to 3 digital exposures, respectively; similarly, for Doug the Pug, mean self-reported frequency ranged from 0.43 (0.21, 0.64) to 1.34 (1.04, 1.64) for those with 0 to 3 digital exposures, respectively. Pairwise comparisons of mean self-reported frequency of ad exposure were significantly higher for the higher digital exposure conditions versus the lower conditions for each pair except for 1 versus 0 and 3 versus 2 digital exposures, which were not significantly different.

Figure 2(B) shows the mean self-reported frequency of exposure to either ad across total digital exposures. The mean self-reported frequency of exposure index ranged from 0.97 (0.61, 1.33) for 0 digital exposures to 2.59 (1.96, 3.23) for 6 digital exposures. Pairwise comparisons between each digital exposure pair showed a significantly higher self-reported frequency of exposure for 5 or 6 digital exposures compared with 3 or fewer, but the self-reported frequency of exposure was not improved with 6 digital exposures versus 4 or with 6 compared with 5.

Hypothesis 3: Digital exposure to non-skippable digital video ads will be positively associated with knowledge of the main messages of the ads.

Table 5 shows ordinal logistic regression models of main message knowledge as a function of digital exposure to each individual ad and total digital exposure to either ad. Results showed that each one-unit increase in digital exposure was associated with a non-significant increase in the odds of improved main message knowledge by 1.23 times for the Ditch Juul ad, 1.10 times for Doug the Pug, and 1.08 times for Either Ad.

Table 5.

Ordinal logistic regression models of main message knowledge as a function of exogenous digital exposure to Ditch Juul, Doug the Pug, and total exposure to either Ditch Juul or Doug the Pug (N = 221).

Digital exposure	Odds ratio	p-value	95% CI
Ditch Juul exposures	1.23	0.09	0.97, 1.58
Doug the Pug exposures	1.10	0.43	0.86, 1.41
Total exposure to either Juul or Doug	1.08	0.23	0.96, 1.21

Figure 3(A) and (B) shows main message knowledge across digital exposure levels for each individual ad and total digital exposures to either ad, respectively. The mean index response on main message knowledge for individual ads ranged from 1.23, 95% CI (1.02, 1.44) for 0 digital exposures to 1.43, 95% CI (1.23, 1.64) for 3 digital exposures for Ditch Juul, and from 1.10, 95% CI (0.93, 1.28) for 0 digital exposures to 1.25, 95% CI (1.07, 1.43) for 3 digital exposures for Doug the Pug. Tests of pairwise comparisons for each ad found no significant differences in mean main message knowledge across digital exposure levels. In Figure 3(B), the mean message index ranged from 2.37 (2.04, 2.71) for 0 digital exposures to 2.67 (2.25, 3.09) for 6 digital exposures and pairwise comparisons showed no significant differences in mean message knowledge across digital exposures.

Discussion

This study presents a virtual experimental protocol for evaluating digital advertising effects on individual-level exposure and targeted campaign outcomes using a naturalistic social media platform among individuals exposed in their own real-world environment. The protocol is valuable for researchers seeking to explore the effects of individual-level exogenous digital advertising on individual outcomes and is also useful for practitioners seeking to rigorously assess advertising content and varying dose effects prior to campaign implementation. This virtual protocol can be extended by examining more distal outcomes beyond ad recognition, such as vaping and smoking intentions, as well as mechanisms of action for ad effects. Increasing the number of digital ad exposures over an extended period of time and evaluating varying dose-response curves for different campaign outcomes would allow assessment of possible threshold and drop-off effects of exposure for targeted outcomes and could also include conditions in which ads are skippable or of different lengths. Such research can inform our understanding of the impact of varying levels of digital ad exposure on longer-term behavior change and inform benchmarks regarding optimal digital exposure levels for digital public education campaigns.

Additional benefits of the approach presented in this paper include potentially greater generalizability of study findings compared with studies examining digital advertising in lab-based or similar settings. Individuals may be less attentive to digital ads in their day-to-day environment where they may have a different cognitive focus, more potential distractions, or the ability to view other online sites while a non-skippable ad is playing. Understanding digital ad effects in the context in which most advertising is consumed is important for improved external validity. An additional benefit is the use of an online panel and a virtual experiment, which allows recruitment from a broader population and will generally be conducted more quickly than studies requiring participants to attend in-person sessions. Lastly, methods for rapid virtual experiments and remote data collection are increasingly important in the post-COVID research context.

In this study, digital exposure occurred over a period of 15 days, or approximately 2 weeks. For a single ad, there was no effect of one digital exposure on ad recognition over that time period, two exposures achieved peak recognition and three (the maximum) did not provide any incremental benefit above two. For exposure to a mix of campaign video ads, which here consisted of two rotating ads for a total of six exposures, our findings suggest that approximately five digital exposures over a two-week period achieved the highest level of ad recognition. This compares to the television advertising benchmark of roughly two exposures every two weeks. We did not find that increasing digital exposure was associated with increased knowledge of the ads’ main messages. Given that our analyses examined knowledge of two distinct and somewhat nuanced messages from each ad, we also conducted post-hoc analyses to evaluate the comprehension of a simpler message in each ad of “do not vape.” Results were similar in finding no relationship between exposure and message knowledge. It is important to note that the ads in this study were only 15 s each so total exposure time for those exposed ranged from a minimum of 15 s in the lowest exposure level to a maximum of 90 s at the highest level, which occurred over several browsing sessions covering approximately two weeks. Our findings suggest that this level of exposure may be acceptable for basic ad recognition but are not sufficient for improving message knowledge.

Limitations

This study has several strengths and some limitations. One strength is the use of a platform that simulated a relatively naturalistic online browsing environment within a social media site that participants could access in their own environment rather than in a more artificial lab-based setting. Further, we utilized two e-cigarette prevention ads from campaigns that were professionally developed and tailored for a young adult audience. Limitations included the fact that the online platform did not include some important contextual features of real-world social media environments, such as likes, comments, shares, and other mechanisms of interactivity that may reinforce ad recognition and message impact. Also, findings are not necessarily reflective of the more passive incidental exposure to digital campaign advertising that participants may encounter on their own in extended browsing sessions of multiple online and social media sites. Females, white non-Hispanics, and individuals with higher education levels were overrepresented in our sample so generalizability to a national population may be somewhat limited. Lastly, there were high levels of ad recognition among individuals in the no exposure condition, which may reflect prior exposure to the campaign ads. We did not measure exposure to the ads prior to the study, which is a limitation of this study. However, exposure to the ads prior to the study would be approximately balanced across groups due to randomized assignment and we did not find any significant differences across assigned groups in participant characteristics measured. Nonetheless, our findings on exposure levels for ad recognition may be most applicable when baseline levels of campaign advertising awareness are already relatively high.

Conclusions

This study presents a rapid and accessible virtual experimental protocol to evaluate the effects of digital advertising and individual-level outcomes in a naturalistic online and offline browsing environment. Results demonstrate that five digital exposures of non-skippable video campaign ads delivered via this platform over a two-week period generate the highest ad recognition when there are six or fewer exposures but higher levels of exposure may be needed for message knowledge change and ad-content-related effects. This protocol can be extended to investigate dose-response effects and mechanisms of action of individual-level exposure to digital advertising for multiple campaign outcomes, including changes in knowledge, attitudes, and beliefs. Findings can inform evidence for adequate levels of digital exposure in public education campaigns.

Click here for additional data file.

Supplemental material, sj-png-1-dhj-10.1177_20552076221102260 for Assessing digital advertising exposure using a virtual experimental protocol by Jennifer Cantrell, Jeffrey Bingenheimer, Shreya Tulsiani, Elizabeth Hair, Donna Vallone, Sarah Mills, Raquel Gerard and William D. Evans in Digital Health

Click here for additional data file.

Supplemental material, sj-png-2-dhj-10.1177_20552076221102260 for Assessing digital advertising exposure using a virtual experimental protocol by Jennifer Cantrell, Jeffrey Bingenheimer, Shreya Tulsiani, Elizabeth Hair, Donna Vallone, Sarah Mills, Raquel Gerard and William D. Evans in Digital Health