Assessing the relationship between ad volume and awareness of a tobacco education media campaign.

BACKGROUND: The relation between aided ad recall and level of television ad placement in a public health setting is not well established. We examine this association by looking back at 8 years of the California's Tobacco Control Program's (CTCP) media campaign.
METHODS: Starting in July 2001, California's campaign was continuously monitored using five telephone series of surveys and six web-based series of surveys immediately following a media flight. We used population-based statewide surveys to measure aided recall for advertisements that were placed in each of these media flights. Targeted rating points (TRPs) were used to measure ad placement intensity throughout the state.
RESULTS: Cumulative TRPs exhibited a stronger relation with aided ad recall than flight TRPs or TRP density. This association increased after log-transforming cumulative TRP values. We found that a one-unit increase in log-cumulative TRPs led to a 13.6% increase in aided ad recall using web-based survey data, compared to a 5.3% increase in aided ad recall using telephone survey data.
CONCLUSIONS: In California, the relation between aided ad recall and cumulative TRPs showed a diminishing return after a large volume of ad placements These findings may be useful in planning future ad placement for CTCP's media campaign.

Introduction

Ironically, the recent proliferation of media communication channels has not made it simpler to reach more people. Instead, it has made it more difficult and time consuming to captivate audiences with the sheer number of outlets available. With traditional media channels, public health mass media campaigns have demonstrated the capacity to effectively influence adult health behaviours provided that campaigns employ a variety of communication tactics directed at changing the whole social system, thereby achieving high message reach and frequency with their intended audience.1 Undoubtedly, large media campaigns that attempt to influence community-level factors aim to alter perceived social norms at the community level.2 Although many of these campaigns have focused on reducing college student drinking norms3–5 tobacco control media campaigns, in particular, have also provided rich sources of information for drawing these conclusions. A number of researchers have demonstrated that state-funded or nationally-funded counter-marketing media campaigns favourably impact youth smoking initiation.6–17 It has been further shown that media campaigns can persuade adults by providing motivation to quit18–25 and even reduce smoking prevalence.26–28 Further information on the relative performance of different anti-tobacco advertising using naturalistic exposure in adults and youths can be found in chapter 11 of the National Cancer Institute's The Role of Media in Promoting and Reducing Tobacco Use.29

For several decades, television advertisers outside of tobacco control have recognised the value of ad recall as a better measure of effectiveness than relying solely on more distal measures such as rating points, reach or frequency.30 However, expenditure data estimated using price or media cost indices have been most widely adopted as the standard for evaluating generic advertising programmes because of their relative accessibility and ease of translation.31 Several studies have also examined the use of healthcare advertising exposure measured by advertisement expenditures or gross rating points (GRPs), which are essentially a measure of people who have seen an advertisement and its processing or communication effects.32 Others have examined downstream effects of exposure to advertising on target audience action, such as product requests or sales data.33 34 However, much of the discussion in public health around advertising effectiveness and GRPs has taken place in the tobacco control arena.

Using GRPs or targeted rating points (TRPs), which are a measure of GRPs within a specific target population as a measure of exposure, anti-tobacco advertising has been shown to be associated with better smoking-related attitudes, beliefs and behaviours among youths35 across gender and race/ethnic groups,36 to generate calls to a telephone smoking cessation helpline,37 38 to increase quitting behaviours among adults23 and to decrease smoking prevalence among adults.39 Most of these studies use short-term GRP levels, such as monthly GRPs, but some use cumulative GRPs as well.

Rather than examining distal behavioural measures of campaign effectiveness, in this study we assess efficiency of ad placement using ad recall as a proximal measure of effectiveness. This approach is comparable to analyses previously presented by Southwell and colleagues40 and Niederdeppe.41 Southwell and colleagues demonstrated that a measure of ad recognition, termed ‘encoded’ ad exposure, for a national anti-drug media campaign is highly correlated with GRP density.40 Niederdeppe provided evidence that both aided recall and confirmed ad awareness are positively correlated with cumulative GRPs, the sum of GRPs for each ad over the life of an ad, for the Florida Anti-Tobacco Media Evaluation surveys.41

As funding for tobacco control in California decreases,42 it has become even more critical to increase the efficiency of an ad reaching the media campaign's targeted audience. Therefore, the primary goal of the present analysis is to examine the relation between television advertisement awareness and the campaign's TRPs over the past decade with the goal of directing the programme's future marketing decisions.

Background

From its inception in 1990 to the present, the intention of the California Tobacco Education Media Campaign has always been to increase public awareness around issues of secondhand smoke and the tobacco industry's deceptive marketing tactics, as well as promoting smoking cessation at the population level.43 The California's Tobacco Control Program (CTCP) is somewhat distinctive by aiming its campaign messages at non-smokers, as much as smokers. Using paid commercials, the campaign incorporates several media components including television, radio, billboards, print and public relations activities.43 The California media campaign has been evaluated in the past using a variety of macro-level approaches to show its effectiveness.6 44–46

In July 2001, CTCP began to monitor advertisement recall related to its media campaign using ongoing surveillance efforts. The first five series of the survey were conducted using telephone methodology by The Research Business International and Field and Tab Incorporation. The telephone survey used an aided recall method, where the interviewer provided a brief description of each advertisement and then asked the respondents if they had seen the ad.

In 2005, CTCP implemented population-based online panel methodology to conduct its media evaluation efforts using Knowledge Networks.47 This prospective study design was intended to address causality issues (not presented in this paper). An aided recall method similar to the telephone survey was used to obtain ad recall.

Methods

Telephone methods

Using random digit-dialling (RDD), cross-sectional statewide telephone interviews were administered to a sample of selected 18–55-year-olds in California after each of five media flights (table 1). Each media flight lasted approximately 3 weeks. Each telephone survey used quotas to obtain a specified number of respondents based on smoking status, gender and age group.

Table 1

Sample sizes and smoking status across data collection waves

Wave	Time period	Survey mode	Non-smokers	Smokers	Total
1	July 2001	Telephone	463	442	905
2	December 2001	Telephone	484	422	906
3	December 2002	Telephone	459	452	911
4	December 2003	Telephone	956	843	1799
5	June 2004	Telephone	962	835	1797
6	December 2005	Web	1609	366	1975
7	June 2006	Web	1994	466	2440
8	December 2006	Web	1512	454	1966
9	June 2007	Web	1403	357	1760
10	December 2007	Web	982	257	1239
11	June 2008	Web	797	231	1028

Note that the web-respondents in waves 6 through 11 are not unique.

Survey samples were purchased from Survey Sampling International (SSI). Based on the American Association for Public Opinion Research's (AAPOR) formula number 4,48 the response rate was 19.9% for wave 5. We were unable to obtain disposition codes for the other waves. The survey instrument included questions related to advertisement recall, tobacco use behaviours and attitudes related to secondhand smoke, and the tobacco industry. On average, the survey took about 17 minutes to complete. Final data were weighted to the year 2000 California population.

Web-based methods

Respondents comprising the online panel are part of an existing nationwide panel recruited by RDD methods. The panel is meant to be a probability-based sample representative of the underlying population,49–51 although for surveillance purposes we sampled only 18–55-year-olds in California and modestly oversampled smokers. More information about the web-panel survey methodology can be accessed at http://www.knowledgenetworks.com/knpanel/index.html. The online survey on average is completed in about 20 minutes and is fielded about 1 week after the media flight. Based on the AAPOR formula number 3,48 the response rate varied from 27.2% to 27.6% with an average of 27.4%. All data were weighted to the ‘most current at the time’ Current Population Survey for California.

Mode effects

Undoubtedly, mode effects exist between the telephone and web-panel surveys. Unlike online-survey methodology, interviewer-led telephone surveys have the disadvantage of increased social desirability bias.52 However for the purposes of assessing aided recall, the difference between auditory and visual stimuli present a greater potential for recall bias.52 These differences may be further compounded by the samples being assembled differently.

Upon inspecting data before using weights to control for potential differences, we see that the demographics by mode are clearly different owing to the quotas placed on the telephone survey. The web-panel is skewed towards slightly higher educational attainment compared to the telephone surveys but this is partially because of the younger age distribution of the telephone sample (table 2). Regardless, each survey mode was then weighted to match the demographics of the California population. To address appropriate concerns regarding survey mode, we will conduct all analyses for each survey mode separately as well as combining both modes together. These combined results may be interpreted as a weighted average of the results from the separate modes.

Table 2

Unweighted demographic percentages for waves 1 to 11

	Sample size	Male	Female	18–24	25–34	35–44	45–55	High school or less	Some college	College graduate
Telephone
Wave 1	905	49.9	50.1	17.9	27.4	28.6	26.1	28.5	38.2	33.3
Wave 2	906	50.2	49.8	17.7	28.4	27.7	26.2	27.4	34.9	37.7
Wave 3	911	50.2	49.8	17.9	27.2	29.4	25.5	29.4	38.0	32.6
Wave 4	1799	50.9	49.1	18.4	28.0	27.4	26.2	27.2	35.7	37.1
Wave 5	1797	50.7	49.3	18.0	27.7	28.5	25.8	29.6	36.4	34.0
Web-panel
Wave 6	1975	42.1	57.9	5.4	17.5	32.5	44.7	15.4	42.4	42.2
Wave 7	2440	38.8	61.2	5.7	18.5	31.0	44.8	15.4	42.2	42.4
Wave 8	1966	39.3	60.7	4.7	16.7	29.2	49.4	15.4	41.4	43.2
Wave 9	1760	39.0	61.0	5.9	15.4	31.1	47.6	15.2	38.6	46.2
Wave 10	1239	41.5	58.5	5.4	14.3	29.5	50.8	14.5	38.5	47.0
Wave 11	1028	41.4	58.6	5.0	13.7	30.5	50.8	11.9	38.4	49.7

The telephone sample included quotas based on gender, age and smoking status, while the web-based survey included quotas resulting in slightly oversampling smokers.

Smoking status

Many ads specifically target smokers differently from non-smokers. We believe that aided ad recall may vary by smoking status owing to targeting of the message but also to other ad characteristics, including executional details. Consequently, we will present data for smokers, non-smokers and overall.

Statistical methods

Statistical analysis was conducted using the CORR and GENMOD procedures in SAS v9.1. The GENMOD procedure was used to fit models to the generalised estimating equations (GEE) method with a normal error distribution and the exchangeable correlation structure to account for correlated data caused by the airing of some of the same ads over more than one wave. We treated the aided ad recall of the same ad (in different waves) as the repeated measure.

Survey measures

Although the campaign's surveillance efforts also measure saliency and engagement, we focused on aided ad recall in this study. This is not meant to be a measure of ultimate effectiveness for a public health mass media campaign, but aided recall is a useful proximal measure of mental processing beyond visual recognition.53–55 In this analysis, aided recall is also a useful proxy measure of reaching the targeted audience for the purpose of estimating an appropriate volume of future ad placements. It may be also useful for estimating a point of no return—that is, when advertisements no longer reach new audiences.

Aided recall was measured by providing respondents with a specific description of the advertisement and then asking respondents if they had seen the anti-smoking television advertisement recently. As an example, the question for the advertisement ‘Crocodile 3′ asked, ‘Over the past 30 days, do you remember seeing an anti-smoking television advertisement that features a crocodile character who talks about the difficulty of running a business that loses 400 000 customers every year?’

Over the 7-year-period, 22 CTCP advertisements have been evaluated. Several of these ads can be viewed at http://www.cdc.gov/tobacco/media_communications/countermarketing/mcrc/index.htm. Six advertisements were evaluated more than one time because they had been aired in more than one media flight, which resulted in 29 distinct evaluation points (table 3). The telephone survey was used to collect information at 17 different time and advertisement points, while the web-based survey was used for 12 advertisement points.

Table 3

Targeted rating point levels, aided awareness and type of advertisement: July 2001–June 2008

Advertisement name	Primary advertisement type	Cumulative TRP	Flight TRP	Smoker-aided awareness	Non-smoker-aided awareness	Overall-aided awareness	Wave	Mode
Echo	Health effects	2150	113	48.7	47.1	47.4	4	Telephone
Echo	Health effects	1729	846	47.5	43.8	44.5	3	Telephone
Clinical	SHS	1463	344	66.7	61.8	62.9	7	Web
Bubbles	SHS	1268	595	30.7	30.8	30.8	5	Telephone
Training	Cessation	1206	210	53.4	49.5	50.4	7	Web
Apartment	SHS	1012	286	60.9	54.1	55.3	11	Web
Icons	CPTI	951	285	35.6	48.0	45.9	11	Web
Ken Lane: Drive Along	CPTI	776	439	50.1	52.4	52.0	2	Telephone
Target Market	CPTI	690	397	48.8	46.8	47.2	5	Telephone
Icons	CPTI	670	238	57.8	50.5	51.7	9	Web
Caution Tape	SHS	590	212	58.7	50.2	51.7	9	Web
Drive	SHS	585	585	49.7	51.5	51.2	2	Telephone
Ken Lane: Lights	CPTI	548	78	51.9	45.7	46.7	4	Telephone
Training	Cessation	542	542	52.6	45.0	46.8	6	Web
Daddy's Tape	Cessation	473	473	50.1	41.2	42.7	3	Telephone
Bubbles 2	SHS	467	467	42.1	36.0	37.4	6	Web
Programmed	CPTI	464	272	62.9	61.1	61.4	10	Web
Apartment	SHS	463	271	46.1	36.3	38.0	10	Web
Ken Lane: Next Generation	CPTI	411	327	53.4	47.1	48.1	1	Telephone
Live Without	SHS	401	401	29.6	27.3	27.7	3	Telephone
Ken Lane: Ethnic Targeting	CPTI	385	385	48.0	43.2	43.9	4	Telephone
Clinical	SHS	384	384	39.7	32.4	33.5	4	Telephone
Ken Lane: Drive Along	CPTI	335	335	40.8	32.5	33.9	1	Telephone
Ghost	Health effects	301	301	36.8	38.6	38.3	2	Telephone
Icons	CPTI	215	215	48.4	39.0	41.5	8	Web
Ken Lane: Race Deal	CPTI	199	199	30.3	20.8	22.4	1	Telephone
Crocodile 3: The Business	CPTI	192	192	38.5	35.6	36.1	2	Telephone
Caution Tape	SHS	176	176	33.8	26.9	28.7	8	Web
Crocodile Tears	CPTI	135	135	41.2	32.1	33.7	1	Telephone

CPTI, countering pro-tobacco influences; SHS, secondhand smoke; TRP, target rating point.

Smoking status was based on two questions that assessed smoking 100 cigarettes over a lifetime and smoking every day or some days at the time of the survey.

TRPs are a measure of advertisement exposure that estimates the potential viewership for a particular advertisement by multiplying the advertisement's estimated reach and frequency. It is similar to GRPs in that it attempts to account for the proportion of households that own a television set and watch a particular television programme during a specific time. However, TRPs differ from GRPs by providing viewership data on specifically targeted demographic groups. Reach is a measure of the number of people from the target population in the audience, while frequency is the number of times that the targeted population views the advertisement. In this paper. We do not use actual TRP levels, instead we rely on planned TRPs that are usually estimated from previous viewership during these time spots. Actual TRPs are not available until the actual media buy is reconciled and although these are monitored and reconciled to obtain ‘make good’ ad placement, historically they have not been catalogued and tracked by the programme's advertising agency. TRP levels were used for each specific advertisement with the targeted group being the 24–55-year-old population. For each advertisement, we created a variable for flight TRPs, which are advertisement-specific TRPs in the 3 months before evaluation for each flight. We also created a cumulative TRP variable, which is the total TRPs for the specific advertisement up to that point in time. Additionally, we created a total TRP density variable, which we defined as the cumulative TRPs for an advertisement divided by the number of weeks that the advertisement actually ran up to that point in time.

The advertisements were placed in 12 exclusive media markets throughout California. Because the advertisements were not placed in all media markets or at the same TRP levels in each media market, we created a statewide estimate of TRPs by weighting the TRP levels in each media market by its 2000 California population.

Results

Since most of the ads target smokers and non-smokers differently, resulting in different levels of aided recall by both groups, we believed that differences might exist in the relation between aided recall and TRP levels. Additionally, we hypothesised that the change in methodology from telephone to web may have impacted this relation (figure 1). Consequently, we examine each survey mode separately as well as combining both modes.

Figure 1

The relation between percentage-aided recall and cumulative target rating points (TRPs).

We first examined simple correlations between aided advertisement awareness and several measures of TRP including flight TRPs, cumulative TRPs and TRP density (table 4) by smoking status and overall. After looking at the distribution of the cumulative TRP measure with aided advertisement awareness, we determined that several large values influenced observed associations; hence, we also considered log-transformed cumulative TRP values.

Table 4

Correlation of aided recall and various target rating point (TRP) measurements by survey mode

	Smoker-aided awareness	Non-smoker-aided awareness	Overall-aided awareness
Telephone only (n=17)
Flight TRPs	0.05	0.19	0.17
Cumulative TRPs	0.23	0.37	0.35
log(cumulative TRPs)	0.34	0.51*	0.49*
TRP density	0.33	0.31	0.32
Web only (n=12)
Flight TRPs	0.03	−0.02	0.00
Cumulative TRPs	0.46	0.69*	0.67*
log(cumulative TRPs)	0.50	0.76**	0.73**
TRP density	0.22	0.46	0.43
Overall (n=29)
Flight TRPs	−0.05	0.03	0.02
Cumulative TRPs	0.29	0.45*	0.42*
log(cumulative TRPs)	0.41*	0.60**	0.57**
TRP density	0.24	0.31	0.30

p<0.05

p<0.01.

Although cumulative TRPs correlated well with aided recall, the log-transformed cumulative TRP measure exhibited greater correlation across smoking status and survey mode. We observed higher correlations for the web survey than the telephone survey for most of the recall measures. Generally, correlations were higher among respondents who were non-smokers than those who were smokers regardless of survey mode.

As one might expect from the correlation analysis, the coefficient estimates of log-cumulative TRP in the non-smoker GEE models were larger than that in the smoker GEE models (table 5). Additionally, the estimates are larger from the web survey mode. A one-unit increase in log-cumulative TRPs led to a 13.6% increase in aided ad recall in all respondents (smokers and non-smokers) in the web surveys compared to a 5.3% increase in the telephone surveys. However owing to small sample sizes (ie, only 12 ads for the web-based survey and 17 ads for the telephone survey), these results should be considered cautiously.

Table 5

Generalised estimating equation model coefficient estimates and 95% CIs for log cumulative target rating points (TRPs) for smokers, non-smokers and overall by each survey mode

	Smokers	Non-smokers	Overall
Telephone only
Log(cumulative TRPs)	3.5 (−0.5 to 7.6)	5.6 (2.0 to 9.2)	5.3 (1.6 to 9.0)
Web only
Log(cumulative TRPs)	10.9 (2.0 to 19.9)	14.5 (8.1 to 20.9)	13.6 (6.8 to 20.3)
Combined
Log(cumulative TRPs)	4.9 (−0.1 to 9.8)	8.0 (2.6 to 13.4)	7.4 (2.0 to 12.8)

All analyses were additionally conducted with unweighted data—that is, before weighting data to the California population, or using post-stratification weights. Largely, the results were similar to the results presented for the weighted data, except that overall results were skewed depending on the relative proportion of smokers in the sample.

Discussion

Previous studies have suggested that cumulative campaign exposure measurements correlate strongly with television advertisement recall measures better than shorter-term campaign exposure measurements.41 Other researchers have examined relations with campaign exposure density or cumulative campaign exposure divided by the total number of weeks on the air.40 Although Niederdeppe41 questions the relative validity of recall measures owing to low correlations with cumulative GRP measures in a public health setting, our findings suggest modestly higher correlations using cumulative TRPs.

Results from the correlation and GEE analyses suggest that aided recall is best predicted using the log transformation of cumulative TRPs in our data. Comparably, we found that only cumulative TRP levels were more highly associated with aided recall rather than flight TRP levels. This was true even though recall questions included a past 30-day or 60-day time restriction. These results were consistent for the telephone and web-based versions of the survey as well as for smokers and non-smokers. Although survey mode and smoking status appeared to impact our aided recall measure in a consistent way, their effects were not statistically different.

The natural shape of the log transformed data suggests that at some TRP level aided recall will reach a plateau. This finding implies that an advertisement may reach a point of diminishing returns that is commonly described as ‘wearout’. Although this term has been interpreted in a variety of ways, in this study, ‘wearout’ may imply that an ad does not have the capability of reaching new viewers. Put differently, an ad could still reinforce the message for an audience that has already seen the ad; however, the need for reinforcement would depend on whether CTCP would prefer to repeat the specific message in an advertisement.

We found a modest relation between TRP density and aided recall in our analysis. This finding along with our log transformed cumulative TRP results provides an argument for placing an advertisement at lower TRP levels over a longer period as comparable to having high TRP placement over a shorter period. In advertising literature, the practice of pulsing advertisements has been addressed in theoretical work but has focused on S-shaped sales demand or sales response to advertising.56–60 This is an area that requires further investigation because the goals of public health advertising are different from typical product advertising.

Several limitations of this study exist. First, we take a macro-level approach to the data rather than looking at each individual respondent. This approach reduced our sample size to 29 data points. Second, the web-panel data are correlated over time as a portion of respondents who responded to multiple surveys over time. Similarly, data collected within the same wave are correlated because the same respondents answer ad recall questions for multiple ads within the same wave of data collection. To address this concern, we repeated our analyses using the wave of data collection as the repeated measure but found that the results were not meaningfully different. Third, we did not control for ad characteristics such as emotional reaction or theme that may have led to some higher or lower correlation values; hence, our estimates are valuable regardless of ad characteristics. Lastly, aided recall is not the ultimate measure of a media campaign's effectiveness. Still, aided recall is a useful measure that may help the media campaign address questions regarding efficiency of ad placement.

Further research is also needed on differences by mode of survey administration. We observed higher aided awareness using web-based surveys compared to telephone interviews. Partly this may be a consequence of having more time to answer questions online and the ability to use visual rather than auditory pathways to process the information.52

To more fully understand the processes behind the longer-term impact of advertisement awareness on smoking behaviour, we intend to examine the web-based surveys from a longitudinal perspective. Each cross-sectional survey collection has provided the CTCP's media campaign with information about the effectiveness of specific advertisements that have played a part in programmatic decisions about which advertisements to place in future media flights and which types of advertisements to produce in the future. In light of declining funding, findings from this paper may offer the media campaign astute choices regarding future advertisement placement considering the important association of intensity and level of cumulative TRPs with aided ad recall.

A large number of public health researchers have shown that tobacco control media campaigns can decrease smoking initiation in youths, increase motivation among adults to quit and reduce overall smoking prevalence. A smaller number of articles have examined the relation between advertising intensity and advertising recall or awareness, a measure of advertising effectiveness.

This study suggests that cumulative targeted rating points (TRPs) are a valid measure of predicting specific television advertisement awareness. The logarithm of cumulative TRPs provided to the best correlation with ad awareness, regardless of the survey mode. We observed a linear relation between advertising awareness and cumulative TRPs, which tapered off at approximately 1000 cumulative target rating points. The study findings are useful in planning levels of ad placement for future media campaigns.