Are mass-media campaigns effective in preventing drug use? A Cochrane systematic review and meta-analysis.

OBJECTIVE: To determine whether there is evidence that mass-media campaigns can be effective in reducing illicit drug consumption and the intent to consume.
DESIGN: Systematic review of randomised and non-randomised studies.
METHODS: We searched four electronic databases (MEDLINE, EMBASE, ProQuest Dissertations & Theses A&I and CENTRAL) and further explored seven additional resources to obtain both published and unpublished materials. We appraised the quality of included studies using standardised tools. We carried out meta-analyses of randomised controlled trials and a pooled analysis of interrupted time-series and controlled before-and-after studies.
RESULTS: We identified 19 studies comprising 184,811 participants. Pooled analyses and narrative synthesis provided mixed evidence of effectiveness. Eight interventions evaluated with randomised controlled trials leaned towards no evidence of an effect, both on drug use (standardised mean difference (SMD) -0.02; 95% CI -0.15 to 0.12) and the intention to use drugs (SMD -0.07; 95% CI -0.19 to 0.04). Four campaigns provided some evidence of beneficial effects in preventing drug use and two interventions provided evidence of iatrogenic effects.
CONCLUSIONS: Studies were considerably heterogeneous in type of mass-media intervention, outcome measures, underlying theory, comparison groups and design. Such factors can contribute to explaining the observed variability in results. Owing to the risk of adverse effects, caution is needed in disseminating mass-media campaigns tackling drug use. Large studies conducted with appropriate methodology are warranted to consolidate the evidence base. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This systematic review is based on an expanded evidence base of both published and unpublished findings and aims to determine whether mass-media campaigns can be effective in preventing the use of or intention to use illicit drugs.

Pooled analyses of eight mass-media interventions provide no evidence of an effect on drug use or intention to use illicit drugs. Four interventions provide evidence of beneficial effects. Two interventions provide evidence of iatrogenic effects.

Owing to the paucity and inconsistency of available evidence, we cannot draw general conclusions as to whether mass-media interventions are effective in preventing the use of or intention to use illicit drugs.

This review provides an insight into research gaps around the impact of mass-media drug prevention interventions and can serve to highlight that new campaigns should be implemented in the framework of rigorous evaluation studies, in order to avoid dissemination of interventions that are ineffective or have unintended effects.

Introduction

Mass-media campaigns are a powerful means for disseminating health promotion messages. A wide and diverse audience can be reached through television commercials, the Internet, mobile phones, newspapers and roadside advertising hoardings. In the field of drug addiction and dependence, advertisements may contribute to shaping patterns of drug use and the intention to use drugs, as well as modifying mediators such as awareness, knowledge and attitudes about drugs.

However, ethical and economic considerations are often raised. Mass-media campaigns—unlike other health interventions—are imposed on populations that have not consented to their implementation.1 This is a considerable ethical issue in modern, person-centred public health, where taking decisions shared with the public is essential for promoting behaviour change. Second, mass-media campaigns can be very expensive, especially when implemented at the national or state level. Large-scale purchasing of public service announcement time during popular shows and broad dissemination via printed media are often accessible only to governmental institutions. For example, the first and second versions of the US Office of National Drug Control Policy's National Youth Anti-Drug Media Campaign cost 2.7 billion dollars over more than 10 years.2 Although such campaigns underwent careful evaluation, most mass-media interventions are not developed in compliance with the classical circle of public health, which consists in designing interventions based on evidence and in evaluating their impact.

A systematic review of the studies assessing media campaigns aiming to prevent use of illicit drugs can inform future strategies and help design effective campaigns. The objective of this review is to assess the effectiveness of mass-media campaigns in preventing or reducing drug use or the intention to use illicit drugs among young people.

Methods

We conducted this systematic review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)3 statement and with the procedures specified in a previously published protocol.4 As described in detail previously,4 we systematically searched four electronic databases: MEDLINE (1966 to 29 January 2013), EMBASE (1974 to 30 January 2013), ProQuest Dissertations and Theses A&I (1861 to 3 February 2013) and CENTRAL (2013, Issue 1). Search strategies are available as supplementary files (see online supplementary appendix 1). We further explored seven additional resources to obtain both published and unpublished materials: four websites of registered studies (ie, http://www.controlled-trials.com, http://apps.who.int/trialsearch/, http://clinicaltrials.gov/, https://eudract.emea.europa.eu/), references embedded in book chapters, references included in the annual national reports written by EMCDDA national focal points and any publications recommended by prominent researchers in the field. We did not set any constraints, such as language or time, to our search.

Selection criteria

As described in detail previously,4 we considered studies involving participants under the age of 26 and evaluating mass-media campaigns explicitly aimed at influencing the use or intention to use illicit drugs.5 The following were deemed acceptable comparison groups: (1) no intervention; (2) community-based or school-based drug prevention programmes; (3) lower exposure to intervention; (4) time before exposure to intervention. We included randomised controlled trials (RCTs), cohort studies, interrupted time-series (ITS) studies and controlled before and after (CBA) studies providing evidence on drug use or intention not to use, to reduce use or to stop use of illicit drugs.

Two authors independently inspected search hits by reading titles and abstracts and assessed studies for inclusion. Any disagreement was solved by consensus. Multiple publications pertaining to the same study were collated as one single study.

Quality Appraisal

Four authors independently performed quality assessments, and any disagreement was solved by consensus. We contacted study authors whenever information was missing or unclear.

We used standardised assessment tools for each study design—details are available as supplementary materials (see online supplementary appendix 2). For RCTs, we used the Cochrane Collaboration Risk of Bias assessment tool.6 For cohort studies, we followed the Scottish Intercollegiate Guidelines Network (SIGN) Quality Criteria.7 For ITS and CBA studies, we used the tool recommended by the Cochrane Effective Practice and Organisation of Care Group.8

Statistical analysis

As for RCTs, we performed a random effects meta-analysis to estimate the pooled effect of mass-media interventions on drug use while accounting for between-study heterogeneity, as described in detail previously.4 We carried out a fixed effects meta-analysis to estimate the pooled intervention effect on intention to use drugs. We tested between-study heterogeneity using the χ2 test and the I2 statistic. A p value lower than 0.10 in the χ2 test and an I2 statistic higher than 50% suggested evidence of heterogeneity. Since most studies assessed their outcome variables with different scales, we used standardised mean difference (SMD) as the summary measure of choice. SMDs were used for both drug use and intention to use drugs and were calculated by dividing the difference in mean outcome between groups by the SD of outcome between participants.6 SMDs and their SEs were then pooled in a meta-analysis performed with RevMan.9 For two clustered RCTs,10 11 we inflated SEs to account for within-cluster correlations.6

We pooled the effect estimates of the Meth Project studies using mixed effects logistic regression.12–16 An ITS design was applied for estimating the differences in prevalence of methamphetamine use before and after the Meth Project intervention, adjusting for any underlying temporal trend. We fitted the following model: logit(useij)=β0+u0j+β1timei+β2agei+β3intervi+β4age×intervi, in which use was prevalence of methamphetamine use, time was a continuous variable, age and intervention were two-level categorical variables, u0j was a random intercept and we allowed log odds of methamphetamine use to vary randomly by each jth state.17 The relatively few data points did not allow exploration of more complex models, for example, the temporal trend could not be assumed to vary randomly across states.

Results

Out of 18 343 titles and abstracts, we selected 24 papers corresponding to 19 individual studies (figure 1).

Figure 1

PRISMA flow diagram. Adapted from a previous publication.4

Study Characteristics

Overall, 184 811 participants were included, with most studies comprising participants who were aged between 10 and 19 years (table 1). Although most studies included both boys and girls, two studies focused on girls.18 19 One study considered Asian-Americans as the only ethnic group eligible for inclusion,18 while the other study did not focus on any specific ethnic groups. Seventeen studies were conducted in the USA, one in the USA and Canada,19 and one in Australia.11

Table 1

Characteristics of included studies

Study	Underpinning theory	Design (goal)	Intervention	Comparison	Primary outcome	Secondary outcome(s)	Analysis sample	Follow-up (total time) (months)
Polansky et al 199925	Decision theory	RCT (E)	PSA	Other	–	Intention to use; attitudes; knowledge and disposition to select socially appropriate responses	312	NA (NS)
Miller et al 200020	Self-regulation theory	CBA (F)	PSA; printed	No intervention	Use of drugs (incl. cannabis and cocaine)	Risk perception; problems related to drug use	1024	12 (18)
Palmgreen et al 200124	Influence of sensation-seeking on drug use	ITS (F)	PSA	No intervention	Past 30-day use of marijuana	–	6371	NA (32)
Yzer et al 200326	Theories of behavioural change: persuasion effects	RCT (E)	PSA	No intervention; other	–	Intention to use marijuana; attitude; perceptions about marijuana	418	NA (NS)
Slater et al 200610	Social-ecological framework (norms and expectations influence drug use)	RCT (E)	PSA; printed	No intervention	Lifetime and past 30-day use of marijuana	–	4216	24 (42)
Zhao et al 200627	Normative beliefs	RCT (E)	PSA	No intervention	–	Intention to use; beliefs towards marijuana; social norms	435	NS (NS)
Hornik 200622	Unclear	Cohort (F)	PSA; printed; internet	Lower exposure	Lifetime, past year, and past 30-day use of marijuana	Intention to use; attitudes and self-efficacy; perceptions and social norms	8117	56 (58)
Scheier and Grenard 201023	Social marketing	Cohort (F)	PSA; printed; internet	Lower exposure	Past 12-month cannabis intoxications	–	2515	NA (48)
Schwinn et al 201019	Social learning theory	RCT (E)	Internet	No intervention	Past 30-day substance use	–	236	6 (NS)
Lee et al 201028	Readiness to change	RCT (E)	Internet	No intervention	Past 90-day use of marijuana	Intention to change marijuana use; consequences	341	6 (NS)
Fang et al 201018	Family-oriented	RCT (E)	Internet	No intervention	Past 30-day use of marijuana	Intention to use marijuana	216	6.25 (16)
Newton et al 201011	Social influence approach	RCT (E)	Internet	Other	Use of cannabis	Cannabis knowledge; attitudes; related harms	724	12 (21)
Meth Project studies12–16	Perception of risk and perception of social disapproval are correlated with drug consumption	4 ITS and 1 CBA (F)	PSA; printed; internet	No intervention	Past 30-day use of methamphetamine	Attitudes on methamphetamine and other drugs; perceptions; information sources and advertising awareness;	26 405	NA(Colorado 26; Georgia 18; Hawaii 25;Idaho 40; Wyoming 34)
Slater et al 201121	Autonomy and aspiration perceptions as mediators of marijuana use	RCT (E); Cohort (F)	PSA; printed	Lower exposure	Lifetime, past 90-day and past 30-day use of marijuana	Autonomy and aspiration inconsistent with marijuana use	3236	24 (42)
Carpenter and Pechmann 20112	Unclear; evaluated many heterogeneous mass-media campaigns	ITS (F)	PSA; printed; internet	Lower exposure	Past 30-day and lifetime use of marijuana	–	130 245	NA (36)

CBA, controlled before and after; Cohort, prospective cohort; E, experimental/efficacy setting; F, field/effectiveness setting; ITS, interrupted time-series; Lower exposure, lower exposure to same intervention; NA, not applicable; NS, not specified; Other, other intervention or different combination of same intervention; PSA, public service announcement (eg, television/radio); RCT, randomised controlled trial.

Eleven studies (58%) evaluated multicomponent interventions, 3 regarding radio/television and printed advertising,10 20 21 and 8 regarding radio and television commercials, printed advertisements and Internet advertising.2 12–16 22 23 Eight studies evaluated standalone interventions, four consisting in radio and television commercials,24–27 and four in Internet-based interventions.11 18 19 28 The included studies in this review were grounded in a wide range of underlying theories (table 1).

Comparison groups varied considerably across studies. For thirteen studies (68%), the comparison group consisted in no exposure to any intervention. Four studies compared high exposure versus low exposure to the same mass media intervention.2 21–23 For one study, the comparison group consisted in the standard drug education curriculum.11 One study had four study arms consisting either in another intervention or no intervention.26

Eight studies were conducted in an experimental setting by explicitly inviting participants and these studies were randomised controlled trials (RCTs).10 11 18 19 25–28 Ten studies were conducted in a field setting without explicitly inviting participants, as would usually happen with most mass-media campaigns. Of them, 2 were cohort studies,22 23 6 were ITS2 12 14–16 24 and 2 were CBA studies.13 20 One study had a double design as it was conducted in an experimental setting with an RCT design, and in a field setting with a cohort design.21 When specified, follow-up varied from 6 months19 28 to 4.7 years.22

Study quality

On the whole, the quality of the RCTs is acceptable (table 2). As described in detail previously,4 the strongest domain appears to be the risk of attrition bias and the weakest domain the risk of selection bias (unclear description of the randomisation procedure). In one paper, findings of secondary outcomes were reported only as a predictor of the primary outcome, and the paper concerned was deemed at high risk for reporting bias.19

Table 2

Risk of bias of included studies

All cohort studies focused on a clear and appropriate question. Subgroup comparisons between participants and dropouts were carried out in only one study.23 The same study, however, failed to control for potential confounders.

The proportion of participants with no missing data was reported in only one controlled CBA study.20 Potential confounders were accounted for in only one ITS study2 and one CBA study.20 A formal test of trend was not performed in the five Meth Project studies.12–16 One ITS study2 and the two CBA studies13 20 had three or less data points, which are generally considered insufficient for drawing reliable conclusions with regard to intervention effectiveness.

Effects of mass-media campaigns

Use of illicit drugs

Experimental studies

Pooled analyses of five RCTs10 11 18 19 28 comprising n=5470 subjects showed no evidence (p=0.79) of an effect of mass-media campaigns in modifying use of illicit drugs (standardised mean difference (SMD) −0.02; 95% CI −0.15 to 0.12; figure 2 and table 3). There was some evidence (p=0.020) of heterogeneity between studies.

Figure 2

Pooling of randomised controlled trials. Adapted from a previous publication.4

Table 3

Main findings for use or intention to use illicit drugs

Pooling	Outcome	Design	References	Subgroups	Number of subjects exp vs ctrl†	Effectmeasure	Effect size (95% CI)or effect direction (p-value)‡	Heterogeneity p-value¶
Pooled analyses	Use of illicit drugs	RCT	Slater et al 2006;10 Fang et al 2010;18 Newton et al 2010;11 Schwinn et al 2010;19 Lee et al 201028	–	2701 vs 2769	SMD, random effects	−0.02 (−0.15 to 0.12)	0.020*
	Intention to use illicit drugs	RCT	Polansky et al 1999;25 Yzer et al 2003;26 Zhao et al 2006;27 Fang et al 201018	–	771 vs 499	SMD, fixed effects	−0.07 (−0.19 to 0.04)	0.840
	Past-month use of methamphetamine	4 ITS and 1 CBA	Meth Project studies12–16	age 12–17	14 865 vs 7497	OR, random effects	1.16 (0.83 to 1.61)	–
				age 18–24	347 vs 632	OR, random effects	1.63 (0.70 to 3.79)	–
	Past-year use of methamphetamine	4 ITS and 1 CBA	Meth Project studies12–16	age 12–17	17 105 vs 7497	OR, random effects	0.59 (0.43 to 0.81)**	–
				age 18–24	1039 vs 632	OR, random effects	0.70 (0.34 to 1.45)	–
Single studies	Lifetime, past 90-day, or past-30-day use of marijuana	RCT (community-media)	Slater et al 201121	–	NA (3236)	OR, random effects	0.60 (0.38 to 0.94)*	–
		Cohort (mass-media)		–		OR, random effects	0.26 (0.19 to 0.35)***	–
	Past-year use of marijuana	Cohort	Hornik 200622	–	NA (3529)	OR, fixed effects	1.21 (1.19 to 1.65)*	–
	Intention to use marijuana			–	NA (2915)	OR, fixed effects	0.89 (0.79 to 1.00)§	–
	Past 12-month episodes of cannabis intoxication	Cohort	Scheier and Grenard et al 201023	age 13–14	NA (2515)	mean difference, SEM	−0.022	–
				age 15–18		mean difference, SEM	0.144*	–
	Past 30-day use of marijuana among high-sensation seekers	ITS	Palmgreen et al 200124	Fayette	NA (3174)	test for slope	↓ (p=0.001)	–
				Knox, first campaign	NA (3197)	test for slope	↓ (p=0.001)
				Knox, second campaign		test for slope	↓ (p=0.002)	–
	Past 30-day use of marijuana (girls, 8th grade)	ITS	Carpenter and Pechmann 20112	–	NA (130 245)	OR, fixed effects	0.67 (0.52 to 0.87)**	–
	Frequency of use of 10 types of drugs	CBA	Miller et al 200020	–	567 vs 431	mean difference, ANOVA	for LSD: ↑ (p<0.001)for marijuana, cocaine, amphetamine, and heroin: ‘no longer significant’ differences	–

§p<0.10 *p<0.05 **p<0.01 ***p<0.001.

†NA=breakdowns of students exposed to the interventions were not available. Number of analysed subjects is between brackets.

‡Whenever the effect size was not reported, ↓=decreased use or intention to use, and ↑=increased use or intention to use.

¶Heterogeneity test for meta-analyses of RCTs.

ANOVA, analysis of variance; CBA, controlled before and after; Cohort, prospective cohort; ITS, interrupted time-series; RCT, randomised controlled trial; SEM, structural equation modelling; SMD, standardised mean difference.

The RCT part of a mixed RCT-cohort study (n=3236) found evidence of effectiveness (p=0.026) for a media-community intervention (OR 0.60; 95% CI 0.38 to 0.94; table 3).21

Field studies

Two studies found that the Office of National Drug Control Policy (ONDCP) National Youth Anti-Drug Media Campaign (first version) increased use of illicit drugs among adolescents (table 3). One study (n=3529) reported a significant increase in past year-use of marijuana (OR 1.21; 95% CI 1.19 to 1.65).22 The other study (n=2515) found some evidence (‘p<0.05’) of an iatrogenic effect among those aged 15–18 (mean change=0.144), while there was no evidence (‘p>0.05’) of an effect among those aged 13–14 (mean change=−0.022).23

The revamped version of the same ONDCP campaign, Above the Influence, was found effective in a mixed RCT-cohort study (n=3236), whose cohort part found strong evidence (p<0.001) of effectiveness (OR 0.26; 95% CI 0.19 to 0.35).21 On a similar note were the findings of an ITS study (n=130 245) which evaluated Above the Influence and found evidence of reductions in marijuana use in the past month (OR 0.67; 95% CI 0.52 to 0.87) among eighth-grade girls.2

The pooled findings of the five Meth Project studies (n=26 273) suggested no evidence of a change in past-month use of methamphetamine among subjects aged 12–17 (OR 1.16; 95% CI 0.83 to 1.61), nor among those aged 18–24 (OR 1.63; 95% CI 0.70 to 3.79) (figure 3A and table 3). There was, however, evidence (p=0.001) of a reduction in past-year use of methamphetamine among those aged 12–17 (OR 0.59; 95% CI 0.43 to 0.81), while there was no evidence of a similar effect among those aged 18–24 (OR 0.70; 95% CI 0.34 to 1.45; figure 3B and table 3).

Figure 3

Pooling of the meth project interrupted time-series studies: predicted and observed probabilities. Adapted from a previous publication.4

One ITS (n=6371) showed evidence of effectiveness for past 30-day use of marijuana among high sensation seekers (p=0.001 for the Fayette sample, p=0.001 for the first campaign in the Knox sample, p=0.002 for the second campaign in the Knox sample).24

One CBA study found an increase in use of LSD (‘p<0.001’; table 3) while no evidence (‘p>0.05’) of differences was found for marijuana, cocaine, amphetamine and heroin.20

Intention to use drugs

In one meta-analysis of four randomised controlled studies involving 1270 participants, there was no evidence of an effect (p=0.21) of media campaigns in changing intention to use drugs (SMD −0.07; 95% CI −0.19 to 0.04; figure 2 and table 3).18 25–27 There was no evidence (p=0.840) of heterogeneity across studies.

One study (n=2915) found some evidence (p=0.053) of a reduction in intentions to use marijuana (OR 0.89; 95% CI 0.79 to 1.00; table 3) for the first version of the ONDCP's media campaign.22

Discussion

Mass-media campaigns are commonly used throughout the world to tackle a broad array of preventable risk factors or injuries. Such campaigns are seldom evaluated, thus making it difficult to inform policymakers regarding their effectiveness and sustainability. In this panorama of overall uncertainty, mass-media campaigns tackling tobacco and traffic accidents are noteworthy exceptions as they have been evaluated more frequently and have shown some evidence for benefit.29 In our attempt to summarise evidence on the effectiveness of mass-media campaigns targeting illicit drugs, we included 19 studies evaluating a number of heterogeneous interventions. We grouped interventions according to whether they were evaluated with studies conducted in experimental settings in which participants were aware of being exposed to media interventions, or were assessed with studies carried out in a field environment which are more likely to show the real-life effects of large national media campaigns, but are also more prone to risk of bias.

Findings appear to vary considerably according to the type of intervention and study design. Pooled analyses of eight interventions evaluated in an experimental setting provided no evidence of beneficial effects for use or intention to use illicit drugs, an indicator of possible future behaviour.30 31 Four interventions evaluated with eight field studies revealed some evidence of beneficial effects: (1) the revamped campaign by the Office of National Drug Control Policy (ONDCP) called Above the Influence, which was found effective in one study and effective among eighth-grade girls in another study; (2) the Be Under Your Own Influence media-community intervention; (3) the Meth Project campaign, which was found effective on past-year methamphetamine use, although only among adolescents aged 12–17 years; and (4) the US televised antimarijuana campaigns broadcast in Fayette County (Lexington), Kentucky and in Knox County (Knoxville), Tennessee, which were found to be effective on high-sensation seekers. Two mass media campaigns showed clear iatrogenic effects, most notably, the first version of the ONDCP's media campaign My Anti Drug, which was evaluated by two studies and was found to increase use of marijuana. An adverse effect was also found for a media-community intervention evaluated by a CBA study which provided evidence of increased frequency of LSD use.

No characteristic emerged clearly as a core feature of successful or unsuccessful campaigns, either regarding their explicit or implicit theoretical background or their communication strategies. However, it is worth noting that two out of the four interventions providing evidence of effectiveness, the ONDCP's Above the Influence national campaign and the Be Under Your Own Influence media-community intervention promoted non-use of drugs as a way to support the goals of autonomy and achievement of competence, both of which have been conceptualised as innate psychological needs that persist over the lifespan.21 Among the interventions which provided evidence of harmful effects, the first version of the ONDCP's media campaign My Anti Drug was based on a social marketing approach which emphasised resistance skills, self-efficacy, normative education and negative consequences of drug use.32 These mediators are suspected to have increased the perception of prevalence of drug use in the target population.33

An important reason for the weak evidence obtained by this review is the large variation in mass-media intervention type and study design. Similar interventions were often evaluated with different study designs while different interventions were sometimes evaluated with the same study design. Pooled analyses could thus be undertaken only for a few similar interventions evaluated with the same study design, and such small sets of pooled studies did not allow sensitivity analyses to be carried out. We did not set any time or language constraints to our search, accepted all types of controlled study designs and obtained unpublished data by establishing direct contact with the authors of the original papers. Unfortunately, owing to the paucity and inconsistency of available evidence, we cannot draw general conclusions as to whether media campaigns are effective in preventing the use or the intention to use illicit drugs. This observation is in line with the findings of similar reviews that used more restrictive inclusion criteria.29 34

The evidence base accrued so far on media campaigns targeting illicit drugs allows us to make at least two remarks. First, such campaigns can be evaluated—a fact that is often questioned in several parts of the world—and properly conducted evaluation studies can provide benefits to both research and practice. Second, in the worst-case scenario, media campaigns can be both ineffective and harmful. Contrary to common belief, antidrug media campaigns may be damaging and their dissemination is ethically unacceptable without a prior assessment of their effects.35 36 New campaigns should be implemented in the framework of rigorous evaluation studies, ideally in field settings with cohort or ITS study designs. A better understanding of which media interventions work best is likely to result in a more effective prevention of drug use and increased efficiency in the management of public resources.