Quality of advertisements for prescription drugs in family practice medical journals published in Australia, Canada and the USA with different regulatory controls: a cross-sectional study.

OBJECTIVE: To assess if different forms of regulation lead to differences in the quality of journal advertisements.
DESIGN: Cross-sectional study. PARTICIPANTS: Thirty advertisements from family practice journals published from 2013 to 2015 were extracted for three countries with distinct regulatory pharmaceutical promotion systems: Australia, Canada and the USA. PRIMARY AND SECONDARY OUTCOME MEASURES: Advertisements under each regulatory system were compared concerning three domains: information included in the advertisement, references to scientific evidence and pictorial appeals and portrayals. An overall ranking for advertisement quality among countries was determined using the first two domains as the information assessed has been associated with more appropriate prescribing.
RESULTS: Advertisements varied significantly for number of claims with quantitative benefit (Australia: 0.0 (0.0-3.0); Canada: 0.0 (0.0-5.0); USA: 1.0 (0.0-6.0); p=0.01); statistical method used in reporting benefit (relative risk reduction, absolute risk reduction and number needed to treat; Australia: 6.7%, n=2; Canada: 10.0%, n=3; USA: 36.6%, n=11; p=0.02); mention of adverse effects, warnings or contraindications (Australia: 13.3%, n=4; Canada: 23.3%, n=7; USA: 53.3%, n=16; p=0.002); equal prominence between safety and benefit information (Australia: 25.0%, n=1; Canada: 28.6%, n=2; USA: 75.0%, n=12; p=0.04); and methodological quality of references score (Australia: 0.4150 (0.25-0.70); Canada: 0.25 (0.00-0.63); USA: 0.25 (0.00-0.75); p<0.001). The USA ranked first, Canada second and Australia third for overall quality of journal advertisements. Significant differences for humour appeals (Australia: 3.3%, n=1; Canada: 13.3%, n=4; USA: 26.7%, n=8; p=0.04), positive emotional appeals (Australia: 26.7%, n=8; Canada: 60.0%, n=18; USA: 50.0%, n=15; p=0.03), social approval portrayals (Australia: 0.0%, n=0; Canada: 0.0%, n=0; USA: 10.0%, n=3; p=0.04) and lifestyle or work portrayals (Australia: 43.3%, n=13; Canada: 50.0%, n=15; USA: 76.7%, n=23; p=0.02) were found among countries.
CONCLUSIONS: Different regulatory systems influence journal advertisement quality concerning all measured domains. However, differences may also be attributed to other regulatory, legal, cultural or health system factors unique to each country. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The information assessed from ads is associated with more appropriate prescribing.

All information was abstracted by two independent authors and disagreements were resolved through consensus or a third author if consensus could not be reached.

The accuracy of information in ads was not assessed.

The effect of ads on prescribing was not assessed.

Other regulatory, legal, cultural or health system factors unique to each country were not controlled for which may also account for differences in the quality of advertisements.

Introduction

Journal advertising in medical journals is a ubiquitous form of drug promotion, although it only represents a small fraction of total promotional spending. Figures for the USA from 2012 show that medical journal advertising cost companies US$90 million out of a total promotional budget of US$27 billion (0.3%).1 The bulk of the budget, US$15 billion, is primarily dedicated to detailing efforts. Canadian data for 2016 are equally skewed in favour of detailing over journal advertising—US$408.9 million for the former compared with US$12.5 million for the latter.2

However, according to a study published in Medical Marketing & Media ‘advertising magnifies the detailing effort at a fraction of detailing expense. In effect, detailing provides the power in the marketing effort and advertising provides the efficiencies’.3 For every dollar spent on medical journal advertisements during the first 4 years, drugs are on the market in the USA, the return on investment (ROI) was US$2.43; after this time, ROI increased to over US$4.00. In addition, advertising magnifies the effects of detailing, increasing the ROI from detailing 75% of the time by 30%–40%.3 Neslin claimed that journal advertising generated the highest ROI of all promotional strategies, ranging from US$2.22 to US$6.86 per advertising dollar spent.4

Journal advertisements are directly influenced by the standards and approaches to regulation in the jurisdiction in which they appear; however, it is unclear how this affects the quality of advertisements. One previous study examined journal advertisements in different countries and concluded that the quality of advertisements, as measured by six characteristics including the relative frequency and size of the generic and trade names and the amount of space allocated to indications and safety information, was affected by the method of regulation; however, it analysed only advertisements published between 1961 and 1977.5 More recent literature has compared drug advertisements in different countries but did not explicitly assess approaches to regulation.6 7 Given that drug promotion has an established effect on physician prescribing practices,8 it is essential to examine how current regulations affect the quality of journal advertisements.

Three methods of regulating medical journal advertising have evolved in developed countries: direct government control (eg, the Food and Drug Administration (FDA) in the USA),9 industry self-regulation (eg, in Australia and New Zealand)10 and regulation by a multistakeholder body (eg, the Pharmaceutical Advertising Advisory Board in Canada, table 1).11 Of note, in Australia, the industry code must be approved by the Australian Competition and Consumer Commission. Despite differences in details in the requirements in the regulations in each country, the overall goals in each country with respect to how advertisements should portray the benefits and harms of the medicines are broadly similar:

Table 1

Forms of promotional regulation in Australia, Canada and the USA

Country	Regulatory body	Composition of body	Compliance with regulation voluntary or mandatory	Code development	Prescreening of advertisements before publication	Active monitoring of compliance or complaints driven	Monitoring body
Australia	Medicines Australia	Representatives from industry association members	Mandatory for members of Medicines Australia	Panel appointed by Medicines Australia, consultations from defined list of groups, public announcement of and advertising of code review Code must be approved by Australian Competition and Consumer Commission	No	Complaints	Chair (consultant with industry experience in marketing) Representatives of Royal Australian College of General Practitioners, Australian Medical Association, Consumers Health Forum of Australia, College and/or Society associated with therapeutic class of product being reviewed, up to two representatives from Medicines Australia members
Canada	PAAB	Representatives from: medical advertising agencies, medical publishers, research-based industry, generic industry, over-the-counter industry, pharmacists association, medical associations, consumer associations	Members of IMC (representing research-based companies) agree to abide by code as condition for membership in IMC	Not stated	Yes	Complaints	Commissioner of PAAB
USA	Office of Prescription Drug Promotion, FDA	Government employees	Mandatory	As per other US government federal regulations	Only in cases where the FDA may require preapproval of promotional materials as part of an enforcement action; otherwise material submitted at time of publication	Active but not all material can be reviewed due to resource restrictions	Office of Prescription Drug Promotion (FDA)

FDA, Food and Drug Administration; IMC, Innovative Medicines Canada; PAAB, Pharmaceutical Advertising Advisory Board.

Australia: ‘The content of all promotional material provided to healthcare professionals must be current, accurate, balanced and fully supported by the Australian Approved Product Information’.10

Canada: ‘PAAB ensures that any information provided about a product is evidence-based and that there is a balance between claims about benefits and possible risks’.11

USA: ‘Product claim ads must present the benefits and risks of a prescription drug in a balanced fashion’.9

The objective of this study is to examine the quality of advertisements in Australia, Canada and the USA to determine if different forms of regulation lead to differences in the quality of the advertisements. Based on previous literature describing the failure of voluntary industry regulation,12 13 our a priori assumption was that advertisements produced under a self-regulatory system (Australia) will be of inferior quality compared with ads produced under the other two systems (Canada and the USA).

Methods

This was a cross-sectional study of medical journal advertisements from Australia, Canada and the USA.

Selection criteria and method of choosing ads

We applied selection criteria for ads for prescription medicines that controlled for as much variability as possible, aside from the type of regulatory control that they are subject to. Table 2 lists the inclusion criteria. We selected ads with both text and images from the same type of journal, targeted at the same audience and published in the same years. Ads came from family practice journals (American Family Physician, Australian Family Physician and Canadian Family Physician) from 2014 to 2015. Family practice journals generally have a greater number of ads and advertise a wider range of drugs compared with specialty journals. Of the ads that met inclusion criteria, we used a random number generator to select 15 ads from each journal in each of the 2 years. Journals were accessed through the library system at the University of Toronto. Ads were scanned, and the electronic versions were used for evaluation.

Table 2

Inclusion criteria for advertisements

Criteria	Rationale
Family practice journals	Advertisements directed to same audience and same type of journals.
Published in same year	Minimises differences in knowledge about product.
Promoted within Australia, Canada or the USA	Standardises the setting to English-speaking developed countries with similar medical practices.
Advertising information must include text and pictorial component	To assess the ads holistically based on textual and visual depictions.
Prescription-only products	In Canada, ads for over-the-counter products are not subject to the same guidelines as ads for prescription-only products. Therefore, to achieve consistency, we restricted our sample to products that were prescription-only in all three countries.
Full advertisements	Reminder ads only give the name of the medication and do not make any claims or provide any safety information.

Evaluation components of ads

For each ad, we recorded the country where it appeared, year, brand and generic name of the drug, manufacturer and the number of pages in the journal that the ad occupied. We recorded the therapeutic category for each drug by using the WHO Anatomic Therapeutic Chemical (ATC)/Defined Daily Dosage Index at the second level (https://www.whocc.no/atc_ddd_index/) to examine whether the drugs being advertised were for a broad range of conditions.

Ads typically consisted of three components—advertising copy, prescribing information and visual messages. Advertising copy was distinguished from prescribing information based on the following criteria: no colour used in the prescribing information (eg, black print on white background/white print on a black background); the clear visual distinction between the advertising copy and prescribing information; no claims made in prescribing information; and the use of different fonts. Only the advertising copy and the visual messages were evaluated.

Our scoring system assessed three main quality domains: (1) information included in the advertisement, (2) references to scientific evidence and (3) advertising appeals and portrayals. The first domain included criteria that assessed whether generic drug names were given the same prominence (ie, mentioned as frequently) as brand names because the use of generic names is associated with more appropriate prescribing.14–16 If the ad made one or more quantitative claims about benefits then, if possible, based on the information in the ad, we assessed whether the claim was in the form of a relative risk reduction (RRR), absolute risk reduction (ARR) or number needed to treat (NNT). Specific mention of ARR and NNT have been shown to lead to more conservative prescribing.17–20 We examined the main claim(s), that is, the one(s) in the largest font to see if they referred to clinically relevant or non-clinically relevant features of the drug. Mention of clinical benefit was considered to be more important than the mention of a surrogate benefit since the latter is not necessarily predictive of a clinical benefit21 and because surrogate outcomes are likely to exaggerate treatment benefits as compared with patient-relevant clinical outcomes.22 Clinical outcomes were defined as ‘a characteristic or variable that reflects how a patient (or consumer) feels, functions or survives’ whereas surrogate endpoints were expected to predict clinical benefit (or harm or lack of benefit or harm) based on epidemiological, therapeutic, pathophysiological or other scientific evidence.23

Other types of claims (eg, on convenience, listing in a guideline, popularity of the product and mechanism of action) were considered to be less relevant to appropriate prescribing. Finally, mention of harm was assessed as physicians must be able to assess the benefit-to-harm ratio to prescribe appropriately. Specifically, we looked at whether the ad gave the same prominence to benefits and harms in terms of font size and position of the information. If more than one claim or harm was mentioned or more than one statement about safety information was provided, each one was evaluated separately.

The second domain included criteria that assessed the methodological quality of all of the references used to support claims made in the advertisement and the degree to which the reference supported the statement in the ad (assessed by reading only the abstract). Peer-reviewed journals are generally considered to publish higher-quality material than non-peer reviewed journals or other types of publications. The rating scales used for the methodological quality of the references and their support for claims came from the study by Lexchin and Holbrook.24 Reliance on observational data to evaluate drug efficacy is highly problematic,25 and the bias is, on average, larger than the estimated effect.26 Furthermore, there are many recent examples where observational studies that suggested a treatment benefit were overturned by randomised controlled trials (RCTs).27 Although there has not been any research into whether the strength of the link between the reference and the claims leads to more appropriate prescribing, it seems logical to assume that a stronger link would be beneficial in improving the reliability of the information.

The third domain included criteria that assessed different appeals and portrayals used by ads to market the product, and by doing so, provide prescribers different impressions regarding the value of the drug. The criteria used—the type of appeal, lifestyle or work portrayal, condition portrayal, the portrayal of effects of product use, product portrayal—were adapted from a study of direct-to-consumer television ads.28 Scott and colleagues have argued that drug ads ‘use images to construct mythical and potentially misleading associations between diseases and products’.29 In particular, drug advertising for psychiatric conditions can replicate and construct stereotypes about mental disabilities,30 especially in the case of women and the elderly. We counted the per cent of ads in each country using each of the different categories of appeals or portrayals.

Online supplementary file 1 outlines in detail the scoring system used for the quality assessment of advertisements. The overall quality of drug advertisements was measured by summing the ranking of selected criteria. Only criteria from the first two domains which revealed significant differences between countries were chosen. The first two domains were selected because they could be objectively measured, whereas the evaluation of the appeals and portrayals involved a subjective element.

Scoring of ads

The initial scoring system was developed based on the results of a systematic review of the quality of journal ads.31 The scoring system was then refined through independent pilot testing by two authors (DD and AMS) with a review by the third author (JL) using 10 ads that were not included in the main study. Subsequently, two independent assessors (DD and AMS) used the scoring system to assess all the ads. Disagreements were solved by consensus or a third author (JL), if consensus could not be reached. The third author (JL) also evaluated the first 10 ads and every subsequent third ad to ensure consistency in coding.

Data analysis

Criteria were scored in one of two ways; some on a yes/no basis and in other cases we computed the percent of the total possible maximum score (eg, if the maximum score was 4 and the particular criterion for that ad was scored as 1 then we recorded a score of 0.25 (1/4)). If an ad had two claims, then the score for each claim was computed separately and then the scores were summed and the mean was calculated and reported. Then we performed two different quantitative analyses:

We compared scores for each criterion for the 30 ads for each country. Nominal data (yes or no) were presented as counts and percentages and compared with the χ2 test. Post-hoc analyses using adjusted residuals with Bonferroni corrections were done for all significant tests. For numerical data, Shapiro-Wilk tests were first used to assess normality. Our data were not normally distributed; hence non-parametric Kruskal-Wallis mean rank comparisons were used.32 Results were presented as medians and ranges. Post-hoc pair-wise comparisons with Bonferroni corrections were made for all significant tests.

In the absence of any validated research about whether any of the 10 criteria were more important in terms of influencing prescribing, we weighted all the criteria equally and ranked the countries from 1 (best score) to 3 (worst score) for each criterion. Ranks for each criterion were then summed, where the total rank was obtained to draw comparisons regarding the overall quality of ads per country. Lower total scores represented a better quality of journal drug advertising in the respective country.

Statistical calculations were done using IBM SPSS V.25.0. A two-sided alpha level of 0.05 was set for significance.

Patients and public involvement

No patients were involved in this study. There was no public involvement in this study.

Results

A total of 30 ads were included from each country. Only 14 unique ads were available from the American Family Physician for 2014, and therefore one ad from 2013 was used. AstraZeneca was the most common manufacturer for Australian ads (13.3%, n=4); Novartis Pharmaceuticals (13.3%, n=4) for Canadian ads; and Boehringer Ingelheim (20.0%, n=6) for USA ads. The mean total number of pages for the advertising copy of the ad was 1.15 (SD ±0.30) for Australia, 1.22 (SD ±0.34) for Canada and 2.18 (SD ±0.87) for the USA. For Australia, Canada and the USA, drugs came from 12, 15 and 16 different second level ATC groups, respectively. For Australia and Canada, the most common therapeutic group was Drugs for Obstructive Airway Diseases (7/30 ads in both); for the USA it was Drugs Used in Diabetes (7/30). Online supplementary file 2 lists the included advertisements.

Information included in the advertisement

There was a statistically significant difference in the number of claims with quantitative benefit among the different countries: Australia 0 (0–3), Canada 0 (0–5), USA 1 (0–6), x2=8.761, p=0.01, with a mean rank of 37.6 for Australia, 43.9 for Canada and 55.0 for the USA. Post-hoc analysis revealed a difference in claims between Australia with a median of 0.0 (0.0–3.0) compared with the USA, with a median of 1.0 (0.0–6.0; p=0.01).

Differences were observed among countries concerning the reporting of RRR, ARR and NNT. RRR was most frequently reported by the USA (33.3%, n=10), followed by Canada (10.0%, n=3) and Australia (6.7%, n=2; p=0.02). Only one US ad provided sufficient information to calculate ARR or NNT.

Information on adverse effects, warnings or contraindications were most frequently reported by the USA (53.3%, n=16), then Canada (23.3%, n=7) and Australia (13.3%, n=4; p=0.002). Similarly, if safety information was given it had the same prominence as benefits information most frequently in the USA (75%, n=12), then Canada (28.6%, n=2) and Australia (25.0%, n=1; p=0.04). There were no statistically significant differences among countries with respect to how often generic names were mentioned compared with brand name mentions, presence of claims of clinical benefit or harm and how close each claim was to a clinically relevant drug characteristic. See table 3 for an overview of the information elements in the advertisements.

Table 3

Information included in advertisement

Criterion	Outcome	Countries
		Australia (n=30)	Canada (n=30)	USA (n=30)	P value
Is generic name mentioned every time brand name mentioned?	Yes	11 (36.7)	5 (16.7)	4 (13.3)	0.06
	No	19 (63.3)	25 (83.3)	26 (86.7)
Are there claims of clinical benefit or harm?	Yes	22 (73.3)	23 (76.7)	26 (86.7)	0.42
	No	8 (26.7)	7 (23.3)	4 (13.3)
Number of claims per ad with quantitative information about benefit	Median (range)	0.0 (0.0–3.0)	0.0 (0.0–5.0)	1.0 (0.0–6.0)	0.01*
Are RRR, ARR or NNT reported or can ARR or NNT be calculated?	No reporting	28 (93.3)	27 (90.0)	19 (63.3)	0.02†‡
	RRR reported	2 (6.7)	3 (10.0)	10 (33.3)
	ARR or NNT reported or can be calculated	0 (0.0)	0 (0.0)	1 (3.3)
Is information provided on one or more adverse effects, warnings or contraindications within the advertising copy?	Yes	4 (13.3)	7 (23.3)	16 (53.3)	0.002¶§
	No	26 (86.7)	23 (76.7)	14 (46.7)
If safety information is provided, is this information given the same prominence as benefit information, as measured by font size?	Yes	1 (25.0)	2 (28.6)	12 (75.0)	0.04
	No	3 (75.0)	5 (71.4)	4 (25.0)
Is the main claim a clinically relevant issue?	Median (range)	2.0 (0.0–3.0)	2.0 (0.0–3.0)	2.0 (1.0–3.0)	0.62

*Significant post-hoc difference between Australia and USA (p=0.010).

†Significantly lower post-hoc observations compared with expected counts for USA and no mention of RRR, ARR or NNT (Bonferroni correction of 9 comparisons, p<0.001).

‡Significantly higher post-hoc observations compared with expected counts for USA and RRR reported (Bonferroni correction of 9 comparisons, p=0.027).

§Significantly higher post-hoc observations compared with expected counts for USA and information provided on adverse effects, warnings or contraindications (Bonferroni correction of 6 comparisons, p<0.001).

¶Significantly lower post-hoc observations compared with expected counts for USA and no information provided on adverse effects, warnings or contraindications (Bonferroni correction of 6 comparisons, p<0.001).

ARR, absolute risk reduction; NNT, number needed to treat; RRR, relative risk reduction.

References to scientific evidence

Advertisements varied per country regarding the citation of scientific evidence (table 4). There was a statistically significant difference in methodological quality of evidence among the different countries, x2=17.066, p<0.001, with a mean rank of 35.9 for the USA, 39.6 for Canada and 61.0 for Australia. Post-hoc analysis revealed a difference in favour of Australia compared with Canada (p=0.003) and the USA (p<0.001). The median score, that is, the methodological quality score, of this criterion for Australia was 0.42 (0.25–0.70) compared with Canada at 0.25 (0.00–0.63) and the USA at 0.25 (0.00–0.75), where the maximum score was 1. There were no significant differences among countries with respect to supportive score for meta-analyses, systematic reviews, and RCTs.

Table 4

References to scientific evidence

Evaluator criterion	Outcome	Countries
		Australia (n=30)	Canada (n=30)	USA (n=30)	P value
Methodological quality of references	Median (range)	0.4150 (0.25–0.70)	0.25 (0.00–0.63)	0.25 (0.00–0.75)	<0.001*†
Meta-analysis, systematic review, RCT supports claim in ad	Median (range)	1.00 (0.40–2.60)	1.00 (0.90–1.00)	1.00 (0.20–1.00)	0.42

*Significant post-hoc difference between Australia and USA (p<0.001).

†Significant post-hoc difference between Australia and Canada (p=0.0030).

RCT, randomised controlled trial.

Overall scoring of advertisements

The overall quality of drug advertisements, as measured by summing the ranking on five criteria that revealed significant differences among countries, was highest in the USA, followed by Canada and then Australia. Table 5 provides a summary of country rank per criterion.

Table 5

Overall ranking of countries on individual criterion

	Countries ranked by criterion score*
	Australia (n=30)	Canada (n=30)	USA (n=30)
Rank by criterion
Number of claims per ad with quantitative benefit	3	2	1
ARR or NNT reported or can be calculated?	2	2	1
Is information provided on one or more adverse effects, warnings or contraindications within the advertising copy?	3	2	1
If safety information is provided then is this information given the same prominence as benefit information, as measured by font size?	3	2	1
Methodological quality of references	1	2	2
Summative rank	12	10	6

*Lower score is better.

ARR, absolute risk reduction; NNT, number needed to treat.

Advertising appeals and portrayals

The distribution of different types of appeals images, portrayals of the effects of product use and product portrayals were equal in all three countries (p=0.55, p=0.34, p=0.15, respectively). However, there were differences in the distribution of lifestyle or work portrayal images and condition portrayals (p=0.04, p=0.02, respectively, online supplementary file 3a-3e). Overall, the most used appeals by all ads were rational (100%), followed by positive emotional appeals (46%). The most used portrayal was that the product enables health, recreational or work activities (48%). Ads were least likely to use product portrayals (36%), the portrayal of effects of product use (23%) and condition portrayals (16%).

There were various statistically significant differences found between countries and types of appeals and portrayals (table 6). Positive emotional appeals were less common in Australia (26.7%, n=8) compared with Canada (60.0%, n=18) and the USA (50.0%, n=15; p=0.03). Humour appeals were more common in the USA (26.7%, n=8) compared with Canada (13.3%, n=4) and Australia (3.3%, n=1; p=0.04). Lifestyle or work portrayals were more commonly employed by the USA (76.7%, n=23) compared with Canada (50.0%, n=15) and Australia (43.3%, n=13). Portrayals that lifestyle change is an adjunct to product use were infrequently used in all countries: USA (26.7%, n=8), Canada (3.3%, n=1) and Australia (0.0%, n=0; p<0.001). Similarly, portrayals of social approval as a result of product use were also rarely used (USA (10.0%, n=3), Canada (0.0%, n=0) and Australia (0.0%, n=0); p=0.04) as were portrayals of loss of control caused by the condition (Canada (20.0%, n=6), Australia (3.3%, n=1), USA (3.3%, n=1); p=0.03). Post-hoc analyses were done for each χ2 comparison to see if there was a specific country that contributed most to the value of significance, but these analyses did not find any countries that were specific contributors of significance in any comparison.

Table 6

Images in ads

Evaluator criterion	Outcome	Countries with different drug advertising regulations
		Australia (n=30)	Canada (n=30)	USA (n=30)	P value
Type of appeal
Rational	Yes	30 (100.0)	30 (100.0)	30 (100.0)	N/A
	No	0 (0.0)	0 (0.0)	0 (0.0)
Positive emotional	Yes	8 (26.7)	18 (60.0)	15 (50.0)	0.03
	No	22 (73.3)	12 (40.0)	15 (50.0)
Negative emotional	Yes	3 (3.7)	3 (10.0)	5 (16.7)	0.66
	No	27 (90.0)	27 (90.0)	25 (83.3)
Humour	Yes	1 (3.3)	4 (13.3)	8 (26.7)	0.04
	No	29 (96.7)	26 (86.7)	22 (73.3)
Fantasy	Yes	5 (16.7)	5 (16.7)	5 (16.7)	1
	No	25 (83.3)	25 (83.3)	25 (83.3)
Sex	Yes	1 (3.3)	0 (0.0)	1 (3.3)	0.6
	No	29 (96.7)	30 (100.0)	29 (96.7)
Nostalgia	Yes	0 (0.0)	1 (3.3)	2 (6.7)	0.36
	No	30 (100.0)	29 (96.7)	28 (93.3)
No appeals used	Yes	4 (13.3)	1 (3.3)	2 (6.7)	0.34
	No	26 (86.7)	29 (96.7)	28 (93.3)
Lifestyle or work portrayal
Condition interferes with health, recreational or work activities	Yes	3 (10.0)	7 (23.3)	7 (23.3)	0.31
	No	27 (90.0)	23 (76.7)	23 (76.7)
Product enables health, recreational or work activities	Yes	11 (36.7)	13 (43.3)	19 (63.3)	0.1
	No	19 (63.3)	21.1 (56.7)	11 (36.7)
Lifestyle change is alternative to product use	Yes	0 (0.0)	0 (0.0)	0 (0.0)	N/A
	No	30 (100.0)	30 (100.0)	30 (100.0)
Lifestyle change is sufficient	Yes	0 (0.0)	0 (0.0)	0 (0.0)	N/A
	No	30 (100.0)	30 (100.0)	30 (100.0)
Lifestyle change is adjunct to product use	Yes	0 (0.0)	1 (3.3)	8 (26.7)	<0.001
	No	30 (100.0)	29 (96.7)	22 (73.3)
No lifestyle or work portrayals	Yes	17 (56.7)	15 (50.0)	7 (23.3)	0.02
	No	13 (43.3)	15 (50.0)	23 (76.7)
Condition portrayal
Loss of control caused by condition	Yes	1 (3.3)	6 (20.0)	1 (3.3)	0.03
	No	29 (96.7)	24 (80.0)	29 (96.7)
Distress caused by condition	Yes	1 (3.3)	4 (13.3)	7 (23.3)	0.08
	No	29 (96.7)	26 (86.7)	23 (76.7)
No condition portrayals	Yes	29 (96.7)	24 (80.0)	23 (76.7)	0.07
	No	1 (3.3)	6 (20.0)	7 (23.3)
Portrayal of effects of product use
Regaining control as a result of product use	Yes	5 (16.7)	4 (13.3)	7 (23.3)	0.59
	No	25 (83.3)	26 (86.7)	23 (76.7)
Social approval as a result of product use	Yes	0 (0.0)	0 (0.0)	3 (10.0)	0.04
	No	30 (100.0)	30 (100.0)	27 (90.0)
Endurance increased as a result of product use	Yes	0 (0.0)	0 (0.0)	0 (0.0)	N/A
	No	30 (100.0)	30 (100.0)	30 (100.0)
Protection as a result of product use	Yes	3 (10.0)	1 (3.3)	4 (13.3)	0.38
	No	27 (90.0)	29 (96.7)	26 (86.7)
No portrayal of effects of product use	Yes	23 (76.7)	26 (86.7)	20 (66.7)	0.19
	No	7 (23.3)	4 (13.3)	10 (33.3)
Product portrayal
Breakthrough/novelty drug	Yes	7 (23.3)	12 (40.0)	4 (13.3)	0.06
	No	23 (76.7)	18 (60.0)	26 (86.7)
Mechanism of action	Yes	0 (0.0)	2 (6.7)	4 (13.3)	0.12
	No	30 (100.0)	28 (93.3)	26 (86.7)
Image of product	Yes	8 (26.7)	11 (36.7)	6 (20.0)	0.35
	No	22 (73.3)	19 (63.3)	24 (80.0)
No product portrayal	Yes	21 (70.0)	17 (56.7)	20 (66.7)	0.53
	No	9 (30.0	13 (43.3)	10 (33.3)

Discussion

Our study revealed significant differences among countries regarding the following criteria: number of claims with quantitative benefit; RRR, ARR and NNT reported or calculated; mention of adverse effects, warnings or contraindications; equal prominence between safety and benefit information; and methodological quality of references. Taken together, our overall scoring ranked the USA first, Canada second and Australia third for the quality of journal ads, which confirms our original hypothesis in that self-regulatory systems (ie, the one used in Australia) may have the greatest influence in yielding the lowest quality ads compared with other regulatory regimes.

Although the US ads ranked first in quality, this finding should not be taken to imply that using them as a source of information would lead to appropriate prescribing. Only 13% of ads in American Family Physician mentioned the generic name every time the brand name was mentioned; only a single ad either gave an ARR or NNT or the information to calculate one; the maximum score for whether the main claim in the ads was to a clinically relevant issue was 3 but the median score was only 2; and only 30% of the ads referenced a meta-analysis, systematic review or RCT. The limitations seen in US advertisement quality might be due to a lack of resources needed to properly evaluate the volume of advertising. As of 2016, the FDA’s Office of Prescription Drug Promotion with a staff of just over 70 people received nearly 100 000 promotional material submissions related to prescription medications annually.33 Finally, the FDA only evaluates ads before they appear in relatively rare circumstances (table 1).

Countries only differed with respect to humorous, positive emotional and social approval portrayals as well as the presence of lifestyle or work portrayals. Although post-hoc testing was not significant, these portrayals were generally most commonly used by the US ads. Some of the pictorial features of the ads such as the frequent use of emotional appeals in ads, the relative absence of both the portrayals of lifestyle change as an adjunct to product use as well as the portrayal of the product enabling health, recreational or work activities all suggest that some aspects of the ads were not intended to give physicians an accurate view of the value of the medications that they were promoting.

Our findings are consistent with a previous study that concluded ad quality was affected by different regulations.8 Although that study examined ads published between 1961 and 1977, it appears that different regulatory regimes continue to influence ad quality. Another study compared ads between Australia, Malaysia and the USA between 2004 and 2006.9 Our study yielded similar results in that warning information was most likely to be provided in the US ads and least likely to be provided in Australian ads. We also found consistently incomplete product information in the advertising copy (eg, lack of safety information and support for claims made in ads) irrespective of the country. However, there was a large contrast between the two studies when comparing the percentage of ads that mention the generic name. Our study yielded a lower percentage, likely due to our more stringent criteria in that the generic name had to be mentioned every time the brand name was mentioned. Our findings regarding the supportive score for references were also higher compared with a past study that analysed the accuracy of scientific claims in Spanish drug ads.34 All known previous studies comparing ads used criteria focused on product information data but did not include additional comparisons known to influence prescriber behaviour, such as references to scientific evidence as well as advertising appeals and portrayals.8–10

Limitations

Despite examining information in ads that may affect prescribers’ behaviour, our study had some limitations. First, we only examined in-print journal advertisements and not other forms of promotion that affect prescribing practices. Additionally, we did not assess the accuracy of the information in the ads. While this would have been desirable, the lack of information about many important aspects of drug efficacy and safety speaks the poor educational quality of the ads. We also did not directly examine whether the ads all conformed to regulatory requirements in the country in which they were published or whether they had been subject to complaints to the regulator. We suspect that violations of regulations may have confounded our results. For instance, we found that advertisements from the USA were significantly more likely to report on adverse events, despite all regulatory bodies requiring a fair balance between benefits and harms, suggesting that advertising originating in Australia and Canada may not have been complaint with the relevant codes. Advertisements for different drugs and from different manufacturers may also yield differences in the type of product information, references to scientific evidence, as well as appeals and portrayals. We only examined one country per regulatory regime and therefore we could not determine whether the differences were due to the regulatory framework or to other regulatory, legal, cultural or health system factors specific to each country. For instance, our finding that USA ads contain more information on adverse effects, warnings or contraindications may also reflect industry concerns with litigation in addition to FDA regulation. To the extent that our findings do reflect different regulatory regimes, they only apply to ads in family practice journals in three developed countries over the period 2014–2015. Finally, we only examined parts of the ads that could be objectively scored and our scoring system for some elements while used before has not been validated against the effects that ads have on prescribing behaviour.

Conclusion

This is the first study to compare advertising quality under different regulatory frameworks. We found differences in the quality of journal advertisements concerning product information, references to scientific information, as well as appeals and portrayals that were produced under different regulatory regimes. Regulation via direct government control (ie, the USA) yielded the highest-quality ads, followed by regulation by autonomous bodies (ie, Canada) and then by industry self-regulation (ie, Australia). Despite this, all forms of regulation as they are currently practiced have limitations in terms of the quality of the ads. Our results suggest that well-resourced government regulation might be the best way to ensure that journal advertising provides physicians with the accurate, complete and objective information that they need.