An evaluation of the evidence submitted to Australian alcohol advertising policy consultations.

BACKGROUND: Industry self-regulation is the dominant approach to managing alcohol advertising in Australia and many other countries. There is a need to explore the barriers to government adoption of more effective regulatory approaches. This study examined relevance and quality features of evidence cited by industry and non-industry actors in their submissions to Australian alcohol advertising policy consultations.
METHODS: Submissions to two public consultations with a primary focus on alcohol advertising policy were analysed. Submissions (n = 71) were classified into their actor type (industry or non-industry) and according to their expressed support for, or opposition to, increased regulation of alcohol advertising. Details of cited evidence were extracted and coded against a framework adapted from previous research (primary codes: subject matter relevance, type of publication, time since publication, and independence from industry). Evidence was also classified as featuring indicators of higher quality if it was either published in a peer-reviewed journal or academic source, published within 10 years of the consultation, and/or had no apparent industry connection.
RESULTS: Almost two-thirds of submissions were from industry actors (n = 45 submissions from alcohol, advertising, or sporting industries). With few exceptions, industry actor submissions opposed increased regulation of alcohol advertising and non-industry actor submissions supported increased regulation. Industry actors cited substantially less evidence than non-industry actors, both per submission and in total. Only 27% of evidence cited by industry actors was highly relevant and featured at least two indicators of higher quality compared to 58% of evidence cited by non-industry actors.
CONCLUSIONS: Evaluation of the value of the evidentiary contribution of industry actors to consultations on alcohol advertising policy appears to be limited. Modifications to consultation processes, such as exclusion of industry actors, quality requirements for submitted evidence, minimum standards for referencing evidence, and requirements to declare potential conflicts, may improve the public health outcomes of policy consultations.

Introduction

The sustained burden of alcohol-caused harm has required governments to consider regulatory options for reducing levels of harm and associated public costs [1, 2]. Stakeholder consultation is a common part of government policy development processes [3, 4], including for alcohol policy [5-8]. Public consultation processes typically involve a general call for submissions [4], providing an opportunity for stakeholders to connect decision-makers with evidence and shape how the policy matter is perceived. In general, few requirements or conditions are placed on submissions to public consultations in terms of who may submit them, how they are presented, or the veracity of their content [6, 9, 10]. Concerns about the participation of alcohol industry actors in public health policy consultations have been raised by health and academic stakeholders due to the industry’s conflicts of interest and documented behaviours of distorting and denying evidence [5, 7, 8, 11–15].

Where submissions are publicly accessible, they can provide a data source for evaluating the evidentiary contribution of different actors to policy consultations. To date, there has been limited work of this nature in the context of alcohol policy. In a study of alcohol industry submissions to a Scottish Parliamentary consultation on minimum unit pricing, Cullen et al. found that peer-reviewed evidence (i.e., evidence appraised to be of sufficient quality by independent academic reviewers) accounted for only 15% of 115 items of evidence cited across 25 industry submissions [5]. Cited evidence was most often drawn from government reports or data (45%) or private/non-government organisations (30%). Several evidence items cited in the industry submissions were published more than a decade before the consultation, and Cullen et al. noted that such dated texts could not provide up-to-date support for the submitters’ claims in this context [5].

In New Zealand, the Ministerial Forum on Alcohol Advertising and Sponsorship was established in 2014 to examine whether additional controls on alcohol advertising and sponsorship were needed to reduce harm from alcohol. The Ministerial Forum held a public consultation process with a request for submitters to focus on evidence that had emerged since 2010. A commissioned analysis of submissions to the Ministerial Forum found that a large body of post-2010 evidence was presented by submitters who supported further restrictions on alcohol advertising, with these submitters most commonly from the health sector, community groups, local governments, and universities [16]. In contrast, a smaller body of post-2010 evidence was presented by submitters who opposed further restrictions, most commonly sporting bodies, advertising and media organisations, alcohol industry bodies, and retailers. Similar quantities of pre-2010 evidence were presented by submitters who supported and opposed further alcohol advertising restrictions.

More detailed assessments of evidence cited in industry actor submissions to government consultations can be found in the tobacco policy context. Hatchard, Fooks, Evans-Reeves, Ulucanlar, and Gilmore [17] and Evans-Reeves, Hatchard, and Gilmore [18] analysed tobacco industry submissions to a UK consultation on standardised packaging in terms of the quality and relevance of cited evidence. Evidence was judged to be higher quality if it was peer-reviewed and independent of industry. Hatchard et al. found that most evidence cited by tobacco companies lacked either policy relevance (i.e., it was not focused on standardised packaging or tobacco packaging) or indicators of quality [17]. Evidence classified as relevant to the issue under consideration was often connected to industry and not peer reviewed [17]. Industry connections to cited evidence were not disclosed in submissions [18]. Evidence cited in submissions by ultra-processed food companies has similarly been found to lack independence from industry and other indicators of quality [19]. Researchers have recognised parallels in evidence-related tactics by different corporate sectors [19-21] and noted the potential value of applying similar analyses to other public health policy areas where corporate interests seek to influence policy, such as alcohol [17].

The focus on alcohol advertising in the present study was chosen for several reasons. First, a substantial body of research evidence is available to inform deliberations about alcohol advertising policy (for recent reviews see [22-26]). The evidence base has developed to a level where it can be concluded that exposure to alcohol advertising is a cause of alcohol use among young people [27, 28]. Second, limited policy progress towards mitigating the effects of alcohol advertising has been achieved [1, 29]. Governments in Australia, as in many countries, continue to rely on industry-led self-regulation of alcohol advertising despite the documented weaknesses of that approach [25, 30–34]. Third, achieving regulatory change to reduce exposure to alcohol advertising remains a priority goal of health stakeholders [35-38]. Exploring the barriers to governments adopting more effective regulatory approaches may provide insights into opportunities that facilitate policy progress.

Investigations by researchers of alcohol industry efforts to influence advertising policy have sought to identify industry actor strategies and common arguments made [15, 39]. The most frequently documented strategies were: (i) policy substitution, including via promotion of voluntary codes and other ineffective initiatives; (ii) constituency building via development of trade associations, mobilisation of allied industries, and formation of front groups; and (iii) information-related tactics, including examples of selective citation of favourable evidence, omission of evidence, and contesting evidence used to support policy [15]. Frames and arguments used by alcohol industry actors when opposing increased regulation closely mirror those used by the tobacco industry [15, 39, 40]. Common industry claims include that alcohol brands are marketed responsibly and only to adults in compliance with self-regulatory codes, making increased regulation unnecessary. Claims of adverse unintended consequences of regulation are also common and often relate to the cost of compliance, job losses in associated industries, and discouragement of healthier product development [8]. Industry actors have also argued that the evidence base is insufficient to support proposals to regulate further, as advertising is simply intended to inform consumer brand selection and influence market share [8, 15, 39].

The present study examined a range of relevance and quality features of evidence cited by industry actors in their submissions to alcohol advertising policy consultations and compared them with evidence cited by non-industry actors. The study aimed to address two key questions: (i) What evidence do industry and non-industry actors cite in submissions to alcohol advertising policy consultations in terms of subject matter relevance, publication type, time since publication, and independence from industry funding sources? and (ii) What proportions of the cited evidence feature indicators of higher quality and relevance? Implications were considered for strengthening the use of evidence in policy processes to improve public health outcomes.

Materials and methods

Design

A comparative analysis was conducted of the evidence cited in submissions to Australian policy consultations about alcohol advertising. Using a quantitative approach informed by Hatchard et al. and Evans-Reeves et al.’s analyses of tobacco industry submissions, relevance and quality features of evidence cited in submissions were assessed [17, 18].

Data

Analysed data were submissions made to the two public consultations from the past 10 years that primarily focused on alcohol advertising policy. The consultations were undertaken by the Australian Federal Government (the Australian National Preventive Health Agency (ANPHA) [41]) and the New South Wales (NSW) State Parliament [42]. The ANPHA consultation considered the effectiveness of existing national regulations in addressing community concerns about alcohol advertising (n = 34 submissions responded to ANPHA’s issues paper) and the NSW Parliament’s 2017 consultation considered a Bill (the NSW Alcoholic Beverages Advertising Prohibition Bill) that proposed to reduce alcohol advertising within the state (n = 42 submissions). Submissions were publicly available from the relevant government or parliamentary websites, except for four confidential submissions that were not released (one was later released by the submitter and included in the analysis). Two submissions to the ANPHA consultation were excluded (one was only a cover letter and another appeared to be incorrectly listed as a submission). In total, 71 submissions (n = 30 from the ANPHA consultation and n = 41 from the NSW consultation) were included in the analysis.

Procedure for submission-level variables

Page lengths of the main bodies of submissions and appendices (if present) and the number of evidence items cited in each submission were counted to describe the composition of submissions made to the two consultations. Each submission was coded by type of actor (industry or non-industry) and expressed position on increased regulation of alcohol advertising (support, neutral, or oppose). Industry actors included alcohol trade associations, alcohol producers and retailers, advertising and media organisations, and sporting organisations with commercial relationships with members of the alcohol industry (such as via sponsorship). All other submissions were classified as originating from non-industry actors. Submissions were classified as supportive of increased regulation of alcohol advertising if they contained at least one statement expressing support for increased regulation. Conversely, submissions were classified as opposing increased regulation of alcohol advertising if they contained at least one statement expressing opposition to increased regulation or government regulation of alcohol advertising. Where no clear indication of position was provided either way, submissions were classified as neutral.

Procedure for coding evidential criteria

For each submission, one researcher (JS) extracted details of cited evidence into a spreadsheet (bibliographic entry was copied and pasted without editing). If an item of evidence was cited more than once within a submission, it was only recorded once in the spreadsheet. To be extracted, sufficient information on the source of cited evidence was required. An appendix to a submission (e.g., a report or journal publication) was treated as an item of evidence and its details (author, title, date) were recorded. Attempts were made to access all cited evidence via the internet (using reasonable minimal effort) and the outcome recorded. The Wayback Machine online archive was used to access website sources as they existed at the date of citation [43].

Each item of evidence was classified against a four-part framework (see Table 1) adapted from Hatchard et al. and Evans-Reeves et al.’s analyses of tobacco industry submissions [17, 18]. In some instances, cited evidence was not accessible but bibliographic entries within the submission contained information that could be classified against the framework (e.g., included a title that clearly indicated the subject matter or year of publication). Where the necessary information was not accessible, items were coded as unknown.

Table 1

Coding framework for classifying evidence.

	Evidential criteria		Coding categories
Relevance	Subject matter	What is the topic of the evidence?	Alcohol advertising (high relevance)
			Alcohol but not advertising (medium relevance)
			Advertising but not alcohol (medium relevance)
			Unrelated to alcohol or advertising (low relevance)
			Unknown
Quality	Publication type	Where was the evidence published?	Peer-reviewed journala
			Academic publication (e.g., book, university-published report, conference abstract)a
			Government publicationb
			Parliamentary publicationb
			Publication by a company or organisation not linked to the alcohol industryb
			Media coverageb
			Publication by an alcohol industry-linked company or organisation
			Publication from an industry self-regulatory body
			Unknown
	Time since publication	When was the evidence published?	Published within 10 years of the consultationc
			Published more than 10 years before the consultation
			Unknown
	Independence	Does the peer-reviewed journal or academic publication have any connection with the alcohol industry?	Independent of the alcohol industryb
			No apparent alcohol industry connectionb
			Alcohol industry-funded
			Alcohol industry-linked

aThis coding category forms part of an indicator of higher quality: Published in a peer-reviewed journal or academic source.

bThis coding category forms part of an indicator of higher quality: Independent of or with no apparent connection to the alcohol industry.

cThis coding category is an indicator of higher quality: Published within 10 years of the consultation.

Subject matter relevance

The subject matter of evidence cited in submissions was evaluated in terms of relevance to the issue under consideration in the consultation (i.e., alcohol advertising). Subject matter was assessed based on the presence or absence of key terms related to alcohol (alcohol, liquor, beer, wine, spirits, brand names) and advertising (advert*, market*, promot*, sponsor*) in the publication. High relevance was coded for publications in which one or more alcohol and one or more advertising terms were present. Medium relevance was coded where either one or more alcohol or one or more advertising terms were present. The low relevance code applied to publications where neither alcohol nor advertising terms were identified.

Publication type

Publication type was coded based on the category that best described where evidence had been published (see Table 1 for the nine coding categories). If a report indicated that it had been commissioned (e.g., by a government department), publication type was determined based on the nature of the commissioning body (e.g., official government publication). Publications produced by an alcohol industry-linked company or organisation included those from: (i) alcohol producers, (ii) alcohol industry trade associations and news services (e.g., The Shout), (iii) social aspects organisations (e.g., DrinkWise), (iv) advertising and media agencies and associations that are likely to have alcohol industry clients (e.g., Google, Facebook), and (v) consulting or research/data organisations where the business community is a key customer (e.g., Deloitte, Access Economics, KPMG, Roy Morgan, IBISWorld, Nielsen, Oztam). Media coverage included general news outlets and advertising industry news services (e.g., Mumbrella, Ad Age).

Time since publication

Time since publication (i.e., age of evidence) was identified by Cullen et al. [5] as important to evidence assessment and was included in this study to account for substantial changes in the advertising landscape in terms of technologies available to advertisers and structure of the global alcohol production industry [29]. Evidence was coded as published within 10 years of the consultation based on publication year (i.e., ≥2003 for evidence cited in submissions to ANPHA and ≥2008 for evidence cited in submissions to NSW Parliament).

Independence

Peer-reviewed journal and academic publications were coded for independence from industry. Information provided within the publication (e.g., funding acknowledgement or conflict declaration) was used to inform the coding decision. Where the coder was unfamiliar with the funder, the funder’s website was searched for evidence of industry connections. To be coded as independent, the publication needed to either: (i) name funding sources that were not connected to alcohol or advertising industries or (ii) declare that there were no conflicts. A publication that declared a funding source within the alcohol industry was coded as industry-funded. To be coded as industry-linked evidence, the publication needed to declare a connection between an author or the publication and either: (i) the alcohol industry other than as a funder of the published work or (ii) the advertising industry. If insufficient information was provided to allow for one of the above classifications, independence was coded as having no apparent alcohol industry connection.

Indicators of higher quality evidence

As shown in Table 1, indicators of higher quality included whether evidence was:

Published in a peer-reviewed journal or academic source (e.g., book, conference abstract). Peer-reviewed journal publications have been assessed by external researchers knowledgeable in the subject area, acting as a filter to prevent poorly designed studies from being published [17];

Independent of or with no apparent connection to the alcohol industry. Connections with financially vested interests can cast doubt on the independence and quality of evidence due to the potential for bias associated with industry influence [17, 44]; and/or

Published within 10 years of the consultation. More recent work is more likely to reflect the contemporary advertising landscape [5, 29, 45] and improvements in research methodologies [46-48].

Inter-coder reliability

Sub-samples (25%) from submissions (n = 18) and cited evidence (n = 307) were randomly selected and individually coded by two researchers (JS, HP) to gauge inter-coder reliability of the assessment of submission-level and evidence-level variables. There was 100% agreement on coding of submission-level variables (actor type and position on increased regulation of alcohol advertising). Coding of evidence-level variables against the four-part framework resulted in 84 coding differences across the 1228 coded cells (6.8%). Differences were discussed and resolved between the two coders. The remaining submissions and cited evidence were coded by JS.

Statistical analysis

Frequencies were calculated for the submission-level variables. For the evidence-level variables, frequencies across the four-part framework were calculated separately for evidence cited by industry and non-industry actors. Frequencies of evidence that featured any two or all three indicators of higher quality were calculated. Chi-square tests were used to assess differences in the proportions of evidence cited by industry and non-industry actors, with a Bonferroni correction to account for the number of comparisons (0.05/19 = 0.0026). Data were eyeballed for potential outlier scores. Where detected, analyses were re-run with outlier scores removed to determine whether results differed.

Results

Submission characteristics

The majority of submissions were made by industry actors (63%, n = 45, Table 2), of which half (51%, n = 23) were from advertising and media organisations (S1 Table). All but one of the industry actor submissions (44 of 45) opposed increased regulation of alcohol advertising (one was neutral). Conversely, 25 of the 26 non-industry actor submissions expressed support for increased regulation of alcohol advertising (one was neutral).

Table 2

Submission characteristics.

		Industry actor submissions			Non-industry actor submissions
		ANPHA (n = 20)	NSW (n = 25)	ANPHA and NSW (n = 45)	ANPHA (n = 10)	NSW (n = 16)	ANPHA and NSW (n = 26)
Main body	Range page length	1–41	1–28	1–41	1–39	2–24	1–39
	Median page length	9	4	6	15	4.5	7
	Mean page length	11	7	9	17	7	11
Appendices	No. of submissions with appendices	5	3	8	3	4	7
	Range page length	1–136	3–10	1–136	6–56	4–34	4–56
	Mean page length (per submission with appendices)	32	6	22	25	18	20
Cited evidence	Total	261	259	520	384	346	730
	Range per submission	0–42	0–110	0–110	2–112	2–66	2–112
	Median per submission	11	2	4	37	16	18
	Mean per submission	13	10	12	38	22	28

Note. ANPHA: Australian National Preventive Health Agency issues paper Alcohol advertising: The effectiveness of current regulatory codes in addressing community concerns. NSW: New South Wales Alcoholic Beverages Advertising Prohibition Bill.

Submission size varied substantially within and between actor groups in terms of page length and quantity of evidence cited (Table 2). A quarter (24%, n = 11) of industry actor submissions did not cite any evidence; non-industry actor submissions each cited at least two items of evidence. In total, 520 items of evidence were cited by industry actors (mean of 12 items per submission) and 730 items of evidence were cited by non-industry actors (mean of 28 items per submission).

Comparison of evidence cited by industry and non-industry actors

Not all evidence was able to be accessed; 14% (73 of 520) of evidence items cited by industry actors and 6% (44 of 730) of evidence items cited by non-industry actors were not accessible (see S2 Table). Publications by alcohol industry-linked organisations were the least accessible of the publication types cited by industry actors (37 of 73); for non-industry actors, government publications were the least accessible (15 of 44).

Table 3 summarises results for comparisons of proportions between industry and non-industry actor submissions for the four key measures relating to evidence cited. Industry actors cited a significantly lower proportion of evidence that was highly relevant (i.e., about alcohol advertising; 50% vs 73%, p < .0001) and a significantly higher proportion of evidence of medium relevance (i.e., about alcohol or advertising but not alcohol advertising; 37% vs 22%, p < .0001) than non-industry actors.

Table 3

Comparisons of subject matter relevance, publication type, time since publication, and independence.

		Evidence cited in industry actor submissions (n = 520)	Evidence cited in non-industry actor submissions (n = 730)	Chi-square statistic	p-value	Phi coefficient
		n (%)	n (%)
Subject matter relevance	Alcohol advertising (High)	262 (50)	531 (73)	65.90	< .0001*	.23
	Alcohol but not advertising (Medium)	145 (28)	125 (17)	37.82 a	< .0001*	.17
	Advertising but not alcohol (Medium)	49 (9)	32 (4)	-	-	-
	Unrelated to alcohol or advertising (Low)	13 (3)	22 (3)	0.29	.5902	.02
	Unknown subject matter	51 (10)	20 (3)	-	-	-
Publication type	Peer-reviewed journal	165 (32)	276 (38)	7.48b	.0062	.08
	Academic publication	29 (6)	52 (7)	-	-	-
	Government publication	85 (16)	131 (18)	0.52	.4708	.02
	Parliamentary publication	11 (2)	15 (2)	0.01	.9203	.00
	Publication by an alcohol industry-linked organisation	76 (15)	21 (3)	58.63	< .0001*	.22
	Publication by an organisation not linked to the alcohol industry	39 (8)	126 (17)	25.13	< .0001*	.14
	Publication from an alcohol advertising self-regulatory body	102 (20)	85 (12)	15.29	< .0001*	.11
	Media coverage	2 (<1)	20 (3)	9.72	.0018*	.09
	Unknown publication type	11 (2)	4 (<1)	-	-	-
Time since publication	Published more than 10 years before the consultation—all evidencec	137 (32)	59 (9)	97.80	< .0001*	.30
	Published within 10 years of the consultation—all evidencec	298 (68)	630 (91)	-	-	-
	Published more than 10 years before the consultation—journal/academicd	110 (57)	44 (13)	111.39	< .0001*	.46
	Published within 10 years of the consultation—journal/academicd	83 (43)	283 (87)	-	-	-
	Unknown publication year	85 (16)	41 (6)	-	-	-
Independencee	Independent	91 (47)	234 (71)	31.42	< .0001*	.25
	No apparent alcohol industry connection	86 (44)	80 (24)	21.26	< .0001*	.20
	Alcohol industry-funded	7 (4)	3 (<1)	6.04f	.0139	.11
	Alcohol industry-linked	10 (5)	9 (3)	-	-	-

Note.

* Significant at Bonferroni-adjusted p value <0.0026.

aMedium relevance categories were combined for the chi-square comparison.

bPeer-reviewed journal and academic publications were combined for the chi-square comparison.

cCount includes all cited evidence, excluding those with unknown publication year.

dCount includes only peer-reviewed journal and academic publications, excluding those with unknown publication year.

eIndependence is reported for peer-reviewed journal and academic publications only.

fAlcohol industry-funded and alcohol industry-linked categories were combined for the chi-square comparison.

Peer-reviewed journal publications were the most cited publication type for both actor groups. Similar proportions of journal, academic, government, and parliamentary publications were cited by both actor groups. Compared to non-industry actors, industry actors cited significantly higher proportions of publications by alcohol industry-linked organisations (15% industry vs 3% non-industry, p < .0001) and self-regulatory bodies (20% industry vs 12% non-industry, p < .0001) and correspondingly fewer publications by organisations not linked to the alcohol industry (8% industry vs 17% non-industry, p < .0001).

Industry actors were significantly more likely to cite older evidence (published more than 10 years before the consultation) than non-industry actors. This applied both to evidence cited overall (32% industry vs 9% non-industry, p < .0001) and to peer-reviewed journal and academic publications specifically (57% industry vs 13% non-industry, p < .0001).

Small proportions of journal and academic publications cited by each actor group were identified as alcohol industry-funded or -linked (9% industry vs 4% non-industry, p = .0139). Many cited publications (n = 166) did not provide a clear funding or disclosure statement, which prevented assessment of independence for a substantial proportion of journal and academic publications (44% industry vs 24% non-industry, p < .0001).

Potential outlier scores were identified. When analyses were re-run with outlier scores removed, no outlier effects were detected.

As shown in Table 4, proportions of evidence cited by industry actors that featured any two (54%) or all three (14%) indicators of higher quality were significantly and substantially smaller than for non-industry actors (80%, p < .0001 and 37%, p < .0001 respectively). The proportions reduced for both actor groups when high relevance was considered in addition to the indicators of higher quality; however, the marked difference remained between evidence cited by industry and non-industry actors. Only 27% (141 of 520) of evidence cited by industry actors was highly relevant and featured at least two indicators of higher quality, compared to 58% (420 of 730) of evidence cited by non-industry actors.

Table 4

Proportions of evidence with indicators of higher quality and relevance.

Indicators of higher quality and relevance	Evidence cited in industry actor submissions	Evidence cited in non-industry actor submissions	Chi-square statistic	p-value	Phi coefficient
	n (% of 520)	n (% of 730)
High relevance a	262 (50)	531 (73)	65.90	< .0001*	.23
Any two (or more) indicators of higher quality b	283 (54)	582 (80)	91.22	< .0001*	.27
High relevance plus any two (or more) indicators of higher quality	141 (27)	420 (58)	113.59	< .0001*	.30
All three indicators of higher quality b	74 (14)	271 (37)	79.64	< .0001*	.25
High relevance plus all three indicators of higher quality	38 (7)	214 (29)	91.38	< .0001*	.27

Note.

* Significant at Bonferroni-adjusted p value <0.0026.

aQuality was not considered.

bSubject matter relevance was not considered.

Discussion

Principal findings

To our knowledge, this is the first time evidence put forward in alcohol policy submissions for consideration by decision makers has been assessed for relevance and quality. Evidence cited by industry actors was less likely to be highly relevant or to feature indicators of higher quality than evidence cited by non-industry actors. Overall, only 27% of evidence cited by industry actors compared to 58% of evidence cited by non-industry actors was about alcohol advertising (i.e., highly relevant) and displayed two or more indicators of higher quality: i) published in a peer-reviewed journal or academic source, ii) independent of or with no apparent connection to the alcohol industry, and/or iii) published within 10 years of the consultation.

Comparison with other studies

A distinct polarisation of views on potential regulatory action was identified, with industry actors opposing and non-industry actors supporting further regulation of alcohol advertising. This finding is consistent with Allen and Clarke’s analysis of submissions to the New Zealand Ministerial Forum on Alcohol Advertising and Sponsorship [16]. In the present study, the substantial participation of advertising and media organisations (accounting for 32% of analysed submissions), alongside major sporting bodies (3% of submissions), reflects the position of these industries as close allies of the alcohol industry in resisting advertising controls [8, 49]. This has previously been observed in the active roles of these industries in resisting tobacco advertising bans [40, 50].

The findings demonstrate several strategies known to be used by the alcohol industry to undermine the evidence base for strengthened policy approaches. First, the strategy of omitting evidence [15] was demonstrated by the substantial number of industry actor submissions that expressed opposition to further regulation of alcohol advertising but provided no supporting evidence. Second, the selective citation of favourable evidence [15] can be seen in industry actors’ substantial use of industry-linked publications and dated sources of evidence. Industry actors may have identified older evidence as more favourable to their position, as opposed to the more contemporary evidence cited by non-industry actors in support of further regulation of alcohol advertising. Third, industry actors appear to have sought to change the evidential landscape within which the alcohol advertising policy debate was conducted [8, 51] via their greater reliance on less relevant evidence, encouraging attention on other aspects of alcohol policy debates rather than more focused attention on alcohol advertising policy. As such, claims by industry actors that the evidence base is insufficient to support proposals to further regulate alcohol advertising [15, 39] appear to relate more to the industry’s selectivity in acknowledging evidence than any genuine deficiency in the evidence base. The exclusion of industry actors from alcohol policy development processes may be necessary if these established industry strategies are to be avoided [8, 12–15].

While the present study is not directly comparable with Hatchard et al.’s analysis of evidence cited in tobacco industry submissions, similar patterns emerged [17]. A substantial proportion of evidence cited by industry actors in both studies was focused on ‘parallel’ subject matter (i.e., of medium or low relevance) rather than being directly relevant to the consultation topic. About 50% of industry actor evidence in the present study and 34% of tobacco industry evidence in Hatchard et al. was of high relevance. In both studies, significantly lower proportions of evidence cited by industry actors were highly relevant and featured indicators of higher quality compared to the points of comparison (i.e., systematic review evidence in Hatchard et al. and evidence cited by non-industry actors in the present study).

Implications for policymakers

In both consultations that were the focus of this study, policymakers received a greater number of submissions from industry actors than non-industry actors. The volume of industry actor opposition to increased regulation of alcohol advertising may have been noticeable to policymakers, tempering their willingness to change the policy status quo. Policymakers who act to restrict alcohol advertising would need to be prepared to tolerate opposition from industry actors but can expect support from non-industry actors based on a solid foundation of evidence.

Industry actor submissions in the present study commonly cited sources published more than 10 years before the consultation. While not necessarily obsolete or irrelevant, as Cullen et al. identified, dated sources of evidence are unlikely to provide up-to-date support for the claims made in submissions [5]. Policymakers were presented with evidence that was less likely to be relevant to the contemporary alcohol advertising environment, given the evolution and proliferation of media channels via which audiences are exposed to alcohol advertising [45]. Older research is also less likely to have incorporated rapid advances in research methodologies, such as improved tools for measuring advertising exposure [46-48]. Policy consultations may benefit from the introduction of quality requirements for submitted evidence [19], which include regard for the timeliness of evidence.

While the age of evidence can be readily assessed by policymakers, assessing independence from vested interests may be more difficult [19, 52], as was found in the present study. As journals adopt more comprehensive disclosure requirements [52-54], conflicts should be more visible in future publications, but this would still require policymakers to consult the cited evidence. To further aid the assessment of independence in policy consultations, it may be useful for the administering agency to define potential conflicts of interests and require submitters to declare any potential conflicts relevant to the evidence they present [17-19].

Action to better control alcohol advertising did not appear to follow either of the consultations included in this study. The consultation reports suggested that evidence presented by non-industry actors was convincing, but was not able to overcome representations of industry actors. The committees found the existing self-regulatory regime to be “inadequate” (p. 6) [55] and “insufficient and failing” (p. 86) [42] to protect young people from alcohol advertising. However, ANPHA was disbanded by the Australian Government in the same year the final report was published and its recommendations to strengthen regulations were not progressed. The NSW Parliamentary committee considering the Alcohol Beverages Advertising Prohibition Bill 2015 recommended the Bill not be passed. Despite having rejected some of the “sophisticated arguments” put forward by the alcohol and advertising industries, such as “why there is no causal link between advertising and consumption” (p. 33) [42], other industry arguments appeared to create room for doubt among committee members. For example, an alcohol producer presented evidence about health benefits of moderate alcohol use. While the committee acknowledged “compelling evidence that alcohol in fact poses no real health benefits” (p. 33) [42], rather than consider the issue settled, the committee recommended that the NSW Health Department “closely examine the issue of whether there is any safe level of alcohol consumption” and that “this research should determine whether alcohol advertising should have further restrictions applied to it” (p. 34) [42]. The committee also appeared to accept the concerns of industry actors that “the Bill will have a detrimental impact on the alcohol and advertising industries and will lead to a range of unintended consequences” (p. 56) [42], such as making it difficult for new market entrants and small businesses to grow.

Strengths and limitations of the study

The primary strength of this study was the inclusion of all accessible submissions made to two consultations on alcohol advertising regulation, capturing a range of policy actors who participated in the consultations and enabling comparisons between actor groups. The inclusion of multiple consultations strengthened our ability to consider the implications of findings for improving the use of evidence in policy processes.

A limitation was that reliance on publicly available submissions precluded access to confidential submissions; however, there appeared to have been only three confidential submissions that could not be accessed. Second, consistent with difficulties experienced by those examining industry submissions to consultations in other policy contexts (e.g., ultra-processed foods and sugar-sweetened beverages [19, 20]), the referencing practices within submissions impacted the researchers’ ability to identify cited evidence. These practices varied between submissions as there was no requirement to reference evidence in a manner that would allow others to locate it or even to reference evidence at all. Third, the approach of the present study did not give an indication of the weight of the use of different sources of evidence. Regardless of whether an item of evidence was cited once or 10 times within a submission, it was recorded once for that submission. Fourth, independence for journal and other academic publications was assessed only on the basis of disclosures within publications; relevant information from other sources may have been missed. Almost one-third of journal and academic publications cited by submissions did not include a statement from which independence could reasonably be assessed. In some of these cases, disclosure statements were made but they were ambiguous (e.g., “the usual disclaimers apply”) and therefore of limited value to readers wishing to assess potential conflicts. A fifth limitation was that the passing of time since the consultations may have reduced the accessibility of cited evidence in a small number of instances (S2 Table). Finally, as this study focussed on a single policy issue, generalizability of results to other alcohol policy issues is unknown.

Unanswered questions and future research

There may be value in replicating this study using consultations that explore other alcohol policy topics (e.g., price or availability controls) to determine whether similar patterns exist. Future assessments of the relevance and quality of evidence may consider weighting the criteria to prioritise the indicators of greatest importance to the issue under consideration. While analyses of submissions provide useful insights into the information collected to inform policymaker deliberations, they are not able to tell us how policymakers assess information they receive. Building a better understanding of how policymakers evaluate and apply evidence would help to identify practical options for improving the uptake of research evidence in policy deliberations.

Conclusions

The value of the evidentiary contribution of industry actors to consultations on alcohol advertising policy appears to be limited. Modifications to consultation processes, such as exclusion of industry actors, quality requirements for submitted evidence, minimum standards for referencing evidence, and requirements to declare potential conflicts, may improve the transparency and application of evidence and public health outcomes of policy consultations.

Number of submissions by actor type.

(DOCX)

Click here for additional data file.

Accessibility of cited evidence by actor group, consultation and publication type.

(DOCX)

Click here for additional data file.

14 Oct 2021

PONE-D-21-26197An evaluation of the evidence submitted to Australian alcohol advertising policy consultationsPLOS ONE

Dear Dr. Stafford,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

==============================The reviewers were highly positive about the value and rigour of this manuscript and make some minor suggestions to strengthen its presentation. Please address each of these comments. I also concur with Reviewer 1 that the material under Acknowledgments is more appropriately located within a competing interests statement - I ask that you review the journal's policy on declaring competing interests to decide what is most appropriately declared. I do not necessarily think this needs to be included in the limitations section, but you might include a paragraph on 'reflexivity' if you think it important to document how the team's expertise and social location shaped the work and in what ways. I look forward to receiving your revised manuscript.

==============================

Please submit your revised manuscript by Nov 28 2021 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Quinn Grundy, PhD, RN

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2. Please amend your current ethics statement to address the following concerns:

a) Did participants provide their written or verbal informed consent to participate in this study?

b) If consent was verbal, please explain i) why written consent was not obtained, ii) how you documented participant consent, and iii) whether the ethics committees/IRB approved this consent procedure.

3. Peer review at PLOS ONE is not double-blinded (https://journals.plos.org/plosone/s/editorial-and-peer-review-process). For this reason, authors should include in the revised manuscript all the information removed for blind review

4. Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: Yes

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: An evaluation of the evidence submitted to Australian alcohol advertising policy consultations

Overview:

This is a very well designed, explained and executed piece of research that will no doubt contribute beneficially and insightfully to the field.

The research has been rigorous and is well contextualised and explained in relation to relevant literature, in particular on alcohol and secondarily, tobacco.

Quite often, questions that did arise were subsequently addressed in the later discussion.

The findings have yielded significant descriptive statistics that help substantiate the conclusions.

The comments below are minor and there were no issues found of any substance that would undermine a very positive evaluation of the standard and significance of the research. The authors are to be commended.

The Abstract is very good and gives a succinct summary of the research

I would recommend adding to the introductory Abstract paragraph ‘Evaluation of “The value of the evidentiary contribution of industry actors to consultations on alcohol

advertising policy appears to be limited”

Body of the article

p.8 of the MS pdf.

Re: ‘ls of cited evidence were extracted and coded against a framework adapted from previous research (primary codes: subject matter relevance, type of publication, time since publication, and independence from industry)’.

Q might the authors consider weighting the criteria with independence from industry the most highly weighted?

p. 12 of the MS pdf.

Line 115-16

Should say investigations by researchers [ie in order to differentiate this from governments]

method

p. 13 of the MS pdf.

Q why would human subject ethics permission be sought to examine publicly available submissions?

Perhaps other aspects of the research should be mentioned in this regard eg. Interviews or whatever necessitated human subject research.

p. 13 of the MS pdf.

the two inquiries that constitute the source of data are well described. However the authors could say why it was important. Eg was it to compare the two as justification that the results were not ideosyncratic to one inquiry? Was it to include both national and state inquiries etc.?

p.16 of the MS pdf.

Most of the codings are well explained and where relevant, the source in other relevant research. Regarding time since publication defined as up to 10 years or more than 10 years. Its there a rationale for this coding? Also, elsewhere in the text 15 years was mentioned. How does this fit the coding? [Also described on p.18 of the MS].

p. 19 MS pdf.

Line 276

‘Where detected, analyses were re-run with extreme scores removed to determine whether results differed.’

Q. isn’t it stated later that there were no cases of extreme scores?

p. 27 of MS pdf.

Principle findings brings together the most significant findings of the research.

The authors give a very good succinct summation of the main findings.

Final para

‘Modifications to consultation processes, such as exclusion of industry actors, quality requirements for submitted evidence, minimum standards for referencing evidence, and requirements to declare potential conflicts, may improve the transparency and application of evidence and public health outcomes of policy consultations’.

These are very reasonable recommendations that come out of this work with application more broadly to government inquiries. If submissions draw on evidence that is no publicly traceable for inspection/accuracy/bias, then this is a shortcoming that could seemingly be risk managed along lines suggested.

Under acknowledgments, this reviewer is not sure of the relevance of the declarations that presumably apply to different authors’ contributions to submissions that would have been included in the inquiry. Acknowledgments usually refer to funding sources and the sort of statements made here should perhaps go in potential limitations and how/why bias is/not introduced and also discuss whether the reported research in this article was designed after or contemporaneously with these organisational submissions in which authors were involved.

Alternatively, some of this may belong in declarations of interest. Since authors have contributed to some of the submissions to the inquires under analysis and in one case, to the final report. This does not mean to say the person should not be an author but to explain why these contributions do/not bias the research.

Reviewer #2: Many thanks for the opportunity to review this well written and important manuscript. The authors have conducted a thorough analysis of responses to two public consultations focusing on alcohol advertising policy using previously published methods. In this way the study is an important contribution to the growing body of literature on industry use and representation of evidence in the consultation process and adds a novel dimension by analysing and comparing industry with non-industry submissions.

I have only minor comments for the authors to consider addressing:

Page 5 – paragraph from lines 91-103 – the authors could extend this further by making reference to additional studies that show that similar tactics are employed by the ultra-process food industry and the sugar sweetend beverage industry e.g.

Fooks GJ, Williams S, Box G, Sacks G. Corporations’ use and misuse of evidence to influence health policy: a case study of sugar-sweetened beverage taxation. Globalization and Health 2019; 15(1): 56.

Lauber K, McGee D, Gilmore AB. Commercial use of evidence in public health policy: a critical assessment of food industry submissions to global-level consultations on non-communicable disease prevention. BMJ Global Health 2021; 6(8): e006176.

This could help to further demonstrate the cross-industry nature of these tactics.

Page 8 lines 167-172 – the authors describe who was classified as industry – where their other commercial actors e.g. those from the tourism/entertainment industry or bars/clubs (where these considered “retailers”) and where these included as industry or non-industry? This may be helpful in thinking though some of the wider partnership and allies that can be built by the industry and whose interests are served or threatened by different policy outcomes as the authors have shown for the advertising industry.

Page 22: “In both studies, significantly lower proportions of evidence cited by industry actors were highly relevant and featured indicators of higher quality compared to the points of comparison (i.e., systematic review evidence in Hatchard et al. and evidence cited by non-industry actors in the present study)”

The authors may want to add some addition text to clarify that industry submissions have lower levels of features indicating high quality.

Discussion/conclusion – the authors make several well-considered recommendations on how the consultation process may be strengthened. They may wish to highlight to the reader that similar recommendations have been made in other studies in the context of other policies and industries, which may strengthen the case for calling for such adjustments given other research groups in other locations and time points have also demonstrated the need for such measures if policy-making is to serve the public interest and draw on the best available evidence.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: Yes: Prof Linda Hancock

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

20 Nov 2021

Please see attached Response to Reviewers.

Submitted filename: Response to Reviewers.docx

Click here for additional data file.

29 Nov 2021

An evaluation of the evidence submitted to Australian alcohol advertising policy consultations

PONE-D-21-26197R1

Dear Dr. Stafford,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Quinn Grundy, PhD, RN

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

3 Dec 2021

PONE-D-21-26197R1

An evaluation of the evidence submitted to Australian alcohol advertising policy consultations

Dear Dr. Stafford:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Quinn Grundy

Academic Editor

PLOS ONE