Accuracy of popular media reporting on tobacco cessation therapy in substance abuse and mental health populations.

BACKGROUND: Tobacco cessation therapy is not consistently provided for alcohol, drug abuse and mental health (ADM) populations, despite the enormous health consequences of tobacco addiction in these groups and research supporting the effectiveness of treatment. Policymakers, however, tend to rely on popular media reports rather than the scientific literature in regulating treatment. Our goal was to determine whether popular reporting accurately reflects findings from the scientific literature on tobacco cessation treatment for ADM populations in treatment.
METHODS: We compared the results of systematic reviews on tobacco cessation therapy published before 2004 with articles published in traditional media and on the internet over the following 8 years. We searched LexisNexis and Google and assessed them using the Index of Scientific Quality (ISQ).
RESULTS: We found that popular reporting on this topic was consistent with findings reported in contemporaneous scientific literature. Our results suggest that the failure to consistently provide tobacco cessation therapy to ADM populations in treatment is not due to poor research translation.
CONCLUSIONS: Our findings also suggest that in this topic area, scientific research findings have diffused relatively quickly. Further study of journalism in this area may suggest new strategies for effective translation of scientific findings into popular reporting on tobacco control. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

This paper provides new evidence about translation of research from bedside to community in tobacco cessation therapy.

Our exploratory study of Internet reporting reveals new insights about the relative role of evidence in Web versus traditional print media.

Unlike a contemporaneous systematic review of scientific literature, we limited our inclusion criteria to articles published in English and drawn from a US news source.

Although many of the news articles and websites we reviewed based their conclusions on whether or not smoking cessation interventions resulted in smoking reductions of any amount, systematic reviews of tobacco cessation focus on point prevalence abstinence.

The majority of articles and websites we reviewed only categorised patients as alcoholics, drug abusers or those with mental illness, without discussing whether or not their conclusions depended on other patient characteristics such as age or sex.

Introduction

Patients receiving treatment for alcohol, drug abuse and mental health (ADM) problems are disproportionately affected by smoking. Tobacco use causes 435 000 annual deaths in the USA, and approximately 200 000 of these deaths occur in ADM populations, where smoking rates are 2–4 times greater than the general population.1–4 People with a mental illness or substance abuse disorder smoke nearly half of all the cigarettes smoked in the USA.5 Despite clinical evidence supporting the benefits of concurrent treatment and a desire and ability of patients to quit smoking,6–8 statewide policy diffusion has been slow and few states have mandated that mental health clinics and drug abuse centres require the provision of smoking cessation as a condition of licensure.9

There are many reasons for the poor translation of clinical evidence into policy, including the belief among healthcare providers that the health risks from smoking are less important than the perceived benefits of smoking, which are thought to calm psychiatric patients and reduce the risk of relapse.10 Other barriers include provider fears that trying to simultaneously quit smoking would compromise efforts to recover from other addictions,8 11 questions regarding the best time to integrate smoking cessation treatment, and the fact that many individuals who staff drug abuse clinics and psychiatric wards are smokers themselves.10

A conceptual model of research translation details the process from the generation of clinical research to the uptake of research by journalists and other policy intermediaries who communicate directly with policymakers. However, a fundamental missing element in understanding research translation has been identifying the extent to which scientific research moves onto the policy agenda by attracting the attention of journalists. To address this knowledge gap, we have situated our discussion around two seminal studies favouring concurrent treatment: (1) a previous meta-analysis by Prochaska et al12 which found smoking cessation therapy was associated with a 25% increased likelihood of long-term abstinence from alcohol and illicit drugs and (2) a report by Covey et al13 concluding that administering smoking cessation therapy to patients with and without major depression produces equivalent smoking quit rates. By comparing these clinical research findings with journalistic claims about smoking cessation therapy in ADM populations, we can identify the extent to which clinical research findings have been translated into general knowledge.

The rationale for understanding how evidence moves along the research translation pathway is to enact evidence-based policy. The general public, health professionals, and policymakers rely heavily on journalistic reports to inform their healthcare decisions and policies.14 Research on policymaking also details the critical importance of journalists to communication within policymaking networks and in shaping constituency opinions.15 16 Although partisanship, ideology, and maintaining consistent voting records all factor into policymakers’ decisions, policymakers also view the extent of public support for proposed policies as critical information in making decisions about whether to enact such changes. The extent and nature of press coverage of issues cues popular opinion and establishes the policy relevance of those issues,17–21 in particular increasing attention to existing problems and fostering demands for political action. Media messages are particularly relevant in establishing popular understanding of health risks and treatments.22–26 Journalists thus provide an independent source of information about the public relevance of proposed treatments to policymakers seeking to make decisions about systemic health interventions.27–30 However, media misunderstanding of research findings is common.23 26 31 32

Journalists are known to play an important information-gathering role for policymakers but the degree to which journalists communicate with clinical researchers is unclear. Previous studies comparing popular news reports about scientific research to the journal articles that inspired them show a lack of consistency between author conclusions.23 26 27 32 Furthermore, a review by HealthNewsReview.org covering 500 health news reports by US journalists determined that journalists often fail to discuss intervention costs, the validity of evidence presented, the magnitude of the findings (effects, risks, or costs), and alternative options.33 Consistent with these trends, we hypothesised (1) that popular reporting on smoking cessation relies on anecdotal evidence (as opposed to scientific) and will deviate from the scientific evidence and (2) that the majority of news reports we review are low-quality. If these hypotheses were correct, poor translation of research findings into the popular media could explain why policies on tobacco cessation are not informed by research evidence.

To test our hypotheses, we reviewed popular reporting of effects of smoking cessation therapy in ADM populations and relied on the Oxman et al14 instrument to assess report quality. We used both the Prochaska et al and Covey et al studies as a source of clinical evidence, which we compared with our review of popular reporting.

Methods

Our analysis of research translation relies on reports from the popular media. We reviewed print articles and websites archived in the public domain that were most likely to be easily found by individuals inexperienced with traditional academic research methods. To identify traditional media (ie, print media) we searched the LexisNexis database for newspaper and magazine articles. To mirror electronic search strategies and identify internet media, we ran keyword searches through the most popular online search engine, Google.

Inclusion criteria

We included articles indexed in LexisNexis and websites in Google that we identified using defined keyword searches (table 1). We included: (1) major newspaper, magazine article, wire service stories, broadcast transcripts, internet content from independent bloggers, or websites containing either scientific or anecdotal claims regarding the effectiveness of smoking cessation in ADM populations, that were (2) published in English and (3) drawn from a US news source. Websites containing links to peer reviewed articles, news reports, grey literature, or fact sheets were included as long as these additional resources met the inclusion criteria.

Table 1

Search strategy and terms for traditional and internet media

Dates of search	Search terms included	Search terms excluded*
Traditional media
July 2004—July 2012	((“tobacco cessation” OR “smoking cessation”) AND (comorbidity OR co-morbidity OR high-risk OR “high risk” OR “mental health” OR “drug abuse” OR “substance abuse” OR alcohol OR alcoholism))	“inhalant abuse”; “marijuana abuse”; “phencyclidine abuse”; “alcohol drinking”; “opioid-related disorders”; “cocaine-related disorders”; “amphetamine-related disorders”; “alcohol-related disorders”; “substance-related disorders”; “substance withdrawal syndrome”
Internet media
NA	Google Search #1: alcohol drug abuse and mental health smoking cessation Google Search #2: smoking cessation in alcoholics Google Search #3: smoking cessation alcoholism Google Search #4: smoking cessation mental health Google Search #5: smoking cessation drug abuse Google Search #6: smoking cessation therapy mental health Google Search #7: tobacco cessation therapy drug abuse Google Search #8: smoking cessation comorbid Google Search #9: smoking cessation high risk alcohol Google Search #10: smoking cessation high risk Google Search #11: smoking cessation high risk mental health Google Search #12: smoking cessation high risk drug abuse Google Search #13: tobacco control cessation and alcohol high risk comorbidity	Any links following the first 50 links provided by Google search

*Search terms excluded on the grounds that they did not yield additional articles.

Exclusion criteria

We excluded sources discussing smoking cessation therapy in ADM populations that assessed only non-smoking related health outcomes. For example, assessments of whether or not alcoholics receiving smoking cessation therapy reduced their intake of alcohol were not included in the final review.

Search strategy and article selection

Our search strategy and terms are provided in table 1. Studies were screened in two stages, as outlined in figure 1. Any articles or websites that did not clearly meet the criteria were discussed by two authors (DA and DK) for a final decision about inclusion.

Figure 1

Flow of included articles (traditional media only). N indicates the number of studies.a

Data extraction

The following information was extracted from each article or website by one reviewer (DK):

Title of the article

Website URL address (if applicable)

Publication type (eg, newspaper, magazine article, wire service stories, broadcast transcripts, website)

Publication date (month/year)

Subpopulation (ie, alcohol, drug abuse, mental health) receiving the intervention

Source of evidence (scientific or personal anecdote)

In coding for content, two reviewers (DA and DK) assessed whether the article claimed that smoking cessation therapy was effective, ineffective, or made no judgment. Both coders worked independently and any discrepancies were discussed. A third coder was available to adjudicate any discrepancies and make a final decision if the discrepancy could not be resolved. The quality of each article was assessed using the Index of Scientific Quality (ISQ).14 We determined whether the following quality criteria were included:

Applicability: Describes whether or not the author clarifies to whom the information in the report applies

Opinions versus Facts: Describes whether or not facts are clearly distinguished from opinions

Validity: Describes whether or not the assessment of the credibility (validity) of the evidence is clear and well-founded (not misleading)

Magnitude: Describes whether or not the strength or magnitude of the findings (effects, risks, or costs) that are the main focus of the article are clearly reported

Precision: Describes whether or not the author provides a clear and well-founded (not misleading) assessment of the precision of any estimates that are reported or of the probability that any of the reported findings might be due to chance

Consistency: Describes whether or not the consistency of the evidence (between studies) is considered and whether the assessment is well-founded (not misleading)

Consequences: Describes whether or not all of the important consequences (benefits, risks, and costs) of concern relative to the central topic of the report are identified

Global: Describes the overall scientific quality of the report

Results—Describes the qualitative (eg, personal anecdotes) and quantitative (eg, relative risk values) data related to the efficacy of implementing smoking cessation interventions in ADM populations

Author conclusion—While item number 9 extracts data on the actual results of the article, this item pertains to the author conclusions regarding the efficacy of implementing smoking cessation interventions in ADM populations which may or may not agree with the empirical or qualitative data that is reported.

The ISQ index uses a five-point scale with a 5 corresponding to the highest level of quality. A score of 4 or 5 pertains to criteria containing clear references to evidence, while a score of 2 or 3 represents partly or definitely unclear references to evidence. An ISQ score of 1 or 2 is assigned to criteria where the evidence base is potentially misleading. We assigned reports with low overall quality (corresponding to low quality for several ISQ criteria) a score of 1 or 2, and reports with high overall quality (corresponding to high quality for several ISQ criteria) a score of 4 or 5.

Analysis

We report the frequency of each item assessed and analyse the content of popular news reports by generating summary statistics based on the content scores for all articles. Each item was assigned equal weight even though each item could differ in the extent it influenced the quality of the article. We compared the results for the subgroup of stories from traditional media to the subgroup of stories identified using Google, and assessed quality rankings for the overall sample and the subgroups (traditional and internet media) by reporting average quality scores.

Comparison with scientific literature

We compared our findings to (1) a previously published meta-analysis12 which provided an empirical basis for supporting concurrent smoking cessation treatment in alcohol and drug abuse populations and (2) an earlier report which showed similar benefits for patients with depression.13

Results

As shown in figure 1, we identified 10 216 potentially relevant articles in LexisNexis, of which 26 met our inclusion criteria and were included for analysis. Similarly, 650 websites were screened in Google, of which 26 met our inclusion criteria and were included for analysis.

Traditional media

Overall, 17 articles focused solely on tobacco cessation therapy in patients with mental health issues, one article focused solely on alcoholic patients, and one article focused solely on drug abuse patients. Four articles focused on tobacco cessation in patients having either alcohol, drug abuse, or mental health problems, two articles were limited to alcohol and drug abuse patients, and one article was limited to mental health and drug abuse patients. Of these articles, two were published in 2004, 1 in 2005, 5 in 2006, 4 in 2007, 3 in 2008, 2 in 2009, 4 in 2010, 3 in 2011, and 2 in 2012.

Among the 26 articles meeting our inclusion criteria in our LexisNexis search, 25 contained author conclusions that supported the use of concurrent tobacco cessation therapy in ADM populations and one article contained an ambiguous conclusion. Nineteen articles contained results that were based on quantitative data, four articles contained results based on minimal evidence, two articles did not contain any evidence and one article was supported by anecdotal claims.

Article quality

In table 2, we provide the mean values and SDs for each ISQ quality criteria for traditional and internet media. Each article clearly stated the generalisability of its results—‘applicability’ received an average score of 5. The item ‘opinions versus facts’ averaged 4.2 with 16 of 26 articles scoring a 5 for this category. An average score of 3.5 was assigned to the item consequences. Similarly, authors described the ‘magnitude’ of treatment effect infrequently—an average score of 3.4 was assigned to this item with 13 of 26 articles scoring a 5 for this item and 7 articles scoring a 1. The item “validity” received an average score of 2.4 with only 1 article scoring a 5 for this item and 8 articles scoring a 1. The item ‘consistency’ received an average score of 1.9 with individual scores ranging from 1 to 4, while the item ‘precision’ received a score of 1.7 with individual results ranging from 1 to 3. Overall, the average ‘global’ score across all studies was 3.2. We provide a table of the data including the media sources and matrix of the review criteria in online supplementary table S1.

Table 2

Summary statistics for article quality in traditional and internet media

	Applicability	Opinions versus facts	Validity	Magnitude	Precision	Consistency	Consequences	Global
Traditional media	5.00±0.00	4.15±1.19	2.42±1.06	3.42±1.75	1.69±0.97	1.92±1.06	3.54±1.30	3.16±0.61
Internet media	4.89±0.65	4.58±0.90	2.68±1.10	3.53±1.26	2.32±1.50	2.89±1.39	3.68±1.33	3.51±0.79

Values are mean±SD.

Source: Data collected by the authors.

Internet media

Overall, 10 websites contained articles that focused solely on tobacco cessation therapy in patients with mental health issues, five articles focused solely on alcoholic patients, and no articles focused solely on drug abuse patients. Seven articles focused on tobacco cessation in patients having either alcohol, drug abuse or mental health problems, no articles were limited to alcohol and drug abuse patients, three articles were limited to mental health and drug abuse patients, and one article was limited to alcohol and mental health patients. Also, 13 of 26 websites assessed were published in the past 3 years, eight websites contained articles published between 2005 and 2009, and five websites contained articles without a publication date.

All 26 articles identified using Google contained author conclusions that supported the use of concurrent tobacco cessation therapy in ADM populations. Of these 26 articles, 23 articles contained results that were based on quantitative data, two articles contained results based on minimal evidence, and one article contained results based on quantitative and anecdotal evidence.

Twenty-four of 26 articles clearly stated the generalisability of its results—‘applicability’ received an average score of 4.9. The item ‘opinions versus facts’ averaged 4.6 with 21 of 26 articles scoring a 5 for this category. An average score of 3.7 was assigned to the item ‘consequences’. Similarly, most authors described the ‘magnitude’ of treatment effect only partially—an average score of 3.5 was assigned to this item with 11 of 26 articles scoring a 5 for this item, 13 articles scoring a 3 for this item, and two articles scoring a 1. The item ‘validity’ received an average score of 2.7 with only one article scoring a 5 for this item and 7 articles scoring a 1. The item ‘consistency’ received an average score of 2.9 with individual scores ranging from 1 to 5, while the item ‘precision’ received a score of 2.3 with individual results ranging from 1 to 5. Overall, the average ‘global’ score across all studies was 3.5. Mean and SD quality data is presented in table 2. We provide a table of the data including the media sources and matrix of the review criteria in online supplementary table S2.

Overall, the 2004 meta-analysis by Prochaska et al12 found that intervention effects for smoking cessation were significant at post-treatment for alcoholic and drug abuse patients, but were no longer significant at 6–12 months follow-up. Specifically, patients receiving smoking cessation therapy experienced a twofold (relative risk of 2.03) increase in smoking abstinence following treatment. Our review found that popular reporting of tobacco cessation therapy in treatment matches the findings from Prochaska's study; authors reported a favourable conclusion regarding concurrent treatment of tobacco addiction in ADM patients in 25 of 25 articles or websites.

The report by Covey et al13 found that quit rates for smokers receiving smoking cessation therapy were the same for patients with and without major depression and the depressive episodes were not adversely affected by the smoking intervention, a finding that supports concurrent treatment. Combining results for men and women, Covey reported that 13 of 49 (27%) non-alcoholic patients with major depression successfully quit smoking while 30 of 110 (27%) non-alcoholic patients without major depression successfully quit smoking. Likewise, we found that popular reporting expressed support for concurrent treatment of tobacco addiction in psychiatric patients in 26 of 27 articles or websites.

Discussion

Although we hypothesised that popular reporting of smoking cessation interventions for ADM populations would rely on anecdotal evidence, we found the opposite to be true. Most articles relied solely on quantitative estimates of tobacco reduction or abstinence. Moreover, consistent with the scientific literature, all but one article presented a favourable conclusion regarding smoking cessation therapy for ADM patients. Author conclusions were similar for LexisNexis and Google.

Our average quality scores for individual items and for pooled items (global) were also comparable for LexisNexis and Google. The item ‘applicability’ differed by only 0.1 (ie, 5.0 vs 4.9 for LexisNexis and Google, respectively). The item ‘consistency’ had the largest difference (ie, 1.9 vs 2.9 for LexisNexis and Google, respectively).

The publication dates for our LexisNexis articles or Google websites ranged from 2004 to 2012. 17 of 26 LexisNexis articles were published before 2010. However, only 8 of 26 Google websites were published before 2010. Although this would suggest that clinical findings are diffused into traditional print media (ie, LexisNexis) faster relative to Web media (ie, Google), it is possible we missed earlier publications archived by Google since we only reviewed the first five web pages from each search.

Study limitations

A limitation of our study is that we may not have identified all published articles in LexisNexis. As we limited our inclusion criteria to only articles published in English and drawn from a US news source, it is possible some articles were missed. Moreover, we cannot rule out the possibility that non-US news reports could have higher quality or different author conclusions regarding concurrent smoking cessation treatment in ADM populations. The systematic review of scientific literature written by Prochaska et al12 did not limit their inclusion criteria to English only articles or studies based in the USA.

Our comparison with the scientific evidence is further limited by the fact that the contemporaneous systematic review12 only included articles that reported point prevalence abstinence, excluding articles that only reported on smoking reduction. However, many of the news articles and websites we reviewed based their conclusions on whether or not smoking cessation interventions resulted in smoking reductions of any amount. Our comparison with the article by Covey et al13 is limited since that article only assessed smoking cessation therapy effects in patients who had major depression. However, most of our findings from LexisNexis or Google did not describe which mental health condition patients undergoing smoking cessation had, or described a different mental health condition such as schizophrenia.

Moreover, the patient backgrounds were adequately described by both Prochaska et al12 and Covey et al 13 and patients not meeting the a priori defined inclusion criteria were excluded. The majority of articles and websites we reviewed only categorised patients as alcoholics, drug abusers or those with mental illness, without discussing whether or not their conclusions depended on other patient characteristics such as age or sex.

Conclusion

Our findings provide new evidence about translation of research from bedside to community in tobacco cessation therapy. We present two new innovations in discussing popular reporting on health: we compare popular reporting to a contemporaneous systematic review, and include online media as well as traditional reporting. We find that popular reporting on this topic is high-quality and mirrors the results of existing clinical data. Our results also suggest that scientific research on this topic diffused relatively quickly into popular reporting. Further study of journalists reporting in this area might identify useful research translation strategies for other areas of tobacco control. Finally, our exploratory study of Internet reporting provides new insights about the relative role of evidence in Web versus traditional print media. Our results suggest that the continuing limited provision of tobacco cessation therapy in drug abuse and mental health treatment is not due to poor research translation.