Influence of medical journal press releases on the quality of associated newspaper coverage: retrospective cohort study.

OBJECTIVE: To determine whether the quality of press releases issued by medical journals can influence the quality of associated newspaper stories.
DESIGN: Retrospective cohort study of medical journal press releases and associated news stories.
SETTING: We reviewed consecutive issues (going backwards from January 2009) of five major medical journals (Annals of Internal Medicine, BMJ, Journal of the National Cancer Institute, JAMA, and New England Journal of Medicine) to identify the first 100 original research articles with quantifiable outcomes and that had generated any newspaper coverage (unique stories ≥100 words long). We identified 759 associated newspaper stories using Lexis Nexis and Factiva searches, and 68 journal press releases using Eurekalert and journal website searches. Two independent research assistants assessed the quality of journal articles, press releases, and a stratified random sample of associated newspaper stories (n=343) by using a structured coding scheme for the presence of specific quality measures: basic study facts, quantification of the main result, harms, and limitations. MAIN OUTCOME: Proportion of newspaper stories with specific quality measures (adjusted for whether the quality measure was present in the journal article's abstract or editor note).
RESULTS: We recorded a median of three newspaper stories per journal article (range 1-72). Of 343 stories analysed, 71% reported on articles for which medical journals had issued press releases. 9% of stories quantified the main result with absolute risks when this information was not in the press release, 53% did so when it was in the press release (relative risk 6.0, 95% confidence interval 2.3 to 15.4), and 20% when no press release was issued (2.2, 0.83 to 6.1). 133 (39%) stories reported on research describing beneficial interventions. 24% mentioned harms (or specifically declared no harms) when harms were not mentioned in the press release, 68% when mentioned in the press release (2.8, 1.1 to 7.4), and 36% when no press release was issued (1.5, 0.49 to 4.4). 256 (75%) stories reported on research with important limitations. 16% reported any limitations when limitations were not mentioned in the press release, 48% when mentioned in the press release (3.0, 1.5 to 6.2), and 21% if no press release was issued (1.3, 0.50 to 3.6).
CONCLUSION: High quality press releases issued by medical journals seem to make the quality of associated newspaper stories better, whereas low quality press releases might make them worse.

Introduction

Media coverage of medical research often fails to provide the information needed for the public to understand the findings and to decide whether to believe them.1 2 3 4 5 6 7 Although it is easy to blame journalists for poor quality reporting, problems with coverage could begin with the journalists’ sources. For example, relevant information in medical journal articles might be missing or difficult to find. Similarly, press releases—the most direct way by which sources (such as the pharmaceutical industry, academic medical centres, and medical journals) communicate with the media—often omit key facts, and fail to acknowledge important limitations.8 9 10

Nevertheless, press releases do have some impact. They can increase the chance of a journal article receiving media coverage.11 12 13 14 15 And journalists seem to find press releases useful;13 15 in fact, some appear to rely on them. An independent news rating website concluded that up to a third of health news stories relied solely or largely on press releases.5

However, the influence of press releases on the quality of subsequent media coverage is unknown. To understand this association, we studied press releases issued by medical journals since they are such a regular and trusted feature of journalists’ work.15 We used a retrospective cohort design to learn how the presence of key information in medical journal press releases—such as quantifying the main result or highlighting important research limitations—influences the presence of that information in subsequent newspaper stories.

Methods

Sample selection

Medical journal articles

Figure 1 shows how we identified original research articles generating newspaper coverage. We reviewed consecutive issues (going backwards from January 2009) of five major medical journals: Annals of Internal Medicine (Annals), British Medical Journal (BMJ), Journal of the National Cancer Institute (JNCI), Journal of the American Medical Association (JAMA), and the New England Journal of Medicine (NEJM). We selected these journals because their articles frequently receive news coverage, yet their editorial practices differ: Annals and JNCI articles include editor notes highlighting study cautions (whereas BMJ, JAMA, and NEJM articles do not), and NEJM is the only journal in the group that does not issue press releases.

Fig 1 Study selection of journal articles, press releases, and newspaper stories

To rate newspaper stories on how they quantified results, we included only journal articles with straightforward quantifiable outcomes (that is, we excluded qualitative studies, case reports, biological mechanism studies, and studies not using an individual unit of analysis). To identify associated newspaper stories, we searched Lexis Nexis and Factiva (news article databases) for stories that included the medical journal name (the time frame for the search extended from two months before the journal article’s print date, to two months after).

We reached our sample size target (100 journal articles) after reviewing 230 potentially eligible journal articles published from January 2009 back to July 2008. Assuming an average of three newspaper stories per study and an intraclass correlation coefficient of 0.5 to account for the clustering of newspaper stories within a study, we needed 40 journal articles to detect an odds ratio of 9 for absolute risks in the story and 48 articles to detect an odds ratio of 4.9 for mentioning harms (assuming α of 0.05 and 80% power).

Medical journal press releases and newspaper coverage

To identify press releases issued by medical journals, we searched www.eurekalert.org, a press release database. Of the 100 journal articles identified, 68 had an associated press release. To ensure that no press releases were missed, we also checked each journal’s website.

We identified 759 unique newspaper stories reporting on the 100 journal articles; 76 additional stories were identified but excluded because they were exact duplicates, too brief (<100 words long), or editorials (fig 1). For the detailed assessment of reporting quality, we selected a random sample of 343 newspaper stories. We included up to six stories for each journal article. For journal articles with more than six stories, we selected a stratified random sample: three stories from major newspapers and three stories from minor newspapers. If one category (for example, minor newspapers) did not have enough stories, we substituted stories from the other category (for example, major newspapers). We classified international newspapers as “major” if they appeared on the Lexis Nexis’s list of major world newspapers, and newspapers from the United States as “major” if they appeared on BurrellesLuce’s list of top 20 US daily newspapers.16 Minor newpapers did not appear in these two lists.

Quality assessment

We created structured coding forms for the quality assessment of the journal articles (we coded only the abstract and, if available, the editor note), press releases, and a random sample of newspaper stories (web appendix). The coding forms were developed and refined during a pilot study. Each form included specific quality measures on the reporting of basic study facts, main results, harms, and limitations (box).

Study size, funding source(s), randomised trials identified, longitudinal study time frames, survey response rates, and accurate description (compared with the abstract) of study exposure and outcome.

Only the first four measures had sufficient variability to compare the quality of press releases and newspaper stories. The accuracy of exposures, outcomes, and survey response rates varied too little across newspaper stories, appearing in 99%, 97%, and 0% of stories, respectively.

Was the main result quantified? If so, was it quantified with any absolute risks (including proportions, means, or medians)?

Were numbers used correctly (for example, did the numbers reported correspond to those in the abstract in terms of magnitude and time frame, or were odds or odds ratios misinterpreted as risks or risk ratios? The odds ratio issue was included under the category of numbers used correctly rather than as a separate measure.

For studies of interventions claiming to be beneficial:

Were harms mentioned (or a statement asserting there were no harms)?

Were harms quantified? If so, were they quantified with any absolute risks?

We did not present data for the correct use of harm numbers, because too few newspaper stories quantified harms (that is, fewer than five stories in each press release category).

Were any major study limitations noted?

We considered limitations noted in either the journal abstract or editor note, or on a design specific list that we created (web appendix) that includes limitations inherent in various study designs: small study size, inferences about causation, selection bias, representativeness of sample, confounding, clinical relevance of surrogate outcomes, clinical meaning of scores or surrogate outcomes,, hypothetical nature of decision models, applicability of animal studies to humans, clinical relevance of gene association studies, uncontrolled studies, studies stopped early because of benefit, and multifactorial dietary or behavioural interventions.

We presented data only for two major limitations (confounding and the clinical meaning of scores or surrogate outcomes), because the other limitations occurred too infrequently in newspaper stories for inclusion (that is, in fewer than five stories in each press release category).

To assess the reliability of the coding forms, two research assistants at doctoral level independently coded all the journal articles and press releases, and the sample of newspaper stories. We compared their responses by calculating κ statistics. For the three coding forms, the mean κ value was 0.78 (range 0.49-1.00) for the journal articles, 0.71 (0.46-1.00) for the press releases, and 0.71 (0.49-1.00) for the newspaper stories. Disagreements were resolved by discussion between the assessors or with the other authors (if the assessors did not reach consensus easily).

Analysis

We used multiple regression (the unit of analysis was the newspaper story) to assess the association between the quality of press releases (exposure) and quality of newspaper stories (outcome). The main outcome was the proportion of newspaper stories reporting on each of the quality measures (for example, if the story used absolute risks to report results, or noted study limitations). The exposure variable had three levels: press release without the quality measure under consideration (reference category), press release with the quality measure, and no press release issued. To isolate the effect of the press release, we adjusted the analyses depending on whether the quality measure was present in the abstract or editor note of the journal article.

Because newspaper stories are clustered within journal articles (that is, different stories can report on the same article), we used generalised linear models (STATA 11.0). We fit Poisson regression models assuming a binary distribution and a log link to estimate adjusted relative rates and proportions. Generalised linear models incorporate variance overdispersion in the estimates of all standard errors to account for the clustering, since quality measures in newspaper stories about the same medical journal article could be correlated.17 Statistical tests were two sided and calculated at the 5% level of significance.

Results

Medical journal articles and press releases

The 100 original research articles identified in our search (that is, those with quantifiable outcomes published in five major medical journals and receiving newspaper coverage) were equally divided between observational studies and randomised trials or meta-analyses (table 1). Sixty eight articles had an associated press release (of the remaining 32, 25 were published in the NEJM which, by policy, does not issue press releases).

Table 1

 Description of medical journal articles and associated newspaper stories, by journal. Data are number or number (%), unless indicated otherwise

	Medical journal
	Annals	JNCI	BMJ	JAMA	NEJM	All
Medical journals
Editorial policy
Editor notes included in articles	Yes	Yes	No	No	No	NA
Press releases issued	Yes	Yes	Yes	Yes	No	NA
Journal articles
All	10	20	20	25	25	100
Has abstract	10 (100)	20 (100)	20 (100)	25 (100)	25 (100)	100 (100)
Has editor note	10 (100)	19 (95)	0	0	0	29 (29)
Study design
Meta-analyses	0	2 (10)	1 (5)	6 (24)	0	9 (9)
Randomised trials	2 (20)	3 (15)	6 (30)	9 (36)	19 (76)	39 (39)
Observational studies (with controls)	8 (80)	11 (55)	13 (65)	10 (40)	3 (12)	45 (45)
Observational studies (without controls)	0	0	0	0	0	0
Gene association studies	0	1 (5)	0	0	3 (12)	4 (4)
Decision models	0	1 (5)	0	0	0	1 (1)
Animal studies	0	2 (10)	0	0	0	2 (2)
Study characteristics
No of participants (median, range)*	2234 (65-61 752)	8171 (413-2 788 715)	1892 (25-161 808)	3037 (57-152 084)	1050 (18-259 387)	2126 (18-2 788 715)
Industry funding
None	8 (80)	16 (80)	15 (75)	12 (48)	5 (20)	56 (56)
Of study	2 (20)	0	1 (5)	1 (4)	13 (52)	17 (17)
Of investigators†	0	4 (20)	4 (20)	12 (48)	7 (28)	27 (27)
Primary outcome surrogate	2 (20)	2 (10)	1 (5)	2 (8)	1 (4)	8 (8)
Beneficial intervention	4 (40)	6 (30)	6 (30)	11 (44)	13 (52)	40 (40)
Associated press release issued	8 (80)	20 (100)	17 (85)	23 (92)	0	68 (68)
Newspaper stories
All	51	104	232	201	171	759
Reported in top 5 US circulation newspapers‡	4 (8)	9 (9)	6 (3)	17 (8)	20 (12)	56 (7)
Reported in top 5 UK circulation newspapers§	1 (2)	3 (3)	18 (8)	8 (4)	6 (4)	36 (5)
Stories appearing on page 1¶	3 (6)	11 (11)	8 (3)	14 (7)	28 (16)	64 (8)
No of stories per journal article (median, range)	2 (1-19)	3 (1-34)	5 (1-72)	3 (1-34)	2 (1-36)	3 (1-72)
Random sample selected for quality assessment	31	61	79	86	86	343

Medical journals are listed according to editorial policy. NA=not applicable.

*Includes 93 articles about primary human research (excluding secondary analyses of population data).

†Investigators disclosed personal funding support from industry (for example, as consultant or lecturer, in the form of honorariums or past salary support).

‡Newspapers with the highest circulation (daily and Sunday editions combined) in the USA in 2008, according to Burrelles-Luce (www.burrellesluce.com/top100/2008_Top_100List.pdf): the New York Times, Wall Street Journal, USA Today, Washington Post, and Los Angeles Times.

§ Newspapers with the highest circulation (daily and Sunday editions combined) in the UK in 2009, according to the Audit Bureau of Circulations (www.guardian.co.uk/media/page/abcs2009): the Sun, Daily Mirror and Sunday Mirror, Daily Mail and Sunday Mail, Sunday Times, and News of the World.

¶We could not determine the page number for 63 newspaper stories, so data might be underestimated.

Table 2 compares the quality of reporting in journal article abstracts (including editor notes) with the quality in associated press releases. Quality was similarly high with respect to basic study facts (for example, study size was reported in 94% of abstracts v 88% of press releases) and similarly low for mentioning and quantifying the harms of intervention (43% v 35% and 20% v 13%, respectively). However, significantly more abstracts than press releases quantified the main result with absolute risks (50% v 34%, P=0.04) and mentioned any research limitations (71% v 40%, P=0.004).

Table 2

 Quality of reporting in journal article abstracts, journal press releases, and associated newspaper stories. Data are number or number (%)

Quality measure	Quality measure reported
	Abstracts	Press releases	Newspaper stories
Basic study facts
All	100	68	343
Accurate description of exposure*	NA	68 (100)	341 (99)
Accurate description of outcome*	NA	67 (99)	328 (96)
Study size mentioned	94 (94)	60 (88)	258 (75)
Funding source mentioned	2 (2)	2 (3)	39 (11)
Randomised trials	45	22	157
Randomised trials clearly identified	43 (96)	20 (91)	30 (19)
Surveys	10	10	39
Survey response rate mentioned	3 (30)	2 (20)	0
Longitudinal studies	87	57	291
Timeframe mentioned	72 (83)	40 (70)	160 (55)
Main result
All	100	68	343
Main result quantified	99 (99)	53 (78)	240 (70)
Numbers used correctly†	NA	49 (92)	207 (86)
Main result quantified with absolute risks	50 (50)	23 (34)	78 (23)
Harms
Studies of beneficial interventions	40	23	133
Harms mentioned‡	17 (43)	8 (35)	54 (41)
Harms quantified	8 (20)	3 (13)	16 (12)
Harms quantified with absolute risks	4 (10)	2 (9)	11 (8)
Study limitations
Studies with important research limitations§	75	57	256
Any limitation mentioned	53 (71)	23 (40)	73 (29)
Design specific limitations§
Observational studies making causal claims	43	34	147
Confounding mentioned	26 (60)	9 (26)	28 (19)
Studies with surrogate primary outcome	18	11	60
Clinical meaning of outcome mentioned	6 (33)	5 (45)	13 (22)

NA=not applicable.

*That is, whether the exposure or outcome in the press release matched those described in the journal abstract.

†Denominator is number of documents quantifying main result.

‡Also includes statements declaring that no harms were observed.

§Included any limitation mentioned in abstract, press release, or predefined list (web appendix).

Newspaper stories

Our selection of 100 journal articles generated 759 newspaper stories—a median of three news stories per article (range 1-72). Of the associated stories, 7% and 5% appeared in the top five newspapers with the highest circulation in the USA and the United Kingdom, respectively (table 1). Overall, 8% of stories were reported on page 1 of the newspaper (table 1). Almost all stories accurately reported the exposure and outcome (table 2). However, stories were missing important information: for example, only 23% quantified the main result using absolute risks, 41% mentioned the harms of beneficial interventions, and 29% mentioned any study limitation.

Association between quality of press releases and quality of newspaper stories

Of 343 newspaper stories reviewed for the quality assessment, 245 (71%) reported on journal articles for which the medical journals had issued a press release.

High quality press releases versus low quality press releases

For all 13 quality measures, newspaper stories were more likely to report the measure if the relevant information appeared in a press release (that is, a high quality press release) than if the information was missing (P=0.0002, sign test; fig 2). This association was significant in separate comparisons for nine quality measures (fig 2). For example, when the press release did not quantify the main result with absolute risks, 9% of the 168 newspaper stories provided these numbers. By contrast, when the press release did provide absolute risks, 53% of the 77 newspaper stories provided these numbers (relative risk 6.0, 95% confidence interval 2.3 to 15.4). Because the accessibility of absolute risks varies with study design, we repeated this analysis for randomised trials only, in which absolute risks are always easily accessible. For the 140 newspaper stories reporting on randomised trials, the influence of press releases reporting absolute risks was the same as the main analysis (relative risk 5.8, 95% confidence interval 2.2 to 15.5).

Fig 2 Association between quality of medical journal press releases and quality of associated newspaper stories. Proportions (%) of stories with specific quality measures adjusted for whether measure was in journal article abstract or editor’s note. The quality measure of whether a randomised trial was clearly identified not included because the adjustment model would not converge (crude proportions of stories with measure: 0% (press release does not identify trial), 29% (press release identifies trial), and 9% (no press release)). *Significant association. †Relative risks with 0 as numerator or denominators calculated by Haldane’s method for bias correction of small samples18

Of the newspaper stories reporting on beneficial interventions, 24% mentioned harms (or specifically declared none) when harms were not mentioned in the press release, compared with 68% when mentioned in the press release (relative risk 2.8, 95% confidence interval 1.1 to 7.4). Of the stories reporting on research with important limitations, 16% reported limitations if limitations were not mentioned in the press release, compared with 48% if mentioned in the press release (3.0, 1.5 to 6.2).

We also tested for a journal effect, because journals that tend to issue high quality press releases might take further steps to improve the quality of subsequent news coverage irrespective of issuing a press release. We re-ran our models by adding an indicator term for each journal. Controlling for a journal effect made little difference to the magnitude and significance of the association between the quality of press releases and the quality of newspaper stories (models for infrequent events failed, such as quantifying harms, quantifying harms with absolute risks, or mentioning confounding or the clinical meaning of surrogate outcomes).

Presence of a quality measure in the press release had a stronger influence on the quality of associated newspaper stories than its presence in the abstract (table 3). The association between information in the press release and in the story was significant for seven of the 12 quality measures that could be compared (table 3). The corresponding association between information in the abstract and in the story was significant for two of the 12 measures. In absolute terms, the independent effect of information in the press release was larger than the effect of information in the abstract for eight of 12 measures. The only measure for which the absolute effect of the abstract was greater than that of the press release was for quantifying the main result (although press releases were substantially more influential for quantifying the main result with absolute risks).

Table 3

 Independent effect of quality measure in press releases and journal abstracts (or editor notes) on the quality of associated newspaper stories

Quality measure	Newspaper stories reporting quality measure (%)
	Quality measure in press release?				Quality measure in journal abstract?
	Yes	No	Relative risk (95% CI)		Yes	No	Relative risk (95% CI)
Basic study facts
All stories (n)	217	28	-		325	18	-
Study size mentioned (%)	78	37	2.1 (0.80 to 5.5)		76	35	2.1 (0.88 to 5.2)
All stories (n)	238	7	-		331	12	-
Funding source mentioned (%)	42	7	6.2 (1.4 to 27.0)*		10	10	1.0 (0.24 to 4.5)
Stories about randomised trials (n)	82	8			325	18	-
Randomised trial clearly identified (%)†	29	0	5.3 (0.3 to 79)		20	0	7.6 (0.5 to 118.7)
Stories about longitudinal studies (n)	158	47			254	37	-
Time frame mentioned (%)	58	21	2.7 (0.78 to 9.6)		54	30	1.8 (0.79 to 4.1)
Main result
All stories (n)	203	42	-		337	6	-
Main result quantified (%)	61	31	2.0 (1.1 to 3.7)*		69	0	9.7 (0.7 to 140)
All stories (n)	77	168	-		159	184	-
Main result quantified with absolute risks (%)	53	9	6.0 (2.3 to 15.4)*		18	15	1.2 (0.59 to 2.4)
Harms
Stories about beneficial interventions (n)	30	54	-		62	71	-
Harms mentioned (%)	68	24	2.8 (1.1 to 7.4)*		43	30	1.5 (0.72 to 2.8)
Stories about beneficial interventions (n)	13	71	-		28	105	-
Harms quantified (%)	18	3	5.5 (0.85 to 35.4)		23	4	6.3 (2.6 to 15.6)*
Stories about beneficial interventions (n)	7	77	-		28	105	-
Harms quantified with absolute risks (%)	43	3	14.1 (1.9 to 105)*		19	4	4.9 (2.0 to 12.3)*
Study limitations
Stories about studies with important research limitations (n)	89	106	-		189	67	-
Limitation mentioned (%)	48	17	3.0 (1.5 to 6.2)*		28	18	1.6 (0.74 to 3.4)
Stories about observational studies making causal claims (n)	43	74	-		99	48	-
Confounding mentioned (%)	24	10	2.5 (0.9 to 7.1)		0	0	0.5 (0.1 to 24)
Studies with surrogate primary outcome (n)	20	22	-		22	38	-
Clinical meaning of outcome mentioned (%)	40	0	18.6 (1.1 to 303)*		0	0	1.7 (0.03 to 82.6)

Data are from models that include variables of both the press releases and journal abstracts and account for clustering within each journal. The quality measure of numbers quantifying main result correctly is not included since it is defined by what is in the abstract. Percentages are adjusted values according to whether relevant information was present in abstract or editor notes, unless indicated otherwise.

*Significant association.

†Crude values are presented because the adjustment model would not converge.

‡Relative risks with 0 as numerator or denominator were calculated by Haldane’s method for bias correction of small samples.24

No press releases versus low quality press releases

Figure 2 shows that for 12 of the 13 quality measures, newspaper stories were more likely to report the measure when no press release was issued than when a press release was issued that did not include the quality measure (P=0.003, sign test). Three comparisons were significant, which favoured no press release over a press release of low quality: mentioning funding, using numbers correctly, and mentioning the clinical meaning of surrogate outcomes (fig 2).

High quality press releases versus no press releases

The quality of subsequent newspaper reporting with a press release of high quality compared with no press release being issued was roughly the same for basic study facts. However, press releases of high quality had a stronger influence than no press release for: quantifying the main result with absolute numbers (53% v 20%, P=0.001), mentioning harms of beneficial interventions (68% v 36%, P=0.01), and limitations of research with important limitations (48% v 21%, P=0.06; fig 2).

Discussion

The news media matter: medical reporting not only educates but also influences health beliefs and behaviours.19 Our study shows that press releases are important as well. Higher quality press releases issued by medical journals were associated with higher quality reporting in subsequent newspaper stories. In fact, the influence of press releases on subsequent newspaper stories was generally stronger than that of journal abstracts. Fundamental information such as absolute risks, harms, and limitations was more likely to be reported in newspaper stories when this information appeared in a medical journal press release than when it was missing from the press release or if no press release was issued. Furthermore, our data suggest that poor quality press releases were worse than no press release being issued: fundamental information was less likely to be reported in newspaper stories when it was missing from the press release than where no press release was issued at all (although the findings were generally not statistically significant).

Limitations of study

Our results should be interpreted in the light of several limitations. The first concern in any observational study is the extent to which unmeasured confounding could explain the results. Although the association between quality of the press releases and quality of the newspaper stories was independent of the quality of the journal abstracts, other confounding factors might exist. For example, journals that tend to issue high quality press releases might take further steps to improve subsequent news coverage, such as improving the readability of the article text, reducing the amount of complex research published, or increasing the authors’ accessibility to the media. We found no evidence for confounding by such a journal effect.

Journalists might also be influenced by non-journal press releases, although such press releases are unlikely to improve the quality of news coverage since numbers and cautions are rarely mentioned in press releases issued by academic medical centres10 or industry.8 Irrespective of quality, non-journal press releases cannot confound our observations because they are issued after journal press releases. We can also exclude other potential confounders that relate to the quality of newspaper stories but that are not plausibly related to the quality of press releases issued by medical journals, such as interviews with investigators or experts, or the journalist’s ability.

Moreover, the repetition of identical information from the journal press release in the newspaper story helps to support a causal association. For example, of 240 stories that quantified results, 101 (42%) reported a number appearing in both the press release and journal abstract. But newspaper stories were almost four times as likely to report numbers that appeared only in the press release compared with numbers appearing only in the journal abstract (62 (26%) v 17 (7%), P≤0.001).

Secondly, we only looked at the association between journal press releases and newspaper stories—we did not examine broadcast, web, and social media platforms. A recent survey by the Pew Research Center showed that, although newspaper circulation has contracted recently, print media are still a major source of health news.20 Nevertheless, non-print sources are gaining in impact, particularly web based media. The effect of press releases on the coverage of medical journal articles in these media is unknown. In view of concerns about the quality of non-print media (such as a lack of dedicated staff for health news, rapid deadlines, and limited or no editing), journal press releases could have a stronger effect in these new media than in the print media.

Thirdly, some subjectivity is inherent in any content analysis. We tried to maximise objectivity by creating and extensively pilot testing a structured coding scheme. We observed a high degree of agreement in our formal assessment of intercoder reliability.

Fourthly, we could have introduced bias from our use of stratified random sampling to select newspaper stories for the content analysis. However, substantial bias is unlikely for two reasons. The distribution of coverage between major and minor newspapers for all 759 associated stories and the 343 stories in the sample was similar (39% (296/759) v 47% (161/343) appeared in major newspapers). Furthermore, stratified analyses of major and minor newspapers yielded similar observations as the main analysis (that the presence of relevant information in press releases was associated with its presence in associated newspaper stories).

Press releases issued by medical journals vary in quality, often omitting key facts and failing to acknowledge important study limitations

The influence of press release quality on the quality of subsequent newspaper coverage is unknown

Press releases of high quality seem to improve the quality of associated newspaper stories, whereas press releases of low quality might make newspaper stories worse.

Presence of key information in press releases had a stronger influence on the quality of newspaper stories than the presence of the same information in journal abstracts

Medical journals should use press releases not simply to make medical news, but to make news reporting better.

Conclusions and policy implications

We found that newspaper stories reporting on medical journal research frequently failed to quantify the main results with absolute numbers and failed to note harms of interventions, study limitations, or study facts—findings consistent with previous investigations.1 2 3 5 6 7 The fact that medical journal abstracts often lack this information too is disappointing, as noted in our study and in other studies evaluating the quality of reporting in abstracts.21 22 23 Abstracts should routinely highlight these items, perhaps under separate headings (as Annals articles do for limitations sections in their abstracts). Routine reporting of absolute risks is the only item that would present a challenge, because particular study designs need unique approaches. The Grading of Recommendations, Assessment, Development and Evaluation (GRADE) working group provides guidance for meta-analyses: use the mean absolute risk either from the control groups or from an external population estimate, and generate the absolute risk for the intervention group by applying the summary ratio measure.24 The same approach with an external population standard should be used for case control studies if absolute risks cannot be calculated directly from the study itself.

Reporting on medical research is challenging: newspapers need to reach readers who vary widely in, for example, statistical literacy and reading levels. But these issues are not unique to medical news. Journalists constantly report quantitative information. Imagine the sports section without scores, player statistics, or team standing tables; or political polls without numbers. Although further work is needed to improve public understanding of medical research, a first step is to ensure that people have access to the fundamental information—basic study facts, quantified results, important study limitations—information they need to understand the findings and to decide whether to believe them. Our results suggest that press releases of high quality increase the chance that readers will receive this information.

High quality abstracts might improve newspaper coverage. But our observations suggest that well written press releases issued by medical journals could do even more to improve the communication of medical news to the public. Our observation that press releases have more influence than journal abstracts on reporting is unsurprising. Abstracts are dense, technical, and written mainly for a professional audience. Press releases are written in a non-technical narrative format that explicitly targets journalists, many of whom have limited scientific training.

High quality press releases are a simple way for medical journals to increase the chance of newspapers receiving key information. We hope our observations encourage medical journals to issue high quality press releases. Press officers could use a checklist to remind them to include the basic facts, numbers, and cautions. A more ambitious approach would be to develop a standardised press release that would help journalists find key information, perhaps by including structured tables quantifying benefits and harms. Some small journals, however, simply lack the necessary staff to produce high quality press releases, emphasising the need for editors to ensure that the relevant information is easily accessible in the journal abstract.

Our study shows that there is substantial room for improving press releases. Medical journals should use press releases not simply to make medical news—but also to make news reporting better.