A Cross-Sectional Analysis of Spin in Randomized Controlled Trials.

INTRODUCTION

Randomized controlled trials (RCTs) are the most reliable form of evidence for evaluating drug safety and efficacy. Because clinicians rely on RCTs to inform clinical practice, accurate representation of clinical trial results is important to patient health and safety. Spin, defined as reporting that distorts results or misleads the reader,1 threatens accurate evidence interpretation and application by clinicians. Given that many clinicians obtain information from the study abstract only, spin in the abstract is concerning.2 We examined the prevalence of spin among abstracts in a random sample of trials focused on the efficacy of drugs and examined the association of spin with study characteristics.

METHODS

We searched Medline for all English-language RCTs examining drug efficacy published in 2013. Our search identified 2851 potential studies. After screening titles and abstracts, 1101 potential studies remained. We randomly selected 646 of this sample, of which 190 studies were focused on drug efficacy and met inclusion criteria. For each study, we extracted information on outcome, funding source, financial ties to the manufacturer, and design characteristics (Table 1). The methods are described in full in a prior publication.3

Table 1

Prevalence of Spin by Study Characteristics (N = 59)

	All studies(n = 59)N	Spin present(n = 25)N (%)	Spin absent(n = 34)N (%)	p-value
Funding source				0.89
Industry involvement	36	15 (42)	21 (58)
No industry involvement	23	10 (43)	13 (57)
Financial ties of PI present				0.08
Yes	29	9 (31)	20 (69)
No	30	16 (53)	14 (47)
Any relationship with manufacturer (funding and/or financial ties)
Yes	41	17 (42)	24 (58)	0.83
No	18	8 (44)	10 (56)
Sample size				0.01*
Q1 (24–109)	15	10 (67)	5 (33)
Q2 (110–270)	15	5 (33)	10 (67)
Q3 (271–410)	14	8 (57)	6 (43)
Q4 (411–13,229)	15	2 (13)	13 (87)
First author affiliation by continent				0.17
Europe	22	10 (45)	12 (55)
North America	27	8 (30)	19 (70)
Asia	7	5 (71)	2 (29)
Other	3	2 (67)	1 (33)
First author affiliation by country				0.35
USA	23	8 (35)	15 (65)
Other	36	17 (47)	19 (53)
Specialty				0.63
Cardiology	9	3 (33)	6 (67)
Oncology	7	4 (57)	3 (43)
Other	43	18 (42)	25 (58)
Trial registration				0.69
Yes	53	22 (42)	31(58)
No	6	3 (50)	3 (50)
RCT type				0.93
Phase 2	21	9 (43)	12 (57)
Phase 3	26	10 (38)	16 (62)
Phase 4	4	2 (50)	2 (50)
Other	8	4 (50)	4 (50)
Type of analysis				0.24
Superiority	58	24 (41)	34 (59)
Non-inferiority	1	1 (100)	0 (0)
Comparator				0.59
Placebo	49	20 (41)	29 (59)
Active	10	5 (50)	5 (50)
Outcome measure				0.20
Clinical	45	17 (38)	28 (62)
Surrogate	14	8 (57)	6 (43)
Blinding				0.40
Double-blinded	46	18 (39)	28 (61)
Open-label	12	6 (50)	6 (50)
Single-blinded	1	1 (100)	0 (0)

*Kruskal–Wallis p-value of 0.03 based on medians for studies with spin (201) vs. those without (352)

PI principal investigator

Identification of RCT Outcome

Trained abstractors used the results section of each study to determine whether the study reported positive or negative primary efficacy outcome. For superiority studies, if the drug of interest was statistically superior to the control (e.g., p-value < 0.05), the study outcome was defined as positive. For non-inferiority studies, if the drug of interest and the control had no significant difference, the study outcome was defined as positive. Study outcomes were assessed independently and in duplicate. Any disagreement on study outcome was resolved by discussion among the research team.

Outcome Measure

The main outcome variable considered was the presence or absence of spin in the abstract of the RCT. We considered spin present if the abstract outcome was positive or mixed and the study reported a negative primary efficacy outcome in the results section of the manuscript.1 Two clinician reviewers (SK, DK) evaluated all abstract conclusion sections, rating each conclusion as positive (in favor of study drug), negative (neutral or in favor of control), or mixed. A mixed rating meant that it was unclear in the abstract whether the study drug or the control was favored (i.e., if a subgroup analysis was emphasized over the primary outcome). The two clinicians remained blinded to the study outcome during this discussion.

Analysis

We report the prevalence of spin in a sample of RCTs. We examined the association between spin and study characteristics using a two-sided, 0.05-level χ2 test of significance. Statistical analysis was performed using SAS version 9 statistical software (SAS Institute Inc., Cary, NC).

RESULTS

Of the 190 RCTs identified, 59 had a negative primary outcome in the results. These 59 studies were evaluated for the presence of spin. Among the 59 studies, clinician reviewers rated 8 abstracts as having a positive outcome and 17 as having a mixed outcome, for a total of 25 (42%) abstracts with spin. Study characteristics were largely similar across studies with and without spin (Table 1). Overall, studies with spin had smaller samples (median: 201) than studies without spin (median: 352; p = 0.03). There was no relationship between any financial tie to the manufacturer and presence of spin in the abstract (p = 0.83).

DISCUSSION

Nearly half of abstracts of RCTs focused on drug efficacy that report negative results contain spin. We did not find an association between spin and financial ties to industry, but our study may be underpowered to detect this association. Many clinicians do not read beyond the abstract, and many readers of the literature may not have the skill to critically analyze a trial themselves to combat spin or other bias in the report.4–6 Given the widespread reliance on the abstract, the peer review process needs to be improved to reduce spin in abstracts. A simple prompt asking reviewers to comment on the presentation of the results of the study with a specific question about spin may help focus reviewer attention on this issue. Editors can also review for spin in the editorial decision process. These simple steps that allow more scrutiny of the abstract and provide feedback to authors may reduce, if not eliminate, spin in the literature.