| Literature DB >> 31050775 |
Muhammad Shahzeb Khan1, Noman Lateef2, Tariq Jamal Siddiqi3, Karim Abdur Rehman4, Saed Alnaimat5, Safi U Khan6, Haris Riaz4, M Hassan Murad7, John Mandrola8, Rami Doukky9, Richard A Krasuski10.
Abstract
Importance: Clinical researchers are obligated to present results objectively and accurately to ensure readers are not misled. In studies in which primary end points are not statistically significant, placing a spin, defined as the manipulation of language to potentially mislead readers from the likely truth of the results, can distract the reader and lead to misinterpretation and misapplication of the findings. Objective: To determine the level and prevalence of spin in published reports of cardiovascular randomized clinical trial (RCT) reports. Data Source: MEDLINE was searched from January 1, 2015, to December 31, 2017, using the Cochrane highly sensitive search strategy. Study Selection: Inclusion criteria were parallel-group RCTs published from January 1, 2015, to December 31, 2017 in 1 of 6 high-impact journals (New England Journal of Medicine, The Lancet, JAMA, European Heart Journal, Circulation, and Journal of the American College of Cardiology) with primary outcomes that were not statistically significant were included in the analysis. Data Extraction and Synthesis: Analysis began in August 2018. Data were extracted and verified by 2 independent investigators using a standard collection form. In cases of disagreement between the 2 investigators, a third investigators served as arbitrator. Main Outcomes and Measures: The classifications of spin type, severity, and extent were determined according to predefined criteria. Primary clinical outcomes were divided into safety of treatment, efficacy of treatment, and both.Entities:
Mesh:
Year: 2019 PMID: 31050775 PMCID: PMC6503494 DOI: 10.1001/jamanetworkopen.2019.2622
Source DB: PubMed Journal: JAMA Netw Open ISSN: 2574-3805
Figure. PRISMA Flowchart Outlining the Search Strategy
RCT indicates randomized clinical trial.
Trial Characteristics
| Characteristic | No. (%) [95% CI, %] |
|---|---|
| Total, No. | 93 |
| End points | |
| Safety of treatment | 3 (3) [1.1-9.1] |
| Efficacy of treatment | 84 (90) [82.6-94.8] |
| Safety and efficacy | 6 (6) [3.0-13.3] |
| Primary outcome | |
| Binary | 38 (41) [31.4-51.0] |
| Other | 52 (56) [45.8-65.6] |
| Binary and other | 3 (3) [1.1-9.1] |
| Sample size, median (IQR) [range] | 1002 (303-2539) [48-18201] |
| Journal impact factor, median (IQR) [range] | 32.3 (20-48) [19-72] |
| No. of citations up to May 2018, median (IQR) [range] | 7 (4-17) [0-86] |
| Experimental treatment | |
| Drug | 40 (43) [33.4-53.1] |
| Surgical operation or procedure | 25 (27) [18.9-36.7] |
| Device | 7 (8) [3.7-14.7] |
| Participative intervention (eg, rehabilitation or education) | 4 (4) [1.6-10.5] |
| Other | 17 (18) [11.7-27.3] |
| Comparator | |
| Placebo | 22 (22) [16.2-33.2] |
| Device | 1 (1) [0.2-5.9] |
| Standard care | 28 (30) [21.7-40.1] |
| Drug | 12 (13) [7.5-21.2] |
| Surgical operation/procedure | 23 (25) [17.1-34.4] |
| Participative intervention (eg, rehabilitation or education) | 2 (2) [0.6-7.5] |
| Other | 5 (5) [2.3-12.0] |
| Funding source | |
| None | 0 (0) [0-0.4] |
| For profit | 25 (27) [18.9-36.7] |
| Nonprofit | 37 (40) [30.4-50.0] |
| For profit and nonprofit | 19 (20) [13.5-29.7] |
| None reported | 12 (13) [7.5-21.2] |
Abbreviation: IQR, interquartile range.
Classification of Spin in the Title, Abstract, and Main Text of Randomized Clinical Trials
| Type of Spin | No. (%) [95% CI] | |
|---|---|---|
| Abstract (n = 93) | Main Text (n = 93) | |
| Title | NA | 10 (11) [5.9-18.7] |
| Results section | 38 (41) [31.4-51.0] | 35 (38) [28.4-47.8] |
| Focusing on statistically significant within-group comparison | 8 (9) [4.4-16.1] | 8 (9) [4.4-16.0] |
| Focusing on statistically significant secondary outcomes | 13 (14) [8.3-22.5] | 11 (12) [6.7-19.9] |
| Focusing on statistically significant subgroup analyses | 10 (11) [5.9-18.7] | 8 (9) [4.4-16.0] |
| Focusing on statistically significant modified population of analyses (eg, per-protocol analyses) | 0 (0) [0-3.9] | 0 (0) [0-3.9] |
| Focusing on statistically significant within-group and between-group comparisons for secondary outcomes | 1 (1) [0.2-5.8] | 1 (1) [0.2-5.8] |
| Other | 6 (6) [3.0-13.4] | 7 (8) [3.7-14.7] |
| Discussion section | NA | 35 (38) [28.4-47.8] |
| Reporting of statistically nonsignificant outcome as if the trial were an equivalence trial | NA | 5 (5) [2.3-12.0] |
| Focusing on statistically significant secondary outcomes | NA | 7 (8) [3.7-14.7] |
| Focusing on statistically significant subgroup analyses | NA | 6 (6) [3.0-13.4] |
| Focusing on statistically significant modified population of analyses (eg, per-protocol analyses) | NA | 0 (0) [0-3.9] |
| Focusing on overall within-group improvement | NA | 7 (8) [3.7-14.7] |
| Ruling out adverse event | NA | 1 (1) [0.2-5.8] |
| Other | NA | 9 (10) [5.2-17.4] |
| Conclusions section | 45 (48) [38.5-58.4] | 50 (54) [43.7-63.5] |
| Focusing only on treatment effectiveness | 17 (18) [11.7-27.3] | 16 (17) [10.9-26.1] |
| Claiming equivalence for statistically nonsignificant results | 9 (10) [5.2-17.4] | 6 (6) [3.0-13.4] |
| Claiming efficacy with no consideration of statistically nonsignificant primary outcome | 4 (4) [1.7-10.5] | 7 (8) [3.7-14.7] |
| Focusing only on statistically significant results | 4 (4) [1.7-10.5] | 3 (3) [1.1-9.1] |
| Acknowledging statistically nonsignificant results for the primary outcome but emphasizing beneficial effect of treatment | 2 (2) [0.6-7.5] | 6 (6) [3.0-13.4] |
| Acknowledging statistically nonsignificant results for the primary outcome but emphasizing other statistically significant results | 9 (10) [5.2-17.4] | 12 (13) [7.1-21.8] |
| Other spin in conclusions section, No. (%) | 13 (14) | 13 (14) |
| Conclusion ruling out an adverse event on statistically nonsignificant results | 2 (2) [0.6-7.5] | 0 (0) [0-3.9] |
| Conclusion focusing on within-group assessment (both treatments are effective or treatment administered in both groups is effective [eg, add-on studies]) | 1 (1) [0.2-5.8] | 1 (1) [0.2-5.8] |
| Recommending to use the treatment | 1 (1) [0.2-5.8] | 3 (3) [1.1-9.1] |
| Focusing on another objective | 2 (2) [0.6-7.5] | 4 (4) [1.7-10.5] |
| Comparing with placebo group of another trial | 0 (0) [0-3.9] | 0 (0) [0-3.9] |
| Reporting statistically nonsignificant subgroup results as beneficial | 0 (0) [0-3.9] | 1 (1) [0.2-5.8] |
| Other | 7 (8) [3.7-14.7] | 4 (4) [1.7-10.5] |
Abbreviation: NA, not applicable.
Extent and Level of Spin in the Conclusions Sections of Randomized Clinical Trials
| Spin | No. (%) [95% CI] | |
|---|---|---|
| Abstract (n = 93) | Main Text (n = 93) | |
| Extent of spin | ||
| None | 40 (43) [33.4-53.1] | 31 (33) [24.6-43.4] |
| 1 Section, other than conclusions | 11 (12) [6.7-19.9] | 12 (13) [7.5-21.2] |
| Conclusions only | 16 (17) [10.9-26.1] | 13 (14) [8.3-22.7] |
| 2 Sections | NA | 19 (20) [13.5-29.7] |
| All sections | 26 (28) [19.9-37.8] | 18 (19) [12.2-29.1] |
| Level of spin in conclusions section | ||
| None | 52 (56) [45.8-65.6] | 44 (47) [37.5-57.4] |
| Low | 30 (32) [23.6-42.3] | 36 (39) [29.5-48.9] |
| Moderate | 6 (6) [3.0-13.4] | 5 (5) [2.3-12.0] |
| High | 5 (5) [2.3-12.0] | 8 (9) [4.4-16.0] |
Abbreviation: NA, not applicable.
Low spin was defined as acknowledgment of the nonsignificant primary outcome or uncertainty in the framing and presence of recommendations for further trials.
Moderate spin was defined as no acknowledgment of the nonsignificant primary outcome but some uncertainty in the framing or the presence of recommendations for further trials.
High spin was defined as no acknowledgment of the statistically nonsignificant primary outcomes, no uncertainty in the framing, and no recommendations for further trials.
Elements Associated With Level of Spin in Cardiovascular Randomized Clinical Trial Reports
| Element | Abstract | Main Text | ||
|---|---|---|---|---|
| χ2 ( | Interpretation | χ2 ( | Interpretation | |
| End point | 4.285 (.12) | No significant association | 10.463 (.005) | Level of spin varied according to type of end point; studies that measured efficacy as their primary outcome were significantly more likely to use spin in their main text when compared with studies measuring safety as their primary outcome ( |
| Primary outcome | 6.033 (.049) | Level of spin varied according to type of primary outcome; studies with nonbinary primary outcomes were significantly more likely to use spin in the abstract compared with studies with binary primary outcomes ( | 3.060 (.28) | No significant association |
| Experimental arm | 3.965 (.41) | No significant association | 1.345 (.85) | No significant association |
| Comparator arm | 15.266 (.02) | Level of spin varied according to type of comparator arm; studies were more likely to have spin in the abstract when the comparator was a drug compared with studies with standard care as the comparator arm ( | 11.877 (.07) | No significant association |
| Subspecialty | 5.363 (.50) | No significant association | 2.462 (.87) | No significant association |
| Funding | 12.011 (.65) | No significant association | 5.101 (.34) | No significant association |
| Journal | 19.681 (.001) | Level of spin varied according to journal; level of spin in the abstract was associated with the journal ( | 10.115 (.72) | No significant association |
Abbreviations: JACC, Journal of the American College of Cardiology; NEJM, New England Journal of Medicine.
P value was found from the Kruskal-Wallis test.
P value was found from the Dunn post hoc method.