Spin in the neurosurgical trauma literature: prevalence and associated factors - a systematic review protocol.

INTRODUCTION: Spin is defined as an inaccurate interpretation of results, intentionally or not, leading to equivocal conclusions and misdirecting readers to look at the data in an overly optimistic way. Previous studies have shown a high prevalence of spin in scientific papers and this systematic review aims to investigate the nature and prevalence of spin in the neurosurgical trauma literature. Any associated factors will be identified to guide future research practice recommendations. METHODS AND ANALYSIS: The Preferred Reporting Item for Systematic Reviews and Meta-Analyses recommendations will be followed. Randomised clinical trials (RCTs) that enrolled only patients with traumatic brain injury and investigated any type of intervention (surgical or non-surgical) will be eligible for inclusion. The MEDLINE/PubMed database will be searched for articles in English published in 15 top-ranked journals. Spin will be defined as (1) a focus on statistically significant results not based on the primary outcome; (2) interpreting statistically non-significant results for a superiority analysis of the primary outcome; (3) claiming or emphasising the beneficial effect of the treatment despite statistically non-significant results; (4) conclusion focused in the per-protocol or as-treated analysis instead of the intention-to-treat results; (5) incorrect statistical analysis; (6) republication of a significant secondary analysis without proper acknowledgement of the primary outcome analysis result. Traditional descriptive statistics will be used to present RCT characteristics. Standardised differences between the groups with or without spin will be calculated. The variables with a standardised difference equal or above 0.2 and 0.5 will be considered weakly and strongly associated with spin, respectively. ETHICS AND DISSEMINATION: This study will not involve primary data collection and patients will not be involved. TRIAL REGISTRATION NUMBER: 10.17605/OSF.IO/H3FGY. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the first systematic review to evaluate the prevalence of spin in the neurosurgical trauma literature.

Inclusion of studies with different interventions will allow identification of good and poor examples of randomised clinical trials.

The inclusion of only randomised clinical trials may strengthen our results.

Exclusion of papers not published in specific journals may mean that important additional findings are missed.

Introduction

Spin is defined as an inaccurate interpretation of results, intentionally or not, leading to equivocal conclusions and misdirecting readers to look at the data in an overly optimistic way.1 Spin is considered by many as a consequence of the highly competitive and commercial academic system all scientists work within, since it puts negative findings in a more palatable way to editors, journals, patients, funders and readers.2

Spin may be present in the results, discussion or conclusion sections. Boutron et al demonstrated in a series of randomised clinical trials (RCT) that spin was present in 29.2%, 43.1% and 50.0% of the studies at the results, discussion and conclusion sections, respectively.3 Series of observational studies exhibited more than 80% of spin prevalence.4 Such findings represent a serious problem for the scientific community.

There are three main different spin strategies. First, to focus on statistically significant results not based on the primary outcome, as a within-group comparisons, secondary outcomes or subgroup analyses. Second, interpreting statistically non-significant results for a superiority analysis of the primary outcome as treatment equivalence or comparable effectiveness. Finally, claiming or emphasising the beneficial effect of the treatment despite statistically non-significant results.3

Spin comes in different shapes and sizes, and medical fields with a less robust, fragile scientific output may be more biased by such malpractice. Considering that the quantity and quality of the neurosurgical literature remains suboptimal, it would be thought-provoking to acknowledge how often spins are encountered, how they are presented and its potential to the specialty practice.5–8

Therefore, this systematic review aims to investigate the nature and prevalence of spin in the neurosurgical trauma literature and identify any associated factors which could guide future good research practice recommendations.

Methods and analysis

Protocol and registration

This systematic review and meta-analysis will be reported following the Preferred Reporting Item for Systematic Reviews and Meta-Analyses guidelines.

Eligibility criteria

Published RCT that enrolled only patients with traumatic brain injury (TBI) and investigated any type of intervention (surgical or non-surgical) will be eligible for inclusion. Studies without a clear primary outcome will be excluded. The following journals will be selected for screening based in the impact factor and importance to the neurosurgical trauma literature: New England Journal of Medicine (NEJM), Lancet, Lancet Neurology, Journal of the American Medical Association (JAMA), JAMA Neurology, Journal of Neurology, Neurosurgery and Psychiatry (JNNP), Neurosurgery, Journal of Neurosurgery, Neurosurgical Focus, World Neurosurgery, Acta Neurochirurgica, Journal of Neurotrauma, Intensive Care Medicine, Critical Care, Neurocritical Care, Journal of Trauma and Acute Care Surgery or Critical Care Medicine.

Information sources and search strategy

The MEDLINE/PubMed database will be searched for articles in English published from January 1960 to July 2020. The descriptors (((((((((((Traumatic brain injury[Title/Abstract]) OR (TBI[Title/Abstract])) OR (Brain trauma[Title/Abstract])) OR (Brain concussion[Title/Abstract])) OR (Brain contusion[Title/Abstract])) OR (Head trauma[Title/Abstract])) OR (Head injury[Title/Abstract])) OR (Brain injury[Title/Abstract])) OR (Traumatic subarachnoid hemorrhage[Title/Abstract])) OR (subdural hematoma[Title/Abstract])) OR (epidural hematoma[Title/Abstract])) OR (traumatic intraparenchymal hemorrhage[Title/Abstract]) will be used. The filters ‘Clinical trial’ and ‘Randomized clinical trials’ will be applied.

Study selection

The search strategy aimed to achieve a sample of RCTs published in the neurosurgical trauma literature. Spin in sections other than the conclusion (manuscript or abstract), such as the results and discussion sections, will not be considered due to a perceived higher subjectivity in their definition. After all, a reasonable presentation of results not based on the primary outcome, and their discussion is not necessarily a bad practice—and indeed expected. All articles’ titles and abstracts will be screened by two authors (A and J) for eligibility. The selected articles will be adjudicated by a third author (DJFS) and disagreements will be resolved by consensus. Additional studies identified in the reference section of the selected articles could be included if the eligibility criteria will be fulfilled.

Data collection and analysis

Data will be abstracted and recorded on a standardised form regarding: journal, year of publication, study country (high income or low-and-middle income), first and last authors affiliations (neurosurgery or other), the presence of a statistician among the authors, single centre or multicentre trial, type of trial design (superiority, non-inferiority or equivalence), the presence of a priori sample size and power calculation, type of TBI (mild, moderate, severe), setting (pre-hospital or intra-hospital), intervention (surgical, drug, other), allocation concealment and blinding, number of patients, follow-up period, the event rates in the two treatment arms, the presence of a post hoc power calculation, the discussion of lack of power as a limitation, funding (industry, independent or none) and conflict of interest (when explicitly stated). The full text of the included articles will be systematically reviewed by two authors (A and J). The interobserver agreement and the κ statistic will be calculated. Disagreements will be resolved by a third author (DJFS). The data analysis will be blinded to the authors and institutions of the study. GRADE (Grading of Recommendations Assessment, Development and Evaluations) framework will be used to assess the level of evidence of each paper, varying from very low up to high.

Spin will be defined as1 a focus on statistically significant results not based on the primary outcome, as a within-group comparisons, secondary outcomes or subgroup analyses2; interpreting statistically non-significant results for a superiority analysis of the primary outcome as treatment equivalence or comparable effectiveness3; claiming or emphasising the beneficial effect of the treatment despite statistically non-significant results4; conclusion focused in the per-protocol or as-treated analysis instead of the intention-to-treat results5; incorrect statistical analysis6; republication of a significant secondary analysis without proper acknowledgement of the primary outcome analysis result.

Traditional descriptive statistics will be used to present the included RCT characteristics. Standardised differences between the groups with or without spin will be calculated as proposed by Yang and Dalton. They are indexes which measure the effect size between two groups.9 The variables with a standardised difference equal or above 0.2 and 0.5 will be considered weakly and strongly associated with spin, respectively. All analyses will be conducted with the SPSS software (IBM, SPSS Statistics for Windows, V.24.0).

Risk of bias in individual studies

The entire text of each included paper will be evaluated in a structured fashion for prespecified attributes relating to the assessment and ultimate claim of equivalency. Cochrane’s tool to assess the risk of bias in randomised trials was used.10 This way, the investigators were able to access power and sample size calculation, allocation concealment and many other criteria regarding the methodological quality of each RCT.

Patient and public involvement

No patient involved.

Ethics and dissemination

This study will not involve primary data collection and patients will not be involved. Therefore, formal ethical approval will not be required. The final systematic review will be published in a peer-reviewed journal and presented at appropriate conferences. This protocol may be adapted for the analysis of other innovative surgical and invasive procedures.