Abstracts for reports of randomised trials of COVID-19 interventions had low quality and high spin.

OBJECTIVES: To assess the reporting quality of abstracts for published randomized controlled trials (RCTs) of interventions for coronavirus disease 2019 (COVID-19), including the use of spin strategies and the level of spin for RCTs with statistically non-significant primary outcomes, and to explore potential predictors for reporting quality and the severity of spin. STUDY DESIGN AND
SETTING: PubMed was searched to find RCTs that tested interventions for COVID-19, and the reporting quality and spin in the abstracts were assessed. Linear regression analyses were used to identify potential predictors.
RESULTS: Forty RCT abstracts were included in our assessment of reporting quality, and a higher word count in the abstract was significantly correlated with higher reporting scores (95% CI 0.044 to 0.658, P=0.026). Multiple spin strategies were identified. Our multivariate analyses showed that geographical origin was associated with severity of spin, with research from non-Asian regions containing fewer spin strategies (95% CI -0.760 to -0.099, P=0.013).
CONCLUSIONS: The reporting quality of abstracts of RCTs of interventions for COVID-19 is far from satisfactory. A relatively high proportion of the abstracts contained spin, and the findings reported in the results and conclusion sections of these abstracts need to be interpreted with caution.

The median reporting score of 40 abstracts presenting the results of RCTs of interventions for COVID-19 was 8 (6, 10) of the 16 items in the CONSORT statement for abstracts.

Fourteen (52%) of 27 abstracts with statistically nonsignificant primary outcomes had spin in the results section and 12 (44%) abstracts had spin in the conclusion section.

Multivariate analyses showed that word count in the abstract was significantly correlated with reporting scores (P = 0.026) and geographical origin was associated with severity of spin (P = 0.014).

The reporting quality of abstracts of RCTs of 19 interventions for COVID-19 is low, and a relatively high proportion of the abstracts contains spin.

This study explores potential predictors for reporting quality and the severity of spin.

We recommend that authors, peer reviewers and editors make better use of reporting guidelines such as CONSORT and its extensions when preparing, appraising and editing research articles. Allowing authors more words in their abstracts might be a simple method to improve the reporting quality of abstracts.

The findings reported in the results and conclusion sections of these abstracts need to be interpreted with caution, especially for those studies from Asian regions.

Introduction

In December 2019, a novel RNA coronavirus, which was subsequently named severe acute respiratory syndrome coronavirus (SARS-CoV-2) caused an outbreak of pneumonia in Hubei province of China, and quickly transmitted to other countries, resulting in millions of infections and deaths globally in just a few months. Unfortunately, there were no efficient methods to kill this virus or to treat the coronavirus disease 2019 (COVID-19) it caused. To address this, a large number of randomized controlled trials (RCTs) emerged very quickly to test interventions to prevent or treat COVID-19, along with a vast number of other pieces of research. This rapid, unprecedented outpouring of research on a specific condition means that policy makers, patients and clinicians are under great pressure to identify the most useful and reliable information from amidst an overwhelming number of articles. They need relevant and valid material which can be accessed quickly and with minimal efforts [1], and as a result, the abstracts of the full reports become key to supporting clinical decision-making [2]. Clear reporting of a study's results in its abstract is likely to aid rational decision-making, but selective outcome reporting and other biases in the abstracts may make readers disoriented. It will be also important for users of these RCTs to consider whether the studies were justified, given that this has been identified as a problem for RCTs in the past [3].

To help readers, the consolidated standards of reporting trials (CONSORT) statement, published in 2010, lists 16 essential items for inclusion in the abstracts for reports of RCTs [4]. This provides the requirements for the content of a well reported abstract, but selectivity in how the authors present their results remains a problem, especially in studies without statistically significant outcomes where the authors might use spin to try to highlight results inappropriately.

The concept of spin was originally used in fields such as culture, politic and commerce, and is defined as a form of propaganda that could affect public views on an organization or public figure [5]. In health care, it was defined by Boutron et al. in 2010 as “specific reporting strategies from whatever motive, highlighting the interventions are beneficial despite the statistically nonsignificant differences for the primary outcomes” [6]. Since then, it has been widely evaluated in several medical specialties [7], including stomatology [5,8], otolaryngology [9], surgery [10,11], and cardiology [12]. Considering that such a large number of COVID-19 articles are being published with shorter periods for peer review and editorial oversight, this raises the possibility that more spin is making its way into the literature and that readers are being misled. Therefore, we have undertaken what we believe to be the first study of spin in reports of RCTs of interventions for COVID-19. This study investigates (1) the reporting quality of the abstracts for published RCTs of interventions for COVID-19 and their use of spin strategies, and the extent and level of spin in abstracts with statistically nonsignificant primary outcomes, and (2) potential predictors for reporting quality and the severity of spin.

Materials and methods

Search, eligibility, and selection of articles

We searched PubMed to identify reports of RCTs that had tested interventions for COVID-19 and were published up to 31 October 2020, using the following search strategy: (("COVID 19" [MeSH Terms] OR "2019 novel coronavirus" [All Fields] OR "2019 nCoV" [All Fields] OR "SARS CoV 2" [All Fields] OR "severe acute respiratory syndrome coronavirus 2" [All Fields] OR "coronavirus" [All Fields]) AND ("RCT" [All Fields] OR "randomized controlled trial" [All Fields] OR "prospective cohort study" [All Fields] OR "longitudinal study" [All Fields] OR "cohort study" [All Fields])) AND ("2019/11/1" [Date - Publication]:"2020/10/31" [Date - Publication]). We included only reports published in English that presented the results of a RCT (defined as a prospective comparative study in which participants are allocated at random to one of the intervention arms). We excluded brief commentaries, research letters, observational studies (eg, cohort, case–control and cross-sectional studies), protocols, meta-analyses, and systematic reviews. We also excluded any reports that exclusively focused on cost-effective evaluations [13] or diagnostic test accuracy. Two reviewers (D.G.W. and Y.G.Z.) independently determined the eligibility of each abstract and a third reviewer (L.W.) was consulted in the event of any disagreements. The full text of the article linked to the abstract was retrieved when necessary, to determine eligibility or obtain additional information on the RCT (eg, source of funding).

Data extraction and reporting quality assessment of selected articles

For each selected RCT report, we extracted key information (journal type, average journal impact factor in recent 5 years, geographical location of first author, number of authors, type of institution, number of study centers, word count in the abstract, objective, structured format in the abstract, sample size, experimental interventions, number of intervention arms, number of primary outcomes, exact reporting of P values, and funding source) into an Excel document for further analyses.

Using the CONSORT explanation and elaboration statement for abstracts of parallel group randomized trials by Moher et al. [4] and previous reports [5,14], two reviewers worked independently (D.G.W. and Y.G.Z.) to make a judgment on whether an item in the RCT abstract was adequately reported. These two reviewers were calibrated through rounds of 10 randomly selected abstracts until strong agreement (Cohen's κ statistic ≥0.90) was reached. The reporting quality of all abstracts was then evaluated by these two assessors independently, with the overall reporting score being calculated for each abstract by giving a score of 1 for each of the 16 CONSORT essential items that were adequately described in the abstract and a score of 0 if the explanation was inadequate or unclear. These scores were summed to obtain a comprehensive reporting score for each selected abstract, which could range from 0 to 16. Any discrepancies were resolved in discussion with a third reviewer (L.W.).

Definition of primary outcomes of the RCTs

For this study, we defined an RCT's primary outcomes using the method outlined by Boutron et al. [6]:

Explicit reporting of the primary outcomes in the original studies or clinical trial registrations; or

Outcomes reported in the calculation of the required sample size for the RCT if no primary outcomes were explicitly reported; or

Outcomes consistent with the primary objectives for the RCT if no primary outcomes were explicitly reported or mentioned in the sample size calculation.

RCTs with statistically nonsignificant primary outcomes (P ≥ 0.05 or a confidence interval for the effect estimate that included no difference) were included in the spin assessment.

Definition of spin and spin strategies

We used the Boutron et al. definition of spin: “specific reporting strategies from whatever motive, highlighting the interventions are beneficial despite the statistically nonsignificant differences for the primary outcomes, distorting the interpretation of study results and misleading readers” [6].

We classified spin strategies as follows, based on previous research [6,12]:

Focusing on statistically significant results (eg, secondary outcomes, within-group comparisons and subgroup analyses).

Claiming equivalent/noninferior/comparable/similar effects for statistically nonsignificant primary outcomes.

Focusing only on primary outcomes or time-points with a statistically significant difference when there are several primary outcomes or multiple time-points for the primary outcomes.

Claiming benefit of interventions with no consideration of the statistically nonsignificant results for primary outcomes or making recommendation for use of experimental interventions.

Spin assessment

Two reviewers (D.G.W. and Y.G.Z.) independently determined the presence of spin and spin strategies in the results and conclusion section of each abstract. As for the assessment of quality, an internal pilot study was conducted to calibrate these two assessors using rounds of 10 randomly selected abstracts until the agreement between them became strong (Cohen's κ statistic ≥0.90). A third reviewer (L.W.) was consulted to resolve any disagreements. We recorded the number of spin strategies in the results and conclusion section of each abstract, and assessed the level of spin in the conclusion section of the abstracts with a method from other research [6,12]: “high” was defined as no acknowledgment of statistically nonsignificant results for the primary outcome, no uncertainty and no recommendation for further trials; “moderate” was defined as no acknowledgment of statistically nonsignificant results for the primary outcome but a mention of uncertainty or recommendations for further trials; “low” was defined as acknowledgment of statistically nonsignificant results for the primary outcome, or no acknowledgment of statistically nonsignificant results for the primary outcomes but a mention of uncertainty and recommendations for further trials, and “none” acted as a default category.

Statistical analyses

We calculated medians and interquartile ranges (IQR) for continuous variables and the number and proportion (%) of articles for categorical variables. We used linear regression analyses to identify factors correlated with reporting quality, and the severity of spin. For factors associated with reporting quality, we used the overall reporting score as a dependent variable in regression analyses, and the univariate regression model was used first in the exploration of reporting quality predictors. Significant predictors were then tested by multivariate analyses. We used a similar process to explore factors relevant to the severity of spin (defined as the number of spin strategies in abstracts). All statistical analyses were performed with SPSS version 22.0 (IBM Corp, Armonk, NY, USA), and two-tailed P values less than 0.05 were considered statistically significant.

Results

General characteristics of included studies

We retrieved a total of 1388 records from PubMed and identified 40 parallel-group RCTs among these, with 27 (67.5%) having a clearly identified, statistically nonsignificant primary outcome (Fig. 1 ).

Fig. 1

Flow diagram for trial selections.

Of the 40 eligible RCTs [[15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51], [52], [53]–54], 18 (45%) were single-center trials and most were conducted in the Asian region (26/40, 65%) and organized by universities (31/40, 78%). Thirty-three (83%) used a structured format for their abstract and same number of studies evaluated pharmacological interventions for the prevention or treatment of COVID-19. Ten (25%) abstracts were identified with more than 400 words and 17 (43%) studies reported a sample size more than 100 participants. The source of funding was not declared in the article for 6 (15%) RCTs. Detailed characteristics of the included studies are available in Table 1 .

Table 1

General characteristics of the 40 RCTs related to COVID-19.

Characteristic	Category	N (%)
Journal type	Specialized medicine	23 (57.5)
	General medicine	17 (42.5)
Average journal impact factor in recent five years, median( [IQR])	4.858 (2.735, 40.063)
Geographical location	Asia	26 (65.0)
	Europe	7 (17.5)
	America	6 (15.0)
	Africa	1 (2.5)
No. of authors	≤7	6 (15.0)
	>7	34 (85.0)
Type of institution	University	31 (77.5)
	Others	9 (22.5)
Centers	Single center	18 (45.0)
	Multicenter	22 (55.0)
Word count in the abstract	<200	2 (5.0)
	200-400	28 (70.0)
	>400	10 (25.0)
Objective	Efficacy	20 (50.0)
	Safety	0 (0)
	Efficacy and safety	20 (50.0)
Structured format in the abstract	Yes	33 (82.5)
	No	7 (17.5)
Sample size	<50	10 (25.0)
	50-100	13 (32.5)
	>100	17 (42.5)
Intervention	Drugs	33 (82.5)
	Nonpharmaceutical interventions	7 (17.5)
Treatment arms	2	33 (82.5)
	>2	7 (17.5)
Primary outcomes	1	26 (65.0)
	>1	14 (35.0)
Exact P value	Yes	28 (70.0)
	No	12 (30.0)
Funding source	None	2 (5.0)
	Industry	5 (12.5)
	Nonindustry	21 (52.5)
	Industry and nonindustry	6 (15.0)
	Not reported	6 (15.0)
Total	40(100.0)

IQR, interquartile range.

Reporting quality assessment of the abstracts

A strong agreement was reached after two rounds of the pilot study for reporting quality evaluations (κ = 0.932) before we assessed the reporting quality of all 40 abstracts (Fig. 2 ). The details for each item and sub-item are shown in Table 2 and Appendix 1 (Table 2a). Thirty-three (83%) studies could be identified as RCTs by the title of the article, 32 (80%) reported specific objectives or hypotheses, and 28 (70%) presented clearly defined primary outcomes in their abstract. Almost all abstracts mentioned random assignment but additional detail on sequence generation and allocation concealment was rare (37/40, 93%, 2/40, 5% and 1/40, 3%, respectively). The number of participants randomized to each group was found in 32 (80%) abstracts, but only one (3%) abstract reported the status of the trial (eg, closed or ongoing) in the results section, and only a quarter (10/40, 25%) provided full results for each intervention group (including primary outcome results, estimated effect size and its precision). Aside from their primary or secondary outcomes, 21 (53%) abstracts reported important adverse events or side effects in abstracts. Finally, 28 (70%) abstracts contained trial register information and 10 (25%) identified the source of funding.

Fig. 2

Bar graph of the reporting quality of forty RCT abstracts based on the sixteen-item CONSORT statement for abstracts.

Table 2

Reporting of each item and subitem in forty RCT abstracts based on the CONSORT 2010 explanation and elaboration for abstracts.

Item	Criteria and subitems	N (%)
1. Title	Identification of the study as randomized in the title.	33 (82.5)
2. Trial design	Structured summary of the trial design (eg, parallel, cluster, crossover).	2 (5.0)
3. Participants	Eligibility criteria for participants and settings or locations where the data were collected.	20 (50.0)
	3a. Eligibility criteria for participants.	29 (72.5)
	3b. Settings or locations for data collections.	22 (55.0)
4. Interventions	Sufficient details of interventions intended for each group (eg, when, how).	25 (62.5)
5. Objectives	Specific objectives or hypotheses.	32 (80.0)
6. Primary outcomes	Clearly defined primary outcomes for this trial in methods.	28 (70.0)
7. Randomization	Scientific descriptions of how participants were allocated to interventions.	1 (2.5)
	7a. Random assignment (eg, random, randomized, randomization, random allocation).	37 (92.5)
	7b. Sequence generation (eg, random-number tables).	2(5.0)
	7c. Referring to allocation concealment.	1 (2.5)
8. Blinding(masking)	Whether or not participants, trial providers, and data collectors were blinded.	1 (2.5)
	8a. Brief descriptions only (eg, single-blind, double-blind, triple-blind).	9 (22.5)
9. Numbers randomized	Numbers of participants randomized to each group.	32 (80.0)
10. Recruitment and following up	Trial status (eg, on-going, closed to recruitment, closed to follow-up).	1 (2.5)
11. Numbers analyzed	Numbers of participants analyzed in each group.	32 (80.0)
	11a. Whether or not analyzed in accordance with the original grouping (eg, intention-to-treat analysis or pre-protocol analysis).	4 (10.0)
12. Reports of primary outcomes	For the primary outcomes, a summary report of results for each group and the estimated effect size and its precision.	10 (25.0)
	12a. Primary outcome results for each group.	20 (50.0)
	12b. Estimated effect size.	17 (42.5)
	12c. Precision of the estimate (eg, 95%CI).	16 (40.0)
13. Harms	Important adverse events or side effects (seeing CONSORT for harms for specific guidance).	21 (52.5)
14. Conclusions	General interpretations corresponding to the results.	39 (97.5)
	14a. Benefits and harms balanced.	20 (50.0)
15. Trial registration	Trial registration number and the name of trial register.	28 (70.0)
16. Funding and supports	Source of funding and supports.	10 (25.0)

Predictors correlated to the reporting quality of abstracts in RCTs

The median overall reporting score for the 40 RCT abstracts was 8 (IQR: 6, 10). As shown in Figure 3 , eight predictors were significantly correlated to a higher score in our univariate linear regression analyses: general journals (P = 0.009), higher average IF (P < 0.001), non-Asian locations (P = 0.015), larger number of authors (P < 0.001), multicenter studies (P = 0.008), longer word count (P < 0.001), structured format of abstracts (P = 0.006), and larger sample size (P = 0.002). In multivariate models, only the predictor of longer word count (β = 0.351, 95% CI 0.044–0.658, P = 0.026) remained as a statistically significant predictor (adjusted R2 = 0.467, P < 0.001).

Fig. 3

Factors associated with overall reporting score in forty RCT abstracts.

aFor the multivariate linear regression analyses, adjusted R² = 0.467, P < 0.001.

bP < 0.05 was considered statistically significant, shown in bold.

Abbreviations: IF, impact factor; CI, confidence interval.

Spin strategies in abstracts

From the 40 RCT abstracts, the 27 (68%) with statistically nonsignificant primary outcomes were included in our assessment of spin and 15 (56%) of these contained spin (κ = 0.977), with multiple spin strategies present (Table 3 and Appendix 2 [Table 3a]). Fourteen (52%) abstracts had spin in the results section, with a focus on statistically significant secondary outcomes being the most frequent spin strategy (10/15, 67%). Other spin strategies found in the results section were focusing on statistically significant within-group comparison (2/15. 13%), focusing on statistically significant subgroup analyses (4/15, 27%), focusing on statistically significant within- or between-group comparison of secondary outcomes (1/15, 7%), and focusing only on primary outcomes of statistical significance when there are several primary outcomes (3/15, 20%).

Table 3

Spin strategies identified in 16 RCT abstracts with spin

	N (%)
Spin strategies in the result section
Focusing on statistically significant within-group comparisons.	2 (12.5)
Focusing on statistically significant secondary outcomes.	12 (75.0)
Focusing on statistically significant subgroup analyses.	4 (25.0)
Focusing on statistically significant within- or between-group comparisons of secondary outcomes.	1 (6.3)
Focusing only on primary outcome of statistical significance when several primary outcomes exist.	3 (18.8)
Spin strategies in the conclusion section
Claiming equivalent/noninferior/comparable/similar effects for statistically nonsignificant primary endpoints.	0 (0)
Focusing only on statistically significant results (eg, secondary outcomes, subgroup analyses, within-group analyses).	4 (25.0)
Claiming benefit with no consideration of the statistically nonsignificant primary outcomes.	12 (75.0)
Recommendation to use the experimental treatment.	5 (31.3)
Focusing only on outcomes with statistical significance when several primary outcomes exist.	3 (18.8)
Focusing only on time-points with statistical significance when multiple time-points for primary outcomes exist.	0 (0)

Spin was also identified in the conclusion section of 12 (44%) of 27 abstracts with statistically nonsignificant primary outcomes. Claiming benefit with no consideration of the statistically nonsignificant nature of the result for the primary outcomes was the most frequent spin strategy in the abstract conclusions (12/15, 80%). Five (33%) abstracts recommended use of experimental treatment, and 4 (27%) focused only on statistically significant results. Three (20%) abstracts focused only on a primary outcome of statistical significance in their conclusion section despite there being several primary outcomes in the RCT. None claimed equivalent/noninferior/comparable/similar effects for statistically nonsignificant results or focused only on a timepoint of statistical significance when there were multiple timepoints for the primary outcomes.

Level of spin evaluation in selected abstracts

Among the 15 abstracts with spin, 9 (60%) contained more than two spin strategies, with one (11%) having seven spin strategies. There were 1 or 2 spin strategies in the remaining 6 (40%) abstracts. Furthermore, among the 12 abstracts with spin in the conclusion section, we found a high level of spin in 6 (50%) abstracts. The level of spin was moderate in 3 (25%) and low in 3 (25%) conclusion sections (Table 4 ).

Table 4

Assessment of level of spin in the conclusion section of RCT abstracts

Level of spin in the conclusion section	N (%)
Low a	3 (25.0)
Moderate b	3 (25.0)
High c	6 (50.0)

Acknowledge statistically nonsignificant results for the primary outcome, or no acknowledgment of statistically nonsignificant results for the primary outcome but reported with uncertainty and recommendations for further trials.

No acknowledgment of statistically nonsignificant results for the primary outcome but reported with uncertainty or recommendations for further trials.

No acknowledgment of statistically nonsignificant results for the primary outcome, no uncertainty and no recommendations for further trials.

Potential predictors associated with the severity of spin

We explored whether the potential characteristic predictors were correlated with spin (Fig. 4 ). In summary, three factors were significantly associated with spin severity in our univariate analyses: less spin occurred in general and higher average IF journals and studies from non-Asian regions (P = 0.005, 0.023 and 0.004, respectively). Only the predictor of studies from non-Asian regions (β = -0.426, 95% CI -0.756 to -0.096, P = 0.014) remained statistically significant in our multivariate analyses (adjusted R2 = 0.388, P = 0.002).

Fig. 4

Factors related to spin severity in 27 RCT abstracts with statistically nonsignificant primary outcomes.

aFor the multivariate linear regression analyses, adjusted R²=0.388, P = 0.002.

bP < 0.05 was considered statistically significant, shown in bold.

Abbreviations: IF, impact factor; CI, confidence interval.

Discussion

To the best of our knowledge, this is the first study to systematically assess the reporting quality and presence of spin in the abstracts of reports of RCTs testing interventions for COVID-19. The CONSORT statement for abstracts provides authors with guidance on the necessary details and clarity required for good reporting. It is intended to improve the reporting quality of abstracts, and recently, has been used as a tool to evaluate the reporting quality of abstracts [[55], [56], [57], [58]–59]. Our results show that the overall reporting quality of the abstracts of these COVID-19 RCTs is far from satisfactory, with adherence of reports to the CONSORT items ranging from 2.5% to 97.5%. Notably, the CONSORT items most inadequately reported were those related to trial design, randomization, blinding, and trial status. Other studies in the medical literature have also shown a high prevalence of nonadherence to the CONSORT guidelines in RCT abstracts [[60], [61], [62], [63], [64]–65]. For example, Mozetic et al. found that the most underreported items were related to the methods items of CONSORT, such as trial design, allocation concealment, implementation of randomization sequence, and blinding [63]. Gallo et al. also found this, with limited adherence to the CONSORT for abstracts checklists among RCT abstracts published in the top plastic surgery journals [64]. They concluded that the most poorly reported items were trial registration (4%), method of randomization (2.4%), and source of trial funding (0%). Furthermore, Janackovic and Puljak evaluated 622 RCT abstracts in anesthesiology and observed the lowest consistency in trial design (18%), recruitment status (9%), number of participants analyzed (8%), randomization (3%), and funding (0.2%) [65]. Even in the top five highest-impact general medical journals, there was lack of adherence to the CONSORT statement for abstracts [66]. Overall, RCT abstracts in medical journals are poorly reported, providing readers with insufficient information, which means that readers might not get useful information in a short time and might even be misled by the incomplete results. One possible explanation for this failure to follow the guidelines is that some researchers are not familiar with the CONSORT statement, and that they may repeat the structure and content they have seen in other abstracts when they draft their report. In addition, the editorial office has certain responsibility, because abstracts are often structured in accordance with the author guidelines of academic journals. To remedy this, we recommend that authors, peer reviewers and editors make better use of reporting guidelines such as CONSORT and its extensions when preparing, appraising and editing research articles.

It should be noted that poor reporting quality of abstracts cannot be misinterpreted as poor study design. Limited length of the abstract means that it is often not possible to show all details of the research. Therefore, consistent with our results, items on RCT methods tend to have lower reporting rates in abstracts while, at least in journal papers with high impact, methodological details are well formulated in the full text.

The concept of spin, applying to clinical research, means selectively reporting significant findings while neglecting nonstatistically significant results. The role of spin is to make imperfect research results more meaningful, and thus stand out from similar studies. Up to now, there is no completely objective evaluation method for spin. In addition to the spin strategies mentioned by Boutron [6], more unrecognized strategies exist. We assessed the most common and widely used eleven spin strategies, and nearly 60% of the RCT abstracts with statistically nonsignificant primary outcomes in our study contained spin. High prevalence of spin has also been reported by other studies, ranging widely from 17% to 86% [5,8–12,[67], [69]–70]. We speculate that high spin prevalence in abstracts of COVID-19 RCTs is due to the specific background at that time, that is, on the one hand, lower standards in medical journals and rapid peer review might lead to lax assessment of manuscripts, while on the other hand, high mortality and morbidity of COVID-19 needed prompt evidence on therapies, resulting in the emergence of large numbers of RCTs of interventions which were subsequently found to be ineffective. Consequently, the existence of spin in these early reports seems not surprising.

Among various forms of spin strategy, the most common one in the results section of the abstract was focusing on statistically significant secondary outcomes to claim benefit, while no consideration of the statistically nonsignificant primary outcomes was the most common strategy found in the conclusion sections. Similar findings were reported by Jellison et al. [68]. In another study, however, focusing on statistically significant within-group analysis was the most common spin strategy used in the result section of abstracts, and claiming equivalence or noninferiority of results with nonsignificance was the most common in the conclusion section [5]. Moreover, Turrentine analyzed 83 scientific publications with spin in abstracts and concluded that the more common types of spin strategies in general obstetrics and gynecology were: emphasizing statistically significant secondary results (40%), interpreting nonstatistically significant primary outcomes as equivalent or similar effectiveness (37%) and claiming beneficial effects of treatment despite the statistical nonsignificance (15%) [70]. Lockyer et al. indicated that there was potential for spin in wound care trials emphasizing study results of significance rather than the importance of outcomes [67]. This is concerning because clinicians are prone to misunderstand the outcomes of a trial when spin is present and make inappropriate clinical decisions [6]. This makes it especially important to find ways to identify and mitigate spin [10]. Readers should keep the concept of spin in mind when reading abstracts and be aware of the diversity and heterogeneity of spin strategies, and researchers should properly report their results and conclusions within the limited word count for the abstract, instead of giving space to only those results that they wish to highlight to show the importance of their research. Journal reviewers and editors also need to be rigorous in their assessment of manuscripts. Our analyses of the predictors potentially associated with reporting quality show that a larger word count was significantly associated with better quality reporting in RCT abstracts. For this reason, allowing authors more words in their abstracts might be a simple method to improve the reporting quality of abstracts. In addition to larger word counts, reporting of trial registration and funding were also positively correlated with high-quality reporting [71]. In the multivariate analysis of relevant factors to spin severity, we found that research from non-Asian regions might be relevant to fewer spin strategies. Both Cooper and Reynolds-Vaughn demonstrated that a majority of abstracts with spin were funded by industry [9,72], while Jellison et al. found no relationship between industry funding and spin in abstracts [68], and use of statistician and article section were further confirmed to be unrelated to the presence of spin in another study [11]. Furthermore, numbers of research centers were reported associated with presence of spin by Wu et al. [8], and Checketts et al. found that word count limit promoted the prevalence of spin [10]. Whatever, these findings raise concerns about the reporting specification of abstracts, and it is of vital importance for researchers to objectively and accurately report their findings.

There are some limitations to our study. Although we included all abstracts for RCTs of interventions for COVID-19 that we identified in PubMed up to the end of October 2020, the number of included RCTs is still relatively small, which means that our estimates might change if the study was expanded to include more abstracts. Second, although we evaluated spin strategies with a predesigned 11-item form used in other studies, some other potential spin strategies might have been omitted, leading to an underestimate of the presence of spin. Third, only RCTs testing interventions for COVID-19 were included when evaluating the reporting quality and spin of abstracts, that is to say, our analyses was focused on abstract sections, and our conclusions of poor reporting quality and high spin should not be extended to the full text. Fourth, evaluations of reporting quality and spin are subjective, and although we adopted an approach of double, independent and calibrated assessment to control the magnitude of subjectivity, if others repeated our assessments they might obtain different results. Despite these limitations, our study provides important new insights for the reporting quality of RCT abstracts which may have implications to research reporting more generally, as well as its specific relevance to the reporting of RCTs of interventions for COVID-19.

Conclusions

As of the end of October 2020, the reporting quality of the abstracts for reports of RCTs of interventions for COVID-19 is far from satisfactory. The frequency, extent and level of spin are relatively high in these abstracts, highlighting a need for the results and conclusion sections of such abstracts to be interpreted with caution. We hope that this assessment will raise readers’ awareness of the need to carefully appraise abstracts, to be aware of the concept of spin, and to be especially cautious if the pressures of the COVID-19 pandemic increase their reliance on the results and conclusions reported in abstracts.

Author contributions

Concept and design: YGZ, HF and WML.

Data extraction, statistical analysis and interpretation: DGW, LMC, YGZ, LW, FH, JL, YXL and MC.

Drafting the manuscript: DGW, LMC and YGZ.

Revising the manuscript: FH and MC.

Supervision: HF and WML.

Funding

This work was supported by National Key R&D Program of China (2017YFC1309703) and 1.3.5 project for disciplines of excellence-Clinical Research Incubation Project, West China Hospital, (2019HXFH008).