Completeness of reporting and outcome switching in trials published in Indian journals from 2017 to 2019: A cross-sectional study.

Context: Randomized controlled trials (RCTs) are among the cornerstones for generation of high-quality clinical evidence. However, incomplete or biased reporting of trials can hamper the process of review of trials and their results. Outcome switching, intentional, or otherwise leads to biased reporting and can result in false inferences. Aims: The aim of this study was to analyze the completeness of reporting Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist items and detect if outcome switching had occurred. Settings and Design: This cross-sectional study was conducted in the department of pharmacology.
Methods: Online editions of journals published by the Indian association of medical specialties from 2017 to 2019 were accessed, and the full-text versions of the published RCTs in them were downloaded. Reporting of each item in the CONSORT checklist was recorded. The effect of trial registration and CONSORT endorsement on reporting of key methodological parameters was also determined. Protocols of registered trials were accessed, and the outcome switching was assessed. Statistical Analysis Used: Descriptive statistics were used to summarize the data.
Results: Average completeness of reporting has significantly improved from 2017 to 2019. Major areas of underreporting were generalizability, protocol availability, trial registration, date of recruitment, allocation concealment, and the patient flow diagram. CONSORT endorsing journals had worse, whereas registered trials had better reporting of key methodological indicators. No overt switching of outcomes was observed in 84 out of 86 registered trials where trial protocols were available online for comparison. Conclusions: Quality of clinical trial reporting in the Indian medical journals has improved but remains inadequate. CONSORT nonendorsement prevents completeness of trial reporting. Copyright:

INTRODUCTION

Randomized controlled trials (RCTs), as per Grading of Assessment, Development, and Evaluation hierarchy, are next only to systematic reviews and meta-analysis of trials with regards to superiority and quality of evidence generated, and they play an important role in the advancement of knowledge and clinical practices. Their major advantage is that due to a controlled setting, there is the elimination of a large number of biases that might otherwise lead to decrease in the quality of a clinical study.[1]

Bearing these facts in mind, it is insufficient to just design and conducts a good clinical trial. The presentation of the study in a transparent and convincing manner that conveys to the users about why the study was done, how it was conducted, and what were the inferences are also essential for the study to have an impact on the current knowledge and practices of medicine. Thus, good reporting is important to establish to the reader transparency, validity, repeatability, and generalizability of a clinical trial. Benefits of good reporting practices are many: to the researchers – helps evaluate and modify their own current research; for clinicians – modify their clinical practice; to organizations/government – policy/decision-making and setting up guidelines.[1]

Consolidated Standards of Reporting Trials (CONSORT), as the name suggest, is a set of guideline for reporting RCTs.[2] These were first formulated in 1994 and later revisions in 2001 and then in 2010. The CONSORT checklist is now widely accepted worldwide as a measure and standard for adequate reporting of the results of a clinical trial, and adoption of the CONSORT statement has been shown to improve the quality of reporting of trials in the journals.[3]

Outcome switching refers to discrepancies between the endpoints/objectives originally defined in the trial protocols and those described in the published article. Although it might at times be justified, especially in long-duration studies, any change in outcomes must be duly mentioned in the article and justified in the protocol and the published article. This might indicate a selective reporting of results that are in alignment with the interests of the researchers and raise questions regarding the reliability and credibility of the results.[4] Three items in the CONSORT checklist – change to the protocol in the trial design section, trial registration details, and the availability of protocol – can help the reader to identify any such inconsistencies.[4]

Previous studies have demonstrated subpar reporting of clinical trials in the Indian medical journals as against the inclusion of the CONSORT 2010 checklist items in the published reports of trials.[5] The objective of the current study was to analyze the completeness of clinical trial reporting across various medical specialty disciplines with respect to the CONSORT 2010 checklist in articles published in major Indian journals, identify the most common areas of underreporting and also to check for outcome switching in the articles where the protocol was available online.

METHODS

We conducted a cross-sectional study on the quality of reporting of RCTs published in the PubMed indexed journals published by the various professional societies of clinical specialties in India from 2017 to 2019.

A randomized controlled study was defined as an intervention/experimental study, in which participants were randomly allocated to two or more treatment groups/arms. The online editions of the journals were accessed, and the full-text versions of the articles published from January 2017 to December 2019, in which RCTs were downloaded and reviewed for the completeness of reporting with respect to the items in the CONSORT-2010 checklist. The ancillary analysis section was excluded from the assessment since it was not universally applicable to trials in our study, we also assessed whether the journal endorses the CONSORT 2010.

For the trials which were registered in the clinical trial registries and whose protocols were available online, we also assessed for the occurrence of outcome switching by comparing the primary and secondary outcomes in the published article with those mentioned in the protocol registered with the trial registry.

Descriptive statistics – means, medians, and proportions to summarize and represent the categorical data. To compare average completeness between the years 2017, 2018, and 2019, Kruskal–Wallis test was used. We performed linear regression analysis to compare whether CONSORT endorsement status affected the overall completeness of reporting. Binomial logistic regression was used to assess the effect of CONSORT endorsement on reporting of key methodological parameters, i. e., sample size calculation, random sequence generation, allocation concealment, descriptions of the primary and secondary outcomes, and details of blinding. For statistical analysis, Microsoft Excel 2016 (Microsoft Corporation, Redmond, Washington, USA) and Statistical Package for Social Sciences (SPPSS) version 20 (IBM Corporation, Armonk, New York USA).

RESULTS

A total of 276 trials from the following journals were included in the analysis as follows:

Indian Journal of Anesthesia.

Indian Journal of Dermatology, Venereology, and Leprology (IJDVL)

Indian Journal of Ophthalmology (IJO).

Indian Journal of Orthopaedics.

Indian Journal of Otolaryngology and Head and Neck Surgery.

Indian Journal of Pharmacology (IJP).

Indian Journal of Preventive and Social Medicine.

Indian Journal of Psychiatry (IJPsy).

Indian Journal of Surgery.

Indian Paediatrics (IP).

Journal of Indian Association of Physicians.

Journal of Obstetrics and Gynecology of India.

Lung India.

Among these, IJP, IJO, IP, IJDVL, IJPsy, and IJOL endorse the CONSORT-2010 checklist.

The basic salient characteristics of the trials included in the study have been represented in Table 1.

Table 1

Description of the basic characteristics of the trials published in PubMed indexed journals of various clinical associations in India from 2017 to 2019 that were included in the study

	2017, n (%)	2018, n (%)	2019, n (%)
Total trials	99	84	93
CONSORT endorsers	21(21.21)	23 (27.81)	33 (35.48)
Blinded	54 (54.55)	42 (50)	49 (52.69)
Double blind	36 (36.36)	31 (36.9)	13 (13.98)
Groups
2	82 (82.83)	68 (86.95)	84 (90.32)
>2	17 (17.17)	16 (19.05)	9 (9.68)
Placebo controlled	17 (17.35)	20 (23.81)	11 (11.83)
Registered	18 (18.85)	21 (25.00)	47 (50.54)
Type of randomization
Simple	82 (82.83)	68 (80.95)	74 (79.57)
Block	13 (13.13)	10 (11.90)	11 (11.83)
Cluster	0	2 (2.38)	0
Not reported	4 (4.04)	4 (4.76)	7 (7.53)
Others	0	0	1

Overall completeness of reporting in the articles was significantly higher in the year 2019 compared to 2017 (Kruskal–Wallis test: P =0.023; Post hoc-Pairwise Dunn's test: P =0.019), while there was no significant difference between the years 2017–2018 and 2018–2019. The major areas of underreporting overall were generalizability, protocol availability, trial registration, date of recruitment, allocation concealment, and patient flow diagram [Table 2]. The blinded trials had better average completeness compared to open-label trials (Mann–Whitney U-test: P <0.00001). Taking CONSORT Endorsement as independent variable and the percentage of average completeness as the dependent variable, we performed a linear regression analysis, which showed a statistically significant negative correlation between CONSORT endorsement and completeness of reporting (R2 = 0.029; 95% confidence interval = −8.303–−1.524; P = 0.005).

Table 2

Number of trials and percentage (in parenthesis) reporting each item in the CONSORT 2010 checklist and the median average completeness of reporting for trials from 2017 to 2019

Checklist items	2017, n (%)	2018, n (%)	2019, n (%)
Total trials	99	84	93
Title and abstract	59 (60.23)	60 (71.43)	69 (73.03)
Introduction and objectives	97 (98.84)	83 (98.81)	91 (97.75)
Trial design	94 (95.45)	82 (97.62)	91 (97.75)
Setting and eligibility criteria	97 (97.73)	79 (95.18)	92 (98.88)
Intervention details	98 (98.86)	84 (100)	93 (100)
Outcomes	74 (77.91)	69 (81.82)	53 (56.18)
Sample size calculation	70 (72.09)	59 (70.24)	72 (77.53)
Sequence generation	82 (82.95)	67 (79.76)	73 (77.53)
Allocation concealment	54 (56.82)	51 (60.71)	62 (66.29)
Randomization implementation	79 (82.95)	66 (78.57)	69 (73.03)
Blinding details	45 (81.82)	34 (80.95)	47 (97.92)
Statistical analysis	99 (100)	83 (98.81)	93 (100)
Patient flow diagram	57 (59.09)	51 (60.71)	70 (74.16)
Dates of recruitment	56 (54.55)	53 (63.1)	65 (72.73)
Baseline	99 (100)	84 (100)	93 (100)
Numbers analyzed	99 (100)	84 (100)	93 (100)
Outcome estimates	99 (100)	84 (100)	93 (100)
Harm to participants	77 (76.14)	60 (71.43)	74 (78.65)
Limitations	76 (79.55)	65 (77.38)	70 (77.53)
Interpretation	99 (100)	84 (100)	93 (100)
Generalizability	1 (1.14)	7 (8.33)	3 (3.37)
Registration	18 (18.18)	21 (25)	47 (48.31)
Protocol available	18 (18.18)	23 (27.38)	45 (46.07)
Funding	93 (98.86)	71 (84.52)	91 (97.75)
Median average completeness	76	80	84

For the 86 trials where registration number was reported, detailed protocols were not available or accessible for two trial publications. For the trials where the protocols were available online, no incidence of overt outcome switching was observed, except for one study where the secondary outcome and primary outcome were exchanged.

With respect to reporting of key methodological parameters, vis-a-vis sample size calculation random sequence generation, allocation concealment, details of blinding, description of outcomes, and the patient flow diagram – trials published in journals with CONSORT endorsement had significantly lower reporting of sample size calculation, random sequence generation, and allocation concealment than nonendorsing journals. The registered trials reported outcomes, sample size calculation, allocation concealment, and patient flow diagram completely compared to unregistered [Table 3].

Table 3

Comparison of odds of reporting of key methodological indicators with regards to trial registration and CONSORT endorsement

Checklist items	OR	Lower CI	Upper CI	P
Trial registration
Outcome reporting	2.01	1.101	2.032	0.023*
Sample size	3.31	1.646	6.679	0.001*
Sequence generation	2.051	0.993	4.238	0.052
Allocation concealment	4.52	1.601	7.445	0.002*
Blinding details	5.183	0.633	42.41	0.125
Patient flow diagram	10.03	4.474	22.475	<0.0005*
CONSORT endorsement
Outcome description	0.717	0.404	1.271	0.255
Sample size	0.33	0.184	0.594	P<0.0001*
Sequence generation	0.47	249	0.877	0.018*
Allocation concealment	0.45	0.244	0.83	0.011*
Blinding details	0.389	1.504	0.101	0.171
Patient flow diagram	0.329	0.329	0.179	<0.0001

Results are represented as OR and 95% CI. *Binary logistic regression, P<0.05 is considered statistically significant. OR: Odds ratio, CI: Confidence interval

DISCUSSION

RCTs are considered as the gold standard based on which the health-care professionals make evidence-based treatment decisions and recommendations. Thus, the reports of these trials in the form of publication should be of the highest possible quality and free from bias. The CONSORT checklist provides a tool for ensuring and improves the quality of clinical trial reporting.[6] The International Committee of Medical Journal Editors (ICMJE) also encourages its member journals to recommend the authors of articles to follow the CONSORT guidelines for reporting of RCTs.[7]

A study by Goenka et al. comparing the completeness of reporting of CONSORT checklist items reported mean completeness of 57%. In the present study, the median completeness of reporting in the years 2017, 2018, and 2019 were 72%, 76%, and 80%, respectively. This discrepancy could probably be attributed to a difference in the journals that they included compared to the present study.[5]

The present study shows that there has been a statistically significant improvement in the completeness of reporting in studies from 2017 to 2019. A study by Hopewell et al. who compared the quality of reporting of CONSORT checklist items of 1444 articles from 2001 to 2005, also showed that there had been a considerable improvement in reporting quality of articles.[8]

In the present study, major areas of underreporting overall were generalizability, protocol availability, trial registration, date of recruitment, allocation concealment, and patient flow diagram. This pattern has remained similar across the years from 2017 to 2019, which alludes to the fact that reporting of these parameters has not improved significantly with time. A meta-epidemiological study of 146 meta-analyses by Wood et al. found that incomplete or unclear reporting of methodological parameters such as sequence generation, allocation concealment, and blinding can lead to biased estimation of interventions on the trial outcomes.[9] A study by Juni also reported similar findings where there was inflated estimation of treatment effects when there was inadequate reporting of methods versus when it was adequately reported. They have also implied that there has been improvement in the reporting quality after the adoption of stricter reporting guidelines.[10]

In the present study, the trials from the journals that endorse CONSORT 2010 had lesser overall completeness of reporting as well as reporting of key methodological parameters compared to those that did not endorse it. A study by Hopewell et al.[8] assessed 616 studies that were published in journals indexed in PubMed showed that endorsement improved reporting with certain parameters such as primary outcome, sample size calculation, sequence generation, allocation concealment, and blinding.[8] This could probably be explained by the fact that there were considerably larger number of articles from nonendorsing journals than endorsers. However, it nonetheless also implies that CONSORT nonendorsement is not a hindrance to the completeness of reporting of RCTs.

Publication of trial protocols makes it easier to identify any type of inconsistencies with regard to actual trial conduct and the subsequent reporting of its results. In the COMPare (CEBM Outcome Monitoring Project) group study led by Goldacreet al., which analyzed trials published in the top five general medicine journals, only 13% of the articles reported the outcomes exactly as published in the clinical trial protocol.[11] In the present study, the registration of trials did help establish that the outcomes mentioned in the process of registration where same as those in the published article. Thus, clinical trial registration is an important checkpoint for transparent and complete reporting. This can also be observed from the fact that overall completeness of reporting was significantly higher in the registered trials than in the trials which did not provide registration details.[4] Clinical trial registration is an important step toward ensuring transparency in clinical trials. ICMJE mandates the registration of trials and the inclusion of the trial registration number in the final publication.[12] The proportion of registered trials has increased from 2017 to 2019, which is probably the consequence of mandating of trial registration by the Drugs Controller General of India.[13] Despite this, even now, a large proportion of trials as per the corresponding published articles were still unregistered or did not provide registration details. This also hindered the process of verification of any discrepancy in the trial methodology or prespecified outcomes in the current study with regards to these studies.

There are certain limitations to our study. We conducted only a single data extraction, and despite our best efforts, there might have been errors in the process. We also have attempted to reduce the inconsistencies in the process of reviewing. Majority of the trials that were included in the study were obtained from the CONSORT nonendorsing journals, which could have skewed the results since there might be an unequal representation. It is also imperative to understand that incomplete or poor reporting does not necessarily equate to the poor conduct of the trial, and multiple studies have reported that review of protocols has shown good methodological robustness despite poor reporting of the trials.

CONCLUSIONS

The completeness and transparency of reporting vital elements of clinical trials in the scientific articles have improved from 2017 to 2019 in the Indian medical journals, but it is still suboptimal. The study also shows that CONSORT endorsement is not a guarantee against incomplete reporting. There needs to be a concerted effort toward reinforcements of better reporting practices with regards to clinical trials so as to help generate high quality of evidence.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.