The assessment of the quality of randomized controlled trials published in Indian medical journals.

AIM: In this retrospective cross-sectional study, we sought to evaluate if the published randomized controlled trials (RCTs) reported in the year 2017 among the Indian medical journals (IMJs) complied with the Consolidated Standards of Reporting Trials (CONSORT) guidelines and identify domains where reporting could be improved.
METHODS: A literature search was performed using PubMed and Google Scholar to identify all the IMJs that published RCTs in the year 2017. In the archives of the identified journals, the number of published RCTs was identified and the full text was obtained. We selected articles that stated RCT in abstract and title and that evaluated the safety and efficacy of all therapeutic and preventive interventions.
RESULTS: A total of seven IMJs comprising of the Indian Journal of Anesthesia, Indian Journal of Dermatology, Venereology and Leprology, Indian Journal of Pharmacology, Indian Journal of Ophthalmology, Journal of Obstetrics and Gynaecology, Journal of Pharmacology and Pharmacotherapeutics, and Indian Journal of Medical and Pediatric Oncology that published a total of 84 RCTs were included. The mean compliance score of all the RCTs was 13.7 ± 2.66 (57%). Most RCTs had serious reporting deficiencies in the methodology and result sections. DISCUSSION: In spite of journals making it mandatory for prospective authors to comply with the CONSORT guidelines, it is intriguing that there continues to be significant lacunae in reporting RCTs adequately in most IMJs.
CONCLUSION: There is an urgent need to impart training to the medical community of our country in clinical research methods and reporting of RCTs.

BACKGROUND

Clinical research plays a vital role for the progress of medicine and it is considered mandatory for evidence-based medicine (EBM). In EBM, a randomized controlled trial (RCT) represents the cornerstone and is the gold standard method for evaluating the safety and efficacy of a treatment. Moreover, RCTs have the ability to showcase the superiority of a new treatment over placebo or an existing treatment.[1] They are placed at the top of hierarchy after meta-analyses, systematic reviews, and critically appraised literatures. RCTs are done to provide substantial evidence to the clinicians and other health-care providers regarding patient care; it also answers all the patient-related questions and the results of the RCTs are mandatory for any governmental regulatory bodies for the approval of any novel or new treatment. One of the major advantages of RCT is that it has the ability to minimize any biases by controlling all the confounders that may have a negative impact on the study outcomes and the treatment effect.[23] Therefore, it is highly imperative to conduct well-designed controlled clinical trials. According to the International Committee of Medical Journal Editors, the uniform requirements for the manuscripts submitted to biomedical journals state that all the RCTs should follow the Consolidated Standards of Reporting Trials (CONSORT) guidelines.[4] It is mandatory that the author guidelines or instructions to authors of any biomedical journal endorse that all RCTs should be reported according to the CONSORT guidelines. In case the authors fail to follow the CONSORT guidelines, the reviewers of the journal must ensure that the manuscript has been revised by the respective authors prior to its publication. Yet, poor reporting of RCTs has been observed in some of the Indian medical journals (IMJs). An observational study by Tharyan et al. was conducted in the past to evaluate if the editorial policies and the reporting quality of RCTs had improved. The study revealed that the Clinical Trials Registry of India (CTRI) template mediated the reporting of valid methods in registered trial protocols.[5] However, it was found that the editors of IMJs were not sufficiently monitoring the reported RCTs to ensure that the results of RCTs were reliable and valid; only one-third of the IMJs provided instructions to follow the CONSORT guidelines and one-half recommended the ICJME requirements. We thereby sought to evaluate if the published RCTs reported in the year 2017 among the IMJs complied with the CONSORT guidelines and identify domains where reporting could be improved.

METHODS

Data sources and Study Selection

This cross-sectional study that evaluated the standards of reporting RCTs was conducted in the Department of Clinical Pharmacology, SRM Medical College Hospital. We performed a literature search using PubMed and Google Scholar to identify all the IMJs that published RCTs in the year 2017. After identifying the IMJs that published RCTs, we did a search in the archives of the identified journals to identify the number of published RCTs and obtained the full text of all the RCTs that were published in the year 2017. We looked for all the RCTs by screening the title and abstract. We selected papers that stated RCT in abstract and title and that evaluated the safety and efficacy of all therapeutic and preventive interventions. We included all the studies which had claimed to be a RCT and which evaluated the safety and efficacy of all therapeutic and preventive interventions. We excluded studies which were observational studies, editorials, review articles, systematic reviews and meta-analysis, case reports, case series, and letter to editors. The Beall's list was used to eliminate predatory journals.

Data extraction

The compliance of each RCT was then checked using the CONSORT 2010 checklist. The CONSORT 2010 checklist comprised of 25 items that mainly focused on how the trial was analyzed, interpreted, and designed. The CONSORT 2010 checklist comprised of a flow diagram that displayed the progress of all the trial participants throughout the trial. The main domains of the CONSORT 2010 checklist included title and abstract, introduction, methods, results, discussion, and other information (Moher et al.).[6] For each of the 25 items, the RCTs were given a score of 1 for fully reporting and 0 for not reporting. A CONSORT Compliance Score (CCS) was calculated based on the above scoring. Two reviewers performed the study selection, study review, and scoring of studies independently. The disagreements between the two reviewers were solved through discussion and by the opinion of a third reviewer when necessary.

Data analysis

Descriptive statistics were described as frequencies and percentages. The data were analyzed using Microsoft Office Excel version 2007 Microsoft (USA).

RESULTS

A total of 53 IMJs were available in PubMed and Google Scholar. Out of the 53 IMJs, a total of 7 (13.21%) IMJs had published RCTs in the year 2017 [Table 1]. The seven IMJs published a total of 84 RCTs, which constituted 23% of the published original articles in these journals. The bulk of the RCTs were published in the Indian Journal of Anesthesia (IJA) 60 (71.43%) which was followed by Journal of Obstetrics and Gynaecology (JOG) 7 (8.33%), Indian Journal of Ophthalmology (IJO) 6 (7.14%), Journal of Pharmacology and Pharmacotherapeutics (JPP) 5 (5.95%), Indian Journal of Dermatology, Venereology and Leprology (IJDVL) 4 (4.76%), Indian Journal of Pharmacology (IJP) 1 (1.19%), and Indian Journal of Medical and Pediatric Oncology (IJMPO) 1 (1.19%). The mean compliance score of all the RCTs was 13.7 ± 2.66 (57%). The adherence of RCT reports to CONSORT 2010 statement has been reported [Table 2]. The journal with the highest overall CCS was observed for the JOG (75%) followed by IJMPO and IJDVL. The overall CCS for other journals was as follows: IJDVL (61.76%), IJMPO (61%), IJA (56.82%), IJO (54.61%), JPP (52.73%), and IJP (52.17%) [Figure 1]. Most RCTs had serious reporting deficiencies in the methodology and result sections. In the methods section, the highest score was obtained by the following domains: interventions (97.02%), statistical methods (89.88%), blinding (65.87%), and allocation concealment (59.52%) [Figure 2]. In the results' section, the highest score was obtained by the domains, outcomes and estimation (94.04%), and baseline data (84.52%) [Figure 3]. Only 14 (16.5%) RCTs were registered in their respective clinical trial registries and 2 (14.3%) RCTs had no access to full protocol. There was no clarity on the trial design, type of randomization used, and implementation procedures for randomization and allocation concealment. There were glaring deficiencies even in the type of analysis used, outcomes, ancillary analysis, and reporting harms.

Table 1

List of Indian medical journals that published randomized controlled trials in the year 2017

Indian Journal of Anesthesia

Indian Journal of Dermatology, Venereology and Leprology

Indian Journal of Pharmacology

Indian Journal of Ophthalmology

Journal of Obstetrics and Gynaecology

Journal of Pharmacology and Pharmacotherapeutics

Indian Journal of Medical and Pediatric Oncology

Table 2

Adherence of randomized controlled trial reports to the Consolidated Standards of Reporting Trials statement 2010 checklist

Checklist of items in CONSORT 2010 checklist	Percentage of adherence to CONSORT 2010 checklist (n=85)
Title and abstract	81
Introduction	98
Methods	49
Results	51
Discussion	79
Other information	36

CONSORT=Consolidated Standards of Reporting Trials

Figure 1

The Overall Consolidated Standards of Reporting Trials compliance score (%) obtained by each journal in the year 2017

Figure 2

The percentage compliance (%) in the methods section of included randomized controlled trials of all Indian medical journals in the year 2017

Figure 3

The percentage compliance (%) in the results section of included randomized controlled trials of all Indian medical journals in the year 2017

DISCUSSION

This study was performed to evaluate the quality of reporting RCTs based on their compliance with CONSORT 2010 checklist. One of the main findings of our study was that the mean compliance of all the RCTs published in IMJs was found to be only 57%. In another earlier study performed in the year 2016, the mean compliance of studies that were published in journals that endorsed the CONSORT guidelines was 69.5% and the mean compliance for articles published in journals that did not endorse CONSORT was 51.9%.[7] This decline in the compliance to CONSORT might be due to the fact that the study performed by Rikos et al. included RCTs published worldwide, while our study only included RCTs published in IMJs.[7] In another study reported by Stevely et al., the compliance with CONSORT was evaluated based on a total of 37 CONSORT checklist items. The median proportions (interquartile range) of RCTs that met the complete and at least partial compliance in reporting the criteria of checklist items were 81% (53%–91%) and 93% (78%–97%), respectively.[8] Thus, partial compliance with CONSORT is still the norm in most IMJs. The reasons for partial compliance could be ignorance of the authors, lack of training on the nuances of doing a clinical trial, and perfunctory reviewing by the reviewers of these articles or nonchalance on the part of the editor to ensure compliance with CONSORT.

Most of the published RCTs had serious reporting deficiencies in the methodology and result sections. In the methodology section, only four domains namely interventions, allocation concealment, blinding, and statistical methods were reported by more than 60% of authors. The other sections such as trial design, randomization, and sample size had poor reporting of only 8.33%, 20.23%, and 39.88%, respectively. The present findings were in agreement with regard to the compliance of sample size calculation (59.5%) performed in earlier studies, where the adherence to statistical methods was evaluated.[910] In contrast to our findings, a study that assessed the quality of reporting RCTs published between 2012 and 2015 in a medical college and associated hospitals in New Delhi found that only few studies reported allocation concealment (28.5%).[10] Hence, there has been a slight improvement in reporting the methodology section. It is not surprising that the only one item in methodology that reported more than 90% compliance was the type of interventions given to the patients and details regarding the administration of the study drug. Another item that showed a 70% compliance rate was the statistical analysis, which could be due to the stringent review in this aspect from the journal as such. Our study also showed that less than 50% of studies reported the method of randomization used. This is certainly a glaring omission in an RCT, as failure to use an appropriate method would undermine the very foundation of an RCT as the study is no more devoid of selection bias. It is a common misconception among authors to consider methods such as alternate administration of drug with comparator and using the days of the week or the date of birth ending with odd or even number as appropriate methods of randomization.[11] It is not known if any of the above methods were used among those articles which remained silent about the method of randomization and allocation concealment used.[6] In a study that assessed pediatric dentistry trials, there was poor reporting of randomization techniques (5%–9%) and sample size calculation (4%).[12] Such deficiencies in methods, which have been reported earlier,[13141516] have a greater propensity to result in larger treatment effects than well-conducted RCTs.

One of the main concerns with the result section in the present study is with respect to the recruitment, i.e., dates defining the periods of recruitment and follow-up and why and when the trial was halted. Two of the earlier studies too reported poor compliance to recruitment. In both the studies, the compliance to recruitment was found to be only 29% and 46%, respectively.[917] The compliance to CONSORT with respect to the participant flow diagram for our study was 60%. Our study was in consensus with an earlier study where the compliance with respect to CONSORT was 62%.[17] RCTs must report regarding the number of patients who were randomly allocated with the treatment, the number of patients who actually received the allocated treatment, and the patients who were analyzed for the outcomes. It is also essential that all the RCTs report the losses and exclusion of patients in each group. The deficit in reporting the above-mentioned information might lead to issues such as the sample failing to be representative of the actual population, losses in comparability between treatment arms, and reduction in the statistical power. Therefore, this might lead to misinterpretations and misconceptions regarding the effect of treatment that is being studied.[181920] The compliance to baseline information of the study participants was almost similar for both the studies, with 84.5% for the present study and 90% for the study performed by Juneja et al. Our study showed that most of the RCTs reported in the IMJs had reported all the outcomes. Therefore, there was a significant improvement with respect to the earlier studies.[17] It is common to see investigators carrying out selective reporting of positive outcomes, but it is heartening to note that we did not observe the same in our study.

Another surprising fact regarding the reporting of trials was that only 16.5% of the RCTs were registered in their respective clinical trial registries. Among the 14 trials that were registered in the clinical trial registry, only 2 of the trials had full access to the clinical trial protocols. Even in an earlier study, only two (7.14%) of the RCTs were registered in the clinical trial registry platform and the information regarding the access to full text was reported in eight (28.5%) studies.[9] Therefore, it is evident that the investigators still neglect to register their clinical trials in their respective clinical trial registries and the reviewers also neglect to verify the information while reviewing the manuscript. According to the regulations of ICJME, it is mandatory that every clinical trial has to be registered in the respective clinical trial registries and the publication should contain the clinical trial registration number. The registration of clinical trials will improve the transparency regarding clinical trials for all the health-care providers and patients, which might also encourage participants to be part of clinical trials that will be beneficial and provide possibility for potential research collaboration.[212223] One of the main limitations of our study was that we had evaluated the studies for only a single year. We did not compare our study across multiple years to observe for any shift in the pattern of reporting. Another limitation of our study was that we did not compare the quality of studies evaluated in the current study with the studies published in International Medical Journals.

CONCLUSION

The main finding of our study was that the RCTs published in the IMJs had very poor compliance with the CONSORT 2010 guidelines. In spite of journals making it mandatory for prospective authors to comply with the CONSORT 2010 guidelines, it is intriguing that there continues to be significant lacunae in reporting RCTs adequately in most IMJs. Moreover, according to the Medical Council of India regulations, it has become mandatory for every clinician to have research publications for career advancement. This regulation has created tremendous pressure on the medical community, who have not been adequately trained with the tools of scientific writing, to publish scientific manuscripts. Henceforth, there is an urgent need to impart training to the medical community of our country in clinical research methods and reporting of RCTs.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.