Is there any quality improvement in the randomized controlled trial abstracts in the Korean Journal of Anesthesiology after the publication of the CONSORT abstract guidelines in 2008?

We evaluated whether there has been an improvement in the quality of randomized controlled trial (RCT) abstracts in the Korean Journal of Anesthesiology (KJA) after the publication of the CONSORT abstract guidelines in 2008. All RCT abstracts published in 2006 and in 2012 were selected as pre-CONSORT and post-CONSORT abstracts, respectively. From the retrieved articles, 102 human RCT abstracts were selected as a pre-CONSORT abstract and 52 were selected as a post-CONSORT abstract. Two reviewers scored every individual item as '0' if the abstract does not adhere to the criteria recommended in the CONSORT abstract statement and scored individual items as '1' if the information suggested by the CONSORT statement was described followed. The k-statistic test demonstrated high agreement between the reviewers (k = 0.914, 95% confidence interval [CI]: 0.878, 0.95]). We compared the frequencies of each item in the abstracts of the pre-CONSORT and post-CONSORT periods using z-tests with Yates correction. More than 90% of the articles from the pre-CONSORT sample group described the five checklist items of intervention (96.1%), objective (95.1%), method-outcome (95.1%), results-outcome (96.1%) and conclusions (100%). With the above-mentioned five checklists, the "randomization" of other sub-items was found in 92.3% of the RCT abstracts in the post-CONSORT sample group, whereas the title, the number of patients analyzed, the harmful effects, and the funding source were noted in less than 20% of the abstracts in both the pre- and post-CONSORT eras. The significant improvement from the pre-era to the post-era samples was related to the trial design, randomization, blinding, and number randomization at a rate increasing from 0 to 53.8% (P < 0.001), from 71.6 to 92.3% (P = 0.006), from 7.8 to 21.2% (P = 0.033) and from 33.3 to 78.8% (P < 0.001), respectively. We calculated the proportion of items which complied with the CONSORT abstract checklist. The mean proportions were 41.7 and 53.0% in the pre- and post-CONSORT period, respectively (P < 0.001, 95% CI for difference: -0.155, -0.071). Absolute points increased significantly at a rate of 11.3% over time (P < 0.001). CONSORT for abstracts aims to improve the reporting accuracy and quality of trial reports and recommends what information should be reported in the constraints when describing a RCT. The results of this article suggest that the reporting quality of RCT abstracts of KJA published in the post-CONSORT period improved, but the quality did not reach the expected level according to the recommendations for reported abstracts. The rate of fulfillment was low, at 41.7%, among pre-CONSORT abstracts, whereas in the post-CONSORT abstracts, the rate was 53.0%, indicating that there has been a significant improvement (11.3%) in the reporting quality over time (P < 0.001, 95% CI for difference: -0.155, -0.071). Authors publishing RCTs in KJA in the post-CONSORT period paid more attention to the trial design, randomization, blinding, and number randomization, identifying the randomized properties of the study more consistently. Authors used 'randomization' in the abstract more frequently over time. They appeared to pay more attention to reporting the title, numbers analyzed, harmful effects, and funding source over time but failed to reveal statistical significance. Appropriate explanations of blinding procedures, side effects, trial registrations and funding sources were poor (Table 1). Although the registration of the clinical research trial with the primary national clinical research trial site, such as the Korea Clinical Research Information Service, is strongly recommended by the KJA editorial board, no trials were published in both the pre- or post-CONSORT eras (Table 1). Explanations of intention-to-treat in the abstract were not given at all (Table 1). Although numerous studies have investigated the quality of RCTs in the past, the quality of RCT abstract has received more attention recently. Chimire et al. [1] found that 11.8% of the 271 RCT abstracts in four high-impact general medical journals contained instances of allocation concealment and only 21.0% reported the details of the blinding procedure used. Reports of allocation concealment in the four high-impact medical journals were described poorly compared to those in pre- and post-CONSORT abstracts in the KJA (Table 1). Statements on details of Institutional Review Board approval for clinical research were increased significantly in the post-CONSORT samples (P < 0.001); however, descriptions of funding sources in the KJA were poor (Table 1). Abstracts reporting RCTs would benefit from a structured approach that ensures more detailed reporting of eligibility criteria because the quality of reporting is suboptimal in many RCT abstracts and the shortcomings arise more frequently in unstructured than structured abstracts. It is necessary for a researcher to describe the key elements of the trial design, the interventions, and the results in the RCT abstract clearly in order to convey information to the reader correctly. Our results indicate that certain components are lacking in KJA abstracts, especially related to allocation concealment, blinding, intention-to-treat and subject recruitment, which are methodological issues. Although many authors report whether patients are randomized, information about the method of allocation concealment was not described adequately. If allocation concealment is inadequate, RCTs tend to report approximately 40% more overstated treatment effects [2]. If patients can decide to which group they belong, selection bias is inevitable [3]. The fact that the CONSORT abstract checklist has not been endorsed by the KJA is another possible reason associated with the low quality of these abstracts. Greenfield et al. [4] found an increase in the frequency of the reporting of some items after the endorsement of the CONSORT guidelines by anesthesia journals. Plint et al. [5] demonstrated that the endorsement of the CONSORT guidelines as an editorial policy is related to a significant improvement in the trial quality.

Table 1

Values Associated with the Methodologic Quality of the Randomized Controlled Trials Published in the Korean Journal of Anesthesiology in the Pre- and Post-CONSORT Periods

	Pre-CONSORT (%) (n = 102)	Post-CONSORT (%) (n = 52)	Differences of sample proportions	95% Confidence interval
CRIS	0	0
Fund	6 (5.9)	8 (15.4)	-0.095	-0.191, 0.001
IRB	59 (57.8)	50 (96.2)	-0.384	-0.536, -0.232
Multicenter	21 (20.6)	12 (23.1)	-0.025	-0.162, 0.112
Double blinding	9 (8.8)	12 (23.1)	-0.143	-0.258, -0.028
Follow-up loss	1 (1.0)	10 (19.2)	-0.182	-0.268, -0.096
Allocation concealment	39 (38.2)	34 (65.4)	-0.272	-0.439, -0.105
Intention-to-treatment	0	0
Mean Jadad score	0.48	1.08

Values are presented as the number of studies (percentages). CRIS: Korea Clinical Research Information Service. IRB: Institutional Review Board.

Z-tests and Yates correction were conducted as statistical analyses. Numerical data were assessed by t-tests.

We believe that the quality of the KJA will improve if the editorial board of the KJA adopts the CONSORT checklist for abstracts with an open mind. Since the publication of the CONSORT checklist for RCT abstracts in January of 2008 and a revision of the guideline of instructions for authors of the KJA in 2009, there has been minimal improvement in the reporting quality of KJA RCT abstracts published in the post-CONSORT era. We recommend that the KJA editorial board endorse the use of the CONSORT abstracts by changing the "Instructions to Authors" section. It needs to be investigated in future studies whether adoption of the CONSORT abstract checklist for RCT by the KJA editorial board and researchers improves the reporting quality.