Are published randomized clinical trials abstracts on periodontics reported adequately?

INTRODUCTION: Randomized clinical trials (RCTs) remain the golden standard in biomedical research, which makes their reporting to a high quality essential to control RCTs' internal validity. The purpose of this study was to evaluate the quality of abstract reporting of RCTs published in periodontic journals and their compliance with the CONSORT guidelines.
METHODS: A hand search was undertaken to identify RCTs published in three periodontic journals [1] Journal of periodontology (JOP) [2], the Journal of periodontal research (JOPR) and [3] the Journal of clinical periodontology (JOCP) from 2015 to 2018.The completeness of abstract reporting was evaluated with a modified CONSORT for abstracts statement checklist.
RESULTS: Abstracts of 177 randomized controlled trials were identified and assessed. The distribution of published reports was in the Journal of periodontology (JOP), (42%) the Journal of periodontal research (JOPR) (7%) and the Journal of clinical periodontology (JOCP) (51%). The mean overall reporting quality score was 49.0%(95% CI: 47.7-50.2%). Most of the abstracts (91-100%) clearly reported and described the study design as randomized in the RCTs' title and recruitment status, as well as study interventions, objective(s), outcome(s) and conclusions. There was insufficient description and reporting of the authors' contact details, trial design, method of randomization, blinding, number of analyzed participants per group, harms, trial registration and source of funding.
CONCLUSIONS: The quality of reporting of abstracts of randomized controlled trials in periodontic journals is suboptimal. In view of the current guidelines of reporting RCTs abstracts, efforts should be made to better reporting.

Introduction

As science is currently advancing at a swift pace, and considering the enormous amount of research published annually, randomized clinical trials (RCTs) remain the gold standard in biomedical research [1]; however, the trials' critical findings and application, in practice, relies on their internal validity, which is based on the execution of an adequate methodology, study design, and interpretation of the findings. The high quality reporting of each component is essential for controlling RCTs’ internal validity [2,3].

Currently, all journals require a limited number of words in the abstract section, leading the authors to compromise the quality of reporting. However, the abstract is an essential component of the published article, as it serves as the foundation for the initial screening in any systematic review. In the hierarchy of evidence, the only type of research that is considered to have higher evidence than RCTs is systematic reviews and meta-analyses [4]. This type of research is based on screening a considerable number of published articles in order to include or exclude them in the review, and this process relies mainly on the abstract section.

In order to assess the quality of a scientific publication , Using an objective tool becomes essential to avoid bias in assessment; therefore, several objective scales, including individual markers and checklists, were suggested to serve this purpose [5]. In a systematic review, 21 scales for assessing the quality of RCTs were found [6]; nonetheless, not all of them were valid and reliable. As a result of these discrepancies, the Consolidated Standards of Reporting Trials (CONSORT) statement was published [7]. CONSORT was developed to formulate the guidelines for reporting RCTs in order to improve the quality of research reporting in the medical field (www.consort-statement.org). More than 585 journals endorse CONSORT, which represents over 50% of the core medical journals listed in the Abridged Index Medicus on PubMed. The latest CONSORT update regarding the abstract reporting of RCTs includes the reporting of 19 essential items [8].

The quality of the reporting of RCTs in the medical field was investigated in regards to their methodologies and findings; it was found that a poor correlation exists between the quality of the published RCTs and their reporting [9,10]. Another study was conducted with the objective of assessing the quality of the abstract reporting of RCTs published in dental journalsfound that the abstract reporting was suboptimal according to the CONSORT guidelines [11]. Therefore, the recommendation to adhere to the CONSORT guidelines was published to enhance the quality of the published RCTs [11,12].

The aim of this study was to evaluate the quality of the abstract reporting of RCTs published in periodontic journals and their compliance with the CONSORT guidelines, as well as to note the areas that need improvement regarding the published RCTs in the periodontic journals.

Materials and methods

Three leading periodontic journals [1]: the Journal of Periodontology (JOP) [2], the Journal of Periodontal Research (JOPR), and [3] the Journal of clinical Periodontology (JOCP), were selected based on the current top ranking impact factor rating. A hand search was undertaken to identify the RCTs that had been published in these periodontic journals from 2015 to 2018. Human trials were included, while in vitro, laboratory-based trials, and conference abstracts were excluded. The keywords “randomized controlled trial”, “randomized controlled trial”, “assigned”, “prospective” or “comparative,” were screened in the title and abstract and then the full text was retrieved for all articles that included one or more of these terms. A literature search was undertaken independently and in duplicate by two authors (AA and FA), with disagreements being solved through open discussion between the authors. One author (AA) screened the potential RCTs using a piloted extraction sheet. A score value was assigned according to the CONSORT abstract items guidelines [8]. Each item was scored either ‘Yes’ if present, ‘No’ if absent, or not applicable ‘NA’ [11]. An item was scored as NA if the design of the study made it impossible to include it. The total score for each trial was calculated and converted into a percentage using the equation: total score= (total number of ‘Yes’/[19{total number of ‘NA’ items])/100.

Additional information, including the number of authors, the continent and country of the first author and the clinical setting of the trial, was also recorded for each article. The authors were calibrated by scoring 10% of the included abstracts together, referring directly to the CONSORT checklist (Fig. 1) and the associated explanations. A random sample of 10% of the papers was scored by a second examiner to the assess inter-examiner reliability of the CONSORT score. To test the intra-examiner reliability, a further random sample of 10% of the papers was scored a second time by one of the authors (AA) three months after the initial data collection had been completed.

Fig. 1

CONSORT for abstract checklist for items to include when reporting a randomized clinical trial.

Statistical analysis

Descriptive statistics and the percentage compliance related to the CONSORT checklist items were reported for the published RCTs. A linear regression with a univariate analysis was carried out using SPSS 22.00 (IBM SPSS Statistics for Macintosh, Version 22.0. IBM Corp, Armonk, NY) to identify the variables associated with the mean CONSORT score. Inter-correlation coefficient (ICC) tests were used to assess the inter- and intra-examiner reliability.

Results

Using ICC tests, the inter- and intra-reliability levels associated with scoring the abstract reporting were high, at 0.93 and 0.98, respectively.

From January 2015 to December 2018, one hundred seventy-seven (177) RCTs were identified out of 1875 articles (approximately 9.4% of the published articles in all journals) in the three periodontic journals (Table 1). The number of published RCTs in the three leading periodontic journals over the 4-year period was almost equally disturbed (ranging from 27 to 29%), except for the year 2018, where the percentage of RCTs constituted approximately 15.2% of the total number of publications (Table 1). According to the home continent of the papers’ first author, Europe produced more than half of the published RCTs (54%) while RCTs for which the first author was based in Africa and Australia represented 3.9% of the total published RCTs in the reviewed journals (Table 1). Most of the included RCTs were published in the Journal of Clinical Periodontology (51%) (Table 1).

Table 1

Characteristics of the 177 RCTs.

Characteristic	Number of Publications	Percentage%	Mean Score	SD	95% CI
Journals
JOP	74	42%	46	8.8	44.0 to 48.1
JOPR	13	7%	54.2	10.1	48.1 to 60.3
JOCP	90	51%	50.6	7.0	49.1 to 52.1
Year
2015	50	28.2%	50.2	7.5	48.0 to 52.3
2016	48	27.1%	48.1	8.5	45.6 to 50.6
2017	52	29.3%	47.1	9.6	44.4 to 49.8
2018	27	15.2%	51.8	6.8	49.1 to 54.5
Authors
> 4	23	12.9%	47.3	5.9	44.8 to 49.9
4 to 6	95	53.6%	48.5	8.2	46.8 to 50.2
< 6	59	33.3%	50.4	9.5	47.9 to 52.8
Continent
AFRICA	5	2.8%	51.5	6.8	43.0 to 60.0
Australia	2	1.1%	42.1	0.1	41.3 to 42.9
ASIA	34	19.2%	46.2	10.2	42.7 to 49.8
EUROPE	97	54%	49.8	7.6	48.2 to 51.2
North AMERICA	17	9.6%	48.2	6.2	45.0 to 51.5
SOUTH AMERICA	22	12.4%	50.2	10.3	45.6 to 54.8
Overall	177		49.0	8.4	47.7 to 50.2

In the majority of the included RCTs (98.3%), the first author worked in an academic institution, and more than half of the included trials (53.6%) had four to six authors, but only a small number of RCTs included the formal involvement of a statistician in the trials (9.6%) (Table 2). Most of the included RCTs were undertaken in university settings (98.3%), while RCTs undertaken in a private clinic setting represented 1.7% of the total number of included RCTs (Table 2).

Table 2

Number of reports and distribution by settings, work environment and statistician involvement.

	Number of Publications	Percentage
Setting
Private	3	1.7%
University	174	98.3%
Work in Academia
Yes	174	98.3%
No	3	1.7%
Statistician Involvement
Yes	17	9.6%
No	160	90.4%

The mean CONSORT score for all of the trials’ abstracts was 49.0% (95% CI: 47.7 to 50.2). In the univariate analysis, the RCTs published by the JOCP had significantly higher CONSORT mean scores for abstract reporting compared with the JOP, however no significant difference was found between the JOP and the JOPR (Table 3). From 2015 to 2018, the mean CONSORT score for the abstracts by year of publication ranged from 47.1% to 51.8% (Fig. 2). RCTs that included more than six authors had the highest CONSORT score for their abstract (mean score 50.4, 95% CI: 47.9 to 52.8). Another finding in this study was that the five African RCTs published in the three main periodontic journals had the best abstract reporting quality, with a wide confidence interval (mean score 51.5, 95% CI: 43.0 to 60), while the two Australian RCTs had the lowest overall CONSORT score for their abstract (mean score 42.1%, 95% CI: 41.3 to 42.9) (Table 1). The Europe-based RCTs were associated with better abstract reporting quality (49.8%, 95% CI: 48.2 to 51.2) compared to the Asia-based ones, which was statistically significant, as the univariate analysis revealed.

Table 3

Univariate linear regression derived coefficients (B) and 95% confidence interval with mean score of compliance with CONSORT as dependent variable for 177 RCTs.

Predictor variables		Univariate analysis
Variable	Category or unit	B	95% CI
Journals	JOP	Baseline (reference)
	JOPR	8.2	3.4 to 13.0*
	JOCP	4.6	2.0 to 7.1*
Continents	Europe	Baseline (reference)
	Africa	1.8	−5.9 to 9.4
	Asia	−3.5	−6.8 to −0.2*
	North America	−1.50	−5.9 to 2.9
	South America	0.40	−3.5 to 4.4
Year	2015	Baseline (reference)
	2016	−2.1	−5.4 to 1.3
	2017	−3.0	−6.3 to 0.2
	2018	1.6	−2.3 to 5.6
Number of authors	4 to 6 authors	Baseline (reference)
	Less than 4	−1.2	−5.0 to 2.7
	More than 6	1.9	−0.90 to 4.6
Statistical significance of main finding	No	Baseline (reference)
	Yes	- 4.9	−12.5 to 2.6

Fig. 2

Mean CONSORT score according to the year of publication.

With regard to the CONSORT items, most of the abstracts (91–100%) clearly reported and described the study design as randomized in the RCTs' title and recruitment status, as well as study interventions, objective(s), outcome(s) and conclusions. However, there was insufficient description and reporting (7–50%) of the authors’ contact details, trial design, method of randomization, blinding, number of analyzed participants per group, the adverse effect of the intervention (harm), trial registration and source of funding. Trial registration (7%) and harm (2%) were the poorest reported items (Table 4).

Table 4

Reported items in the modified 19- item CONSORT checklist for abstracts.

Item Reported	All Journals	AJOP	JOPR	JOCP
Title	100%	100%	100%	100%
Authors/contact details	25%	6%	100%	30%
Trial design	48%	79%	42%	23%
Description and types of participants	33%	15%	58%	44%
Description of interventions	100%	100%	100%	100%
Description of the objective	100%	100%	100%	100%
Primary outcomes description	100%	100%	100%	100%
Method of randomization	8%	5%	8%	11%
Clinician blinding	9%	15%	17%	3%
Patient blinding	8%	11%	17%	4%
Assessment blinding	10%	11%	0%	10%
Number of participants randomized to each group	25%	27%	42%	22%
Recruitment	91%	83%	100%	96%
Number analyzed	15%	17%	25%	13%
Description of the final outcomes (results)	97%	94%	100%	99%
Harms	2%	3%	0%	1%
Conclusions	99%	100%	100%	99%
Trial registration	7%	5%	0%	10%
Funding	50%	0%	17%	97%

Discussion

Assessing the quality of research in different fields of medicine and dentistry has been ongoing for a long time to ensure the validity and quality of the published research(9–14); as a result, several scales have been developed throughout the years to enable readers, researchers, reviewers, and editors to evaluate the quality of published RCTs. In orthodontics, for example, Jadad [3] developed a simple quantitative scale for assessing RCTs, and this scale was used to assess published RCTs in orthodontic journals from 1989 to 1998(15). Later, after the CONSORT assessment checklist was published, 117 RCTs in orthodontics were assessed between 2006 and 2011, and the findings of this study showed that these RCTs had been inadequately reported [[9], [10], [11], [12], [13], [14], [15]]. Another study looked at the RCTs published in orthodontic journals that have endorsed the CONSORT guidelines for reporting RCTs and observed that, despite this endorsement, a deficit in the RCT reporting quality still existed [16].

Several studies have assessed the quality of RCTs in different areas over the years and, in both the dental and medical field [14], they all have reached the conclusion that the reporting quality of the published articles is suboptimal. This conclusion has led many authors to recommend closer adherence to the CONSORT guidelines [9,17,18].

One study examined the quality of the reporting of RCTs in the same three journals for the period 1996–1998 and concluded that this was poor [19]. The primary issue with this study is that the RCTs assessed were published in journals that had not endorsed the CONSORT guidelines at the time when the study was conducted. Moreover, although the study duration was shorter than that of the current study, a higher number of RCTs were reported, which may be attributed to the initial search methodology employed. A recent study was published that assessed the quality of RCTs in periodontics in 2012 [20]; the researchers concluded that the overall reporting quality was poor, and suggested that compliance with the CONSORT guidelines would help to raise the quality of the RCTs being published. The search was performed electronically via Pubmed, without any restriction on the journals, using the Mesh terms “Periodontal Diseases” so, because not all of the RCTs had complied with the CONSORT guideline that the term “RCT” should be mentioned in the title, a number of RCTs may have been unintentionally excluded. In our study, we included studies published in three periodontics journals with the highest impact factor in the field, where all of the included articles met the CONSORT recommendation to mention that the study was a RCT in the title.

An interesting finding of this study is that, in most of the papers (75%), the author(s) failed to provide a physical or email address for correspondence, despite mentioning the affiliation of all authors of the paper. This diverges from the perfect reporting in other dental specialties [13,16,21]. Nonetheless, the reporting of author(s)’ contact details improved throughout 2017–2018.

As the abstract component is of utmost importance in regard to reporting, especially when a systematic review or meta-analysis is conducted [4], the mean CONSORT score for abstract reporting for the RCTs included in this study was deemed to be suboptimal (mean score 47.9%), although all of the journals had endorsed the CONSORT guidelines. The main deficiencies in the abstract reporting were observed to be associated with the author(s)’ contact details, the trial design, method of randomization, blinding, number of analyzed participants per group, adverse effect of the intervention (harm), trial registration, and the source of funding. An investigation into the effect of the editor's implementation of the CONSORT guidelines on the abstract reporting of RCTs was conducted, and it was observed that the implementation of these guidelines may help to improve the RCTs' reporting quality [10]. This improvement was also observed when the quality of the RCTs were assessed before and after the implementation of the CONSORT guidelines by the American Journal of Orthodontics and Dentofacial Orthopaedics [12].

Conclusion

This study observed suboptimal quality of abstract reporting in published RCTs. After reviewing the literature that addressed the effect of adhering to the CONSORT guidelines and the effect of this on the quality of the reporting, it is clear that the abstracts of published RCTs are failing to comply with the CONSORT guidelines and that greater efforts should be made by reviewers and editors to control abstract reporting.

Protocol registration

The study protocol was not registered

Key findings

The abstracts of published RCTs in periodontics are failing to comply with the CONSORT guidelines and that greater efforts should be made by reviewers and editors to control abstract reporting.

Declaration of competing interest

The authors did not receive any funding or have financial interest for undertaking this review.