Endorsement of reporting guidelines and study registration by endocrine and internal medicine journals: meta-epidemiological study.

OBJECTIVES: To improve the trustworthiness of evidence, studies should be prospectively registered and research reports should adhere to existing standards. We aimed to systematically assess the degree to which endocrinology and internal medicine journals endorse study registration and reporting standards for randomised controlled trials (RCTs), systematic reviews (SRs) and observational studies (ObS). Additionally, we evaluated characteristics that predict endorsement of reporting or registration mechanism by these journals.
DESIGN: Meta-epidemiological study.
SETTING: Journals included in the 'Endocrinology and Metabolism' and 'General and Internal Medicine' 2017 Journal Citation Reports. PARTICIPANTS: Journals with an impact factor of ≥1.0, focused on clinical medicine, and those who publish RCTs, SRs and ObS were included. PRIMARY OUTCOMES: Requirement of adherence to reporting guideline and study registration as determined from the journals' author instructions.
RESULTS: Of the 170 (82 endocrinology and 88 internal medicine) eligible journals, endorsing of reporting standards was the highest for RCTs, with 35 (43%) of endocrine journals and 55 (63%) of internal medicine journals followed by SRs, with 21 (26%) and 48 (55%), respectively, and lastly, by ObS with 41 (50%) of endocrine journals and 21 (24%) of internal medicine journals. In 78 (46%) journals RCTs were required to be registered and published in adherence to the Consolidated Standards of Reporting Trials statement. Only 11 (6%) journals required registration of SRs. Internal medicine journals were more likely to endorse reporting guidelines than endocrine journals except for Strengthening the Reporting of Observational Studies in Epidemiology. No other journal characteristic proved to be an independent predictor of reporting standard endorsement for RCTs besides trial registration.
CONCLUSION: Our results highlight that study registration requirement and reporting guideline endorsement are suboptimal in internal medicine and endocrine journals. This malpractice may be further enhanced since endorsement does not imply enforcement, impairing the practice of evidence-based medicine. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

This is the most contemporary and comprehensive assessment evaluating the extension of which journals in endocrinology and internal medicine are aligned with endorsing reporting guidelines and study registration.

Our systematic approach to answer this question should provide confidence in our results.

Requirement of registration and adherence to reporting guidelines by journals is a surrogate of the use of these mechanisms in each individual study.

Introduction

Evidence-based care requires the application of the best available evidence.1 For clinicians to appraise how the research was conducted,2 studies should be properly reported.3–8 Clinical study registration, required by several organisations,9 10 allows for the detection of publication bias and selective reporting.11–13 We sought to determine the extent to which journals in the fields of endocrinology and internal medicine endorse adherence to reporting guidelines and prospective study registration, and the characteristics associated with journal endorsement.

Methods

This article adheres to the guideline for reporting meta-epidemiological methodology research14 (online supplementary file 1).

Study design

To determine the frequency in which journals require randomised controlled trials (RCTs) and systematic reviews (SRs) to be registered in a public registry web site (eg, clinicaltrials.gov or the International Prospective Register of Systematic Reviews (PROSPERO)), we systematically assessed the instructions for authors of eligible journals. Similarly, we determined whether these journals endorsed the use of Consolidated Standards of Reporting Trials (CONSORT), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) or Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) for RCTs, SRs and observational studies (ObS), respectively.

Study selection

We searched within the InCites Journal Citation Reports by Clarivate Analytics for ‘Endocrinology and Metabolism’ and ‘General and Internal Medicine’ journals that published RCTs, SRs or ObS. This was achieved by examining in the scope of each journal the study designs they consider for publication; if journals failed to state this, we searched on all issues within the last year for published RCTs, SRs and ObS. This procedure was performed in all journals that had a 2017 impact factor of ≥1.0; we arbitrarily set this impact factor threshold to identify ‘best case’ journals with more editorial and quality control processes. Journal eligibility was ascertained after achieving perfect chance-adjusted inter-rater agreement as measured with Cohen’s kappa coefficient in a pilot test of 10 journals. Journals solely focused on basic science, other types of non-patient-oriented research, case reports or reviews were excluded.

Data collection

Using an online spreadsheet form, reviewers extracted journals’ name, country of origin, citation metrics (ie, impact factor), language and region of origin directly from the InCites website. Type of access (open, conventional (paid access) or hybrid (combined open and paid access)), registration requirement for RCTs and SRs, and endorsement of reporting guidelines were obtained from the journals’ online site and its ‘Instructions for Authors’ or equivalent.

A journal endorsed the use of reporting guideline if in its Instructions for Authors it noted that authors must complete a checklist of the reporting guideline of interest (CONSORT, PRISMA or STROBE), along with their manuscript during the submission in order to be considered for publication. A journal endorsed study registration if it noted that authors must provide their clinical trial or SR number from a public registry database (eg, clinicaltrials.gov or PROSPERO, respectively) for their submission to be considered for publication.

Statistical analysis

In addition to descriptive statistics, we used SPSS V.22 to conduct three logistic regression multivariate analyses to assess for possible predictors of trial registration, CONSORT endorsement and simultaneous trial registration and CONSORT endorsement. We included specialty, impact factor, geographical region and type of access as covariates. We also evaluated the association between endorsement of trial registration and of CONSORT adherence. Frequencies and percentages were used to describe categorical variables, while median and IQR were used for continuous variables. Associations were described using ORs and their corresponding 95% CIs. Logistic regression analyses were also attempted for the registration of SRs and the endorsement of PRISMA and STROBE, but a limited number of cases prevented these association analyses.

Patient and public involvement

We did not involve patients or the public in our work.

Results

Of the 296 journals identified, 59 had an impact factor of <1.0, and 67 published only basic science, narrative reviews, animal research, non-patient-oriented research or case reports, leaving 82 endocrine and 88 internal medicine journals for inclusion (figure 1). The list of 170 journals is available in online supplementary file 2. Table 1 reports journal characteristics.

Figure 1

Flow diagram of included journals and reasons for exclusion. JCR, Journal Citation Reports.

Table 1

Characteristics of included journals

Characteristics	Total (N=170)	Endocrinology (n=82)	Internal medicine (n=88)
Citation metrics
Impact factor	2.6 (1.7–3.9)	3.1 (2.4–4)	2 (1.5–3.4)
Impact factor without-self cites	2 (2–4)	3 (2–4)	2 (1–3)
5-year impact factor	2.7 (1.8–4.1)	3.1 (2.4–4.2)	2.1 (1.6–3.5)
Normalised eigenfactor	0.7 (0.3–1.4)	0.7 (0.4–1.5)	0.6 (0.3–1.3)
Article influence score	0.8 (0.5–1.4)	0.9 (0.7–1.4)	0.6 (0.4–1.3)
Language
English	159 (94)	79 (96)	80 (91)
English/non-English	10 (6)	3 (4)	7 (8)
Non-English	1 (1)	0 (0)	1 (1)
Type of access
Hybrid	74 (44)	47 (57)	27 (31)
Conventional	50 (29)	18 (22)	32 (36)
Open access	46 (27)	17 (21)	29 (33)
Geographical region
Europe	91 (54)	47 (57)	44 (50)
North America	49 (29)	26 (32)	23 (26)
Asia	20 (12)	7 (9)	13 (15)
Oceania	4 (2)	1 (1)	3 (3)
South America	4 (2)	1 (1)	3 (3)
Africa	2 (1)	0 (0.0)	2 (2)

Data are in n (%) or median (IQR).

Only 90 (53%) of the 170 journals endorsed the CONSORT statement; 103 (61%) endorsed the RCT registration; and 78 (46%) endorsed both (table 2). STROBE was endorsed by 62 (37%) journals and PRISMA by 69 (41%); only 11 (6%) journals required SR registration.

Table 2

Journal findings by study design and specialty

Parameter	RCT			SR			ObS
	Total (N=170)	Endocrinology (n=82)	Internal medicine (n=88)	Total (N=170)	Endocrinology (n=82)	Internalmedicine (n=88)	Total (N=170)	Endocrinology (n=82)	Internal medicine (n=88)
Reporting guideline
Yes	90 (53)	35 (43)	55 (63)	69 (41)	21 (26)	48 (55)	62 (37)	41 (50)	21 (24)
No	77 (45)	45 (55)	32 (36)	89 (52)	50 (61)	39 (44)	103 (61)	37 (45)	66 (75)
NP	3 (2)	2 (2)	1 (1)	12 (7)	11 (13)	1 (1)	5 (3)	4 (5)	1 (1)
Registration
Yes	103 (61)	47 (57)	56 (64)	11 (6)	2 (2)	9 (10)	n/a	n/a	n/a
No	64 (38)	33 (40)	31 (35)	147 (87)	69 (84)	78 (89)	n/a	n/a	n/a
NP	3 (2)	2 (2)	1 (1)	12 (7)	11 (13)	1 (1)	n/a	n/a	n/a
Reporting guideline and registration
Only reporting guideline	12 (7)	4 (5)	8 (9)	59 (35)	19 (23)	40 (46)	n/a	n/a	n/a
Only registry	25 (15)	16 (20)	9 (10)	0 (0)	0 (0)	0 (0.0)	n/a	n/a	n/a
Both	78 (46)	31 (38)	47 (53)	10 (6)	2 (2)	8 (9)	n/a	n/a	n/a
Neither	52 (31)	29 (35)	23 (26)	89 (52)	50 (61)	39 (44)	n/a	n/a	n/a
NP	3 (2)	2 (2)	1 (1)	12 (7)	11 (13)	1 (1)	n/a	n/a	n/a

Data are in n (%). NP indicates journals that do not publish the relevant study design; n/a indicates that the parameter could not be evaluated due to lack of dedicated registry for ObS.

ObS, observational studies; RCT, randomised controlled trial; SR, systematic review.

Internal medicine journals (OR 3.1, 95% CI 1.3 to 7.1) and journals endorsing RCT registration (OR 17.5, 95% CI 7.2 to 42.8) were more likely to endorse CONSORT, with no discernible significant association with other journal characteristics (table 3). CONSORT endorsement (OR 17.9, 95% CI 7.3 to 43.9) and hybrid access journals (OR 3.8, 95% CI 1.3 to 11.6) were more likely to endorse RCT registration with no discernible significant association with other journal characteristics. We were unable to identify predictors of endorsing both CONSORT and RCT registration.

Table 3

Multivariate analysis of journals that publish RCTs

Predictor variable	OR (95% CI)
CONSORT
Specialty (IM vs Endo)	3.1 (1.3 to 7.1)
Trial registration (yes vs no)	17.5 (7.2 to 42.8)
Impact factor	1.03 (0.99 to 1.08)
Geographical region
North America	Ref
Europe	0.4 (0.1 to 1.2)
Other	1.7 (0.6 to 5.2)
Type of access
Conventional	Ref
Open access	0.71 (0.32 to 1.6)
Hybrid	1.4 (0.6 to 3.2)
Trial registration
Specialty (IM vs Endo)	0.5 (0.2 to 1.2)
CONSORT (yes vs no)	17.9 (7.3 to 43.9)
Impact factor	1 (0.98 to 1.05)
Geographical region
North America	Ref
Europe	0.7 (0.2 to 2.6)
Other	0.4 (0.1 to 1.5)
Type of access
Conventional	Ref
Open access	1.4 (0.6 to 3.7)
Hybrid	3.8 (1.3 to 11.6)
CONSORT and trial registration
Specialty (IM vs Endo)	1.8 (0.9 to 3.5)
Impact factor	1.03 (0.99 to 1.08)
Region
North America	Ref
Europe	0.4 (0.2 to 1.2)
Other	1.0 (0.4 to 2.5)
Type of access
Conventional	Ref
Open access	0.6 (0.3 to 1.4)
Hybrid	1.4 (0.6 to 3.3)

CONSORT, Consolidated Standards of Reporting Trials; Endo, endocrinology; IM, internal medicine; RCT, randomised controlled trial.

Discussion

Approximately half of the journals included in our study endorse CONSORT and more than half endorse RCT registration. Around a third of these journals endorsed PRISMA and <10% required registration. The STROBE statement was endorsed by around a third of these journals. Except for STROBE, internal medicine journals were more likely to endorse reporting guidelines than endocrine journals; journals that endorsed CONSORT were more likely to require RCT registration and vice versa.

The evolution of journal endorsement of reporting guidelines and study registration in the last decade is depicted in figure 2.15–24 Our results showed that the journals we considered were more likely to endorse the CONSORT statement, the most commonly endorsed standard, and other reporting guidelines than journals in prior evaluations and covering other fields. It is possible that these differences occurred due to less journals being included and a different methodology in the selection of the assessed journals in previous research. Less variation across fields is evident in the requirement of RCT registration. The requirement of registration of SRs was not evaluated in the majority of previous studies.

Figure 2

Timeline describing previous studies analysing guideline and registration adherence by specialty. The number of journals assessed in each study is described in parentheses. CONSORT, Consolidated Standards of Reporting Trials; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; STROBE, Strengthening the Reporting of Observational Studies in Epidemiology.

Despite our study showing a suboptimal scenario, that is, the lack of awareness in the scientific community to develop better strategies to advocate for the compliance of reporting and registration of studies, our study has several limitations. For instance, we focused on high-impact journals, which may have overestimated the journal endorsement rate of reporting guidelines and study registration as the included journals may not be a representative sample of all journals in the fields of internal medicine and endocrinology. These fields were chosen because they cover the area of work of the authors; figure 2 shows that journals in these areas, particularly high-impact internal medicine journals at this time, are leaders in endorsing these requirements for complete and full reporting of clinical studies. Also, we relied on the instructions for authors offered on each journal’s website without verifying if additional instructions, including adherence to reporting guidelines or RCT registration, appear at the time of article submission. Perhaps our assessment has not given these journals enough time to consider whether to require prospective registration of observational studies (possible now in sites such as clinicaltrials.gov).25 We did not ascertain whether these endorsements translated into adherence to reporting standards or to prospective trial registration, another reason our results may be overestimated.

In order to provide an evidence-based clinical practice, physicians should be able to identify the best available evidence.26 This evidence, however, might overestimate the effect of an intervention if flaws to conduct, design and/or analysis distort the treatment effect.27 As such, a thorough critical appraisal of this evidence is then mandatory for clinicians. To facilitate this process, reporting guidelines and study registration databases have been created to improve the accuracy, transparency and completeness of manuscripts.28–30 According to our results, however, the degree in which journals in the field of endocrinology and internal medicine endorse this reporting guidelines and study registry is insufficient. Based on our findings, clinicians should be more judicious in the confidence placed on studies whenever they are applying its results to patient care.

Wistfully, despite the WHO’s 2015 position in which they double downed on their initial 2005 statement that ‘the registration of all interventional trials is a scientific, ethical, and moral responsibility’, our results elucidate that endorsement of reporting guidelines and registry of studies in the field of endocrinology and internal medicine is still far from being acceptable.31 Further investigation is needed to explain the reasons behind this apparent reluctance of endorsing the usage of reporting guidelines and study registry by journals. Our study ought to serve as a wakeup call for journals to take an unwavering stance of requiring more transparency and demand for studies to follow reporting guidelines and study registration. Additionally, we need more meta-epidemiological studies to obtain empirical data about the rate in which reporting guidelines and registry of studies are required in the rest of the fields of medicine.

Endorsement of reporting guidelines and study registration remains far from being universal. The reasons behind incomplete adoption—insufficient resources to enforce compliance, the desire to reduce barriers to submission or to maintain a unique editorial look and feel, editorial inertia, the persistence of word or page limits, or the use of alternative practices to improve study publication—remain also unclear. It is the scientific community’s responsibility to ensure full, transparent and complete publication of clinical studies to improve the chance that their endeavours will translate properly into evidence-based care.