Transparency in surgical randomized clinical trials: cross-sectional observational study.

BACKGROUND: RCTs provide the scientific basis upon which treatment decisions are made. To facilitate critical review, it is important that methods and results are reported transparently. The aim of this study was to explore transparency in surgical RCTs with respect to trial registration, disclosure of funding sources, declarations of investigator conflicts and data-sharing.
METHODS: This was a cross-sectional review of published surgical RCTs. Ten high-impact journals were searched systematically for RCTs published in years 2009, 2012, 2015 and 2018. Four domains of transparency were explored: trial registration, disclosure of funding, disclosure of investigator conflicts, and a statement relating to data-sharing.
RESULTS: Of 611 RCTs, 475 were eligible for analysis. Some 397 RCTs (83.6 per cent) were registered on a trial database, of which 190 (47·9 per cent) had been registered prospectively. Prospective registration increased over time (26 per cent in 2009, 33·0 per cent in 2012, 54 per cent in 2015, and 72·7 per cent in 2018). Funding disclosure was present in 55·0, 65·0, 69·4 and 75·4 per cent of manuscripts respectively. Conflict of interest disclosure was present in 49·5, 89·1, 94·6 and 98·3 per cent of manuscripts across the same time periods. Data-sharing statements were present in only 15 RCTs (3·2 per cent), 11 of which were published in 2018.
CONCLUSION: Trial registration, disclosure of funding and disclosure of investigator conflicts in surgical RCTs have improved markedly over the past 10 years. Disclosure of data-sharing plans is exceptionally low. This may contribute to research waste and represents a target for improvement.

Introduction

RCTs are widely considered to provide the best evidence when evaluating the efficacy of surgical interventions. It is essential that data, and the method by which they are obtained, are reported transparently to facilitate open critical review.

The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network is an international initiative established to promote transparent and accurate reporting of research manuscripts. A number of reporting checklists now exist, including the CONSORT statement for RCTs and the PRISMA statement for systematic reviews , . Previous studies have examined compliance with these checklists, and many have identified a need for improved reporting of key items, such as the method of randomization and allocation concealment . The reporting of other key administrative information is also important for facilitating open critical review. One such example is study registration on a publicly available site with details about how this record can be accessed , . Another includes the disclosure of funding sources and declaration of investigator conflicts, which permit an appraisal of independence between commercial and academic partners . The availability or absence of original data to support the study results should also be reported. This allows investigators to reproduce, examine and compare data, while reducing research waste through unnecessary duplication , , .

The aim of this study was to investigate the transparency of surgical RCT manuscripts across four domains of academic publishing. These were: trial registration, disclosure of funding sources, declaration of investigator conflicts, and the provision of data‐sharing. A longitudinal assessment was planned to explore the trajectory of change and to identify priorities for improvement.

Methods

As a review of existing literature, approval by a research ethics committee was not required. This review was not eligible for registration on the PROSPERO database of systematic reviews as none of the outcomes was of direct relevance to patients or research participants. The results are reported according to the STROBE checklist .

Definitions

For the purpose of this study, an RCT was defined as a clinical study involving human participants randomized to at least two study groups. Surgery was defined as a procedure involving an external incision with manipulation of underlying tissues. Minimally invasive procedures were included within this definition, but studies involving radiologically guided interventions alone were not. RCTs exploring perioperative interventions (intraoperative or anaesthetic interventions) were included within the definition of a surgical trial, provided all other study eligibility criteria were satisfied. Trials were considered to have been registered prospectively if they were registered during the same month or before the start date of the study. All others were considered to have been registered retrospectively.

Data sources

Six high‐impact surgical journals were chosen (Annals of Surgery, British Journal of Surgery, Surgical Endoscopy, World Journal of Surgery, American Journal of Transplantation and Journal of the American College of Surgeons). These reflect widely read and highly cited journals within the field of surgery (impact factors (2018): range 2·768–9·476). In addition, four high‐impact general medicine journals that include surgical research were chosen (New England Journal of Medicine, The Lancet, Journal of the American Medical Association and British Medical Journal). The impact factors (2018) for these ranged from 27·604 to 70·670.

Eligibility criteria

RCTs involving adult participants were eligible. RCTs published in 2009, 2012, 2015 and 2018 were included. Studies involving children, preclinical/animal models and other non‐clinical/non‐interventional studies were excluded. Owing to the prespecified selection of journals, all manuscripts were published in the English language.

Study outcomes

The study outcomes were: the presence of trial registration details; the disclosure of study funding (or statement indicating no funding); the declaration of investigator conflicts (or statement indicating no conflicts); and the presence of a data‐sharing statement. Trial registration was assessed by examining manuscripts for a unique registry identifier. It was assumed that the absence of this identifier indicated an absence of registration. During the course of the study, this outcome was refined to consider the timing of registration (prospective versus retrospective). This was determined by comparing the dates of registration with the study start date. The presence of funding disclosures, personal conflict declarations and data‐sharing statements was assessed through inspection of the manuscript text. If a funding statement was not disclosed, evidence of the funding source was sought from the trial registry to maximize the completeness of data.

Data collection

All assessments were performed by one of three investigators. A 20 per cent sample was selected and validated by an independent assessor. All investigators discussed and agreed the assessment protocol before undertaking assessments, and all had a background of formal research training. Descriptive variables of interest were: publication year, journal type (surgical, general medical), recruitment (national, international), number of centres (single‐centre, multicentre), blinding (none, single‐investigator, single‐participant, double), surgical specialty and type of intervention.

Statistical analysis

Data are summarized as frequency with percentage for categorical items, and as median with interquartile range (i.q.r.) values for continuous items. Where relevant, the χ2 test was used to compare differences in categorical trial design variables. Subgroup analyses for each outcome were performed for key study characteristics, including funding, number of centres, blinding, study intervention and sample size. Statistical significance was set at the level of P < 0·050. All analyses were performed using SPSS® version 21.0 (IBM, Armonk, New York, USA).

Results

A total of 611 RCTs were considered for inclusion and 475 were determined to be eligible (Fig.  ). Most RCTs were studies involving gastrointestinal (347, 72·6 per cent), cardiothoracic (27, 5·7 per cent) and vascular (21, 4·4 per cent) surgery (Table  ). The most common types of intervention were operative/perioperative (275, 58·7 per cent) and drug therapies (146, 29·9 per cent). The median number of participants across all RCTs was 138 (i.q.r. 79–311).

Fig. 1

PRISMA diagram of study inclusion

Table 1

Characteristics of included studies

	No. of trials (n = 475)
Year
2009	109 (22·9)
2012	137 (28·8)
2015	111 (23·4)
2018	118 (24·8)
Journal type
Surgical	364 (76·6)
General medicine	111 (23·4)
Recruitment
National	377 (79·4)
International	96 (20·2)
Unknown	2 (0·4)
Centres
Single‐centre	264 (55·6)
Multicentre	211 (44·4)
Blinding
None	175 (36·8)
Single, investigator*	70 (14·7)
Single, participant	30 (6·3)
Double	140 (29·5)
Unknown	60 (12·6)
Subspecialty
Gastrointestinal	347 (73·1)
Cardiothoracic	27 (5·7)
Vascular	21 (4·4)
Breast	17 (3·6)
Urology	11 (2·3)
Orthopaedics	13 (2·7)
Gynaecology	13 (2·7)
ENT	11 (2·3)
Miscellaneous	9 (1·9)
Neurosurgery	3 (0·6)
Plastics	3 (0·6)
Intervention
Operative†	275 (57·9)
Drug	146 (30·7)
Medical device	35 (7·4)
Investigation‡	19 (4·0)

Values in parentheses are percentages.

Surgeon or assessor blinding.

Includes operative and perioperative interventions.

Includes diagnostic tools and imaging interventions.

Trial registration

Trial registration on a public database was disclosed in 397 of the 475 RCTs (83·6 per cent). The most commonly used database was ClinicalTrials.gov (276, 69·5 per cent), followed by the ISRCTN registry (38, 9·6 per cent) and UMIN clinical trials registry (18, 4·5 per cent). Registration of RCTs increased chronologically, with 73 of 109 (67·0 per cent) registered in 2009, 115 of 137 (83·9 per cent) in 2012, 99 of 111 (89·2 per cent) in 2015, and 110 of 118 (93·2 per cent) in 2018 (Fig.  ). Of the 397 registered RCTs, 190 (47·9 per cent) were registered prospectively. This also increased chronologically, with 19 (26 per cent) in 2009, 38 (33·0 per cent) in 2012, 53 (54 per cent) in 2015, and 80 (72·7 per cent) in 2018. There were no statistically significant differences in the presence of registration between types of funding source, study blinding or the study intervention. Multicentre RCTs were associated with higher compliance regarding prospective registration compared with single‐centre RCTs (49·8 versus 32·2 per cent respectively; P < 0·001). Likewise, RCTs with a sample size greater than 100 were associated with higher compliance than those with smaller sample sizes (47·5 versus 26·7 per cent respectively; P < 0·001) (Table  ).

Fig. 2

Changes in transparency of reporting between 2009 and 2018

Table 2

Trial registration

	Trial registered retrospectively (n = 207)	Trial registered prospectively (n = 190)	Trial not registered (n = 78)	Total (n = 475)	P ¶
Funding *					0·217
Non‐industry	150 (50·3)	122 (40·9)	26 (8·7)	298
Industry	44 (40·7)	54 (50·0)	10 (9·3)	108
Unknown†	13 (19)	14 (20)	42 (61)	69
Centres					< 0·001
Single‐centre	119 (45·1)	85 (32·2)	60 (22·7)	264
Multicentre	88 (41·7)	105 (49·8)	18 (8·5)	211
Blinding					0·127
None/open‐label	89 (50·9)	62 (35·4)	24 (13·7)	175
Single	46 (46·0)	40 (40·0)	14 (14·0)	100
Double	53 (37·9)	70 (50·0)	17 (12·1)	140
Unknown†	19 (32)	18 (30)	23 (38)	60
Intervention					0·392
Operative‡	130 (47·3)	99 (36·0)	46 (16·7)	275
Drug	54 (37·0)	70 (47·9)	22 (15·1)	146
Device	15 (43)	13 (37)	7 (20)	35
Investigation§	8 (42)	8 (42)	3 (16)	19
Sample size					< 0·001
≤ 100	76 (44·2)	46 (26·7)	50 (29·1)	172
> 100	131 (43·2)	144 (47·5)	28 (9·2)	303

Values in parentheses are percentages.

Where funding source was not disclosed, the trial registry was inspected for this information.

Unknown cases excluded from statistical tests.

Includes operative and perioperative interventions.

Includes diagnostic tools and imaging interventions.

χ2 test.

Disclosure of funding sources

Of 475 RCTs, 315 (66·3 per cent) disclosed the presence or absence of funding sources. Presence of a funding statement increased over time, with statements disclosed in 60 RCTs (55·0 per cent) in 2009, 89 (65·0 per cent) in 2012, 77 (69·4 per cent) in 2015, and 89 (75·4 per cent) in 2018. There was no statistically significant difference in funding disclosure between types of blinding. RCTs funded by industry had higher compliance with funding disclosure than those with non‐industry funding sources (96·3 versus 70·8 per cent respectively; P < 0·001) (Table  ). RCTs recruiting from multiple centres (82·5 versus 53·4 per cent; P < 0·001) and those recruiting more than 100 participants (76·9 versus 47·7 per cent; P < 0·001) were associated with higher compliance than RCTs recruiting from single centres and with smaller sample sizes. RCTs involving an operative/perioperative intervention (58·9 per cent) were less likely to provide a funding statement than those involving devices (62·9 per cent), investigations (63 per cent) or drugs (81·5 per cent) (P < 0·001) (Table  ).

Table 3

Funding statement

	Funding statement disclosed (n = 315)	Funding statement not disclosed (n = 160)	Total (n = 475)	P ¶
Funding *				< 0·001
Non‐industry	211 (70·8)	87 (29·2)	298
Industry	104 (96·3)	4 (3·7)	108
Unknown†	0 (0)	69 (100)	69
Centres				< 0·001
Single‐centre	141 (53·4)	123 (46·6)	264
Multicentre	174 (82·5)	37 (17·5)	211
Blinding				0·984
None/open‐label	119 (68·0)	56 (32·0)	175
Single	69 (69·0)	31 (31·0)	100
Double	96 (68·6)	44 (31·4)	140
Unknown†	31 (52)	29 (48)	60
Intervention				< 0·001
Operative‡	162 (58·9)	113 (41·1)	275
Drug	119 (81·5)	27 (18·5)	146
Device	22 (63)	13 (37)	35
Investigation§	12 (63)	7 (37)	19
Sample size				< 0·001
≤ 100	82 (47·7)	90 (52·3)	172
> 100	233 (76·9)	70 (23·1)	303

Values in parentheses are percentages.

Where funding source was not disclosed, the trial registry was inspected for this information.

Unknown cases excluded from statistical tests.

Includes operative and perioperative interventions.

Includes diagnostic tools and imaging interventions.

χ2 test.

Declarations of conflicts of interest

Disclosure of conflicts of interest was present in 397 of the 475 RCTs (83·6 per cent). This increased chronologically, with statements provided in 54 (49·5 per cent) of RCTs in 2009, 122 (89·1 per cent) in 2012, 105 (94·6 per cent) in 2015, and 116 (98·3 per cent) in 2018. There were no significant differences in the reporting of conflicts between funding sources, number of centres, or types of blinding. RCTs recruiting more than 100 participants were more likely to report a conflict of interest statement than those recruiting smaller samples (89·8 versus 72·7 per cent; P < 0·001). RCTs studying a device (66 per cent) were less likely to report a statement than those studying a drug (84·2 per cent), an operative/perioperative intervention (84·7 per cent) or an investigation (95 per cent) (P = 0·017) (Table  ).

Table 4

Conflict of interest

	Conflict of interest statement
Yes (n = 397)	No (n = 78)	Total (n = 475)	P ¶
Funding *				0·770
Non‐industry	256 (85·9)	42 (14·1)	298
Industry	94 (87·0)	14 (13·0)	108
Unknown†	47 (68)	22 (32)	69
Centres				0·057
Single‐centre	213 (80·7)	51 (19·3)	264
Multicentre	184 (87·2)	27 (12·8)	211
Blinding				0·320
None/open‐label	152 (86·9)	23 (13·1)	175
Single	80 (80·0)	20 (20·0)	100
Double	117 (83·6)	23 (16·4)	140
Unknown†	48 (80)	12 (20)	60
Intervention				0·017
Operative‡	233 (84·7)	42 (15·3)	275
Drug	123 (84·2)	23 (15·8)	146
Device	23 (66)	12 (34)	35
Investigation§	18 (95)	1 (5)	19
Sample size				< 0·001
≤ 100	125 (72·7)	47 (27·3)	172
> 100	272 (89·8)	31 (10·2)	303

Values in parentheses are percentages.

Where funding source was not disclosed, the trial registry was inspected for this information.

Unknown cases excluded from statistical tests.

Includes operative and perioperative interventions.

Includes diagnostic tools and imaging interventions.

χ2 test.

Disclosure of data‐sharing plans

Data‐sharing statements were present in only 15 (3·2 per cent) of 475 RCTs; 11 of these were in studies published most recently, in 2018. Of these 15 manuscripts, 12 stated that data sets were available on request, two stated that no additional data were available, and one study made an anonymized primary data set available as an appendix.

Discussion

This cross‐sectional review explored four key domains of transparency in surgical RCTs. Compliance with trial registration, disclosure of funding sources and declarations of personal conflicts were good, with considerable improvement over the past 10 years. Across most domains, compliance was better in RCTs recruiting across multiple centres with larger sample sizes. In contrast, disclosure of data‐sharing plans was exceptionally low, with only 3·2 per cent of RCTs stating whether this was possible and how data were to be made available. Despite efforts to improve data‐sharing, this still seems to be a key challenge in surgical publishing.

Reporting practices have been explored previously in a broad range of RCTs. An analysis of RCTs in ten high‐impact surgical journals between 2009 and 2010 showed that 34·9 per cent of published RCTs were not registered on a clinical trials database, and 21 per cent were registered after the completion of recruitment . This is problematic as it may encourage unexplained deviations from prospectively determined items for analysis and study design. Elsewhere, less than 15 per cent of trials published in psychiatry journals between 2009 and 2013 were registered prospectively, whereas this was considerably better (77 per cent) for medical specialty trials published between 2010 and 2015 , . The reporting of data‐sharing practices, funding disclosures and conflict of interest declarations have also been explored previously. In a review of studies published between 2000 and 2014, only one of 441 studies provided information on how to request original data . In the same study, a conflict of interest statement was provided in only 5·6 per cent of studies published in 2000, which improved to 65·4 per cent for trials published in 2014. In a review of surgical RCTs published between 2005 and 2010, funding disclosures were provided in 47 per cent and conflict of interest declarations in 25·1 per cent. Although the present study found greater levels of compliance across both of these domains, the difference may be explained by the later inclusion period (2009–2018) or the selection of articles from high‐impact journals. As shown previously , editorial policy for disclosure of statements is often variable.

The increasing level of transparency in surgical RCTs is encouraging as it suggests that the conduct of these studies is improving. Still, in 2018, registration was retrospective in 27·2 per cent of registered trials, and 6·8 per cent were not registered at all. Potential barriers to prospective trial registration may include: lack of awareness, error of omission, or the registration process taking longer than expected . Improvements in funding and personal conflict disclosures should also be considered with some caution, as these are self‐declared and difficult to validate. In a recent study of robotic surgical trials, financial ties with industry were undeclared in as many as 40 per cent of industry‐supported studies .

The results indicate the practice of data‐sharing as an area of persisting challenge. Low levels of data‐sharing may contribute to research waste through unnecessary duplication of results. For policy‐makers this increases the financial burden of research on healthcare systems, and for surgeons it reduces the financial resources available to answer other research questions. The ethical and legal positions of sharing patient‐level data can be contentious. Issues of consent, confidentiality and data security must be considered, and may present a barrier to studies that have already completed recruitment. For new studies, these challenges may be addressed through secure data repositories and early consideration in the design of protocols and consent materials , .

This study has limitations. The generalizability of the present findings should be interpreted with caution as the sample of RCTs was from a collection of high‐impact surgical and general medical journals. It is possible that these journals were more likely to endorse transparent reporting policies than those with lower impact factors. As the same journals were considered over a 10‐year period, it is possible that their impact and editorial policies may have changed. In assessing RCTs, it was assumed that trials were not registered if a unique identifier was absent from the report. This is a necessary limitation because the existence of numerous databases makes it difficult to locate studies confidently through manual searches. Some registrations may have been missed, but the absence of such identifiers can itself be considered a barrier to critical appraisal. Finally, the present study considered only author‐reported statements of data‐sharing. It is not possible to evaluate compliance with this offer or whether requests for data would be returned in a timely manner .

This study provides an overall optimistic outlook on transparent reporting in surgical RCTs, but highlights an area of need with respect to data‐sharing. This might reflect a lack of awareness around data‐sharing and the possible gains for research efficiency. Logistical and ethical challenges associated with sharing data, such as lack of time and resources to prepare data sets and the absence of a universal open‐access data platform, remain barriers. These issues should be addressed to determine how data‐sharing practices may be improved.

Coded data from this study are available upon reasonable request from the corresponding author.

Disclosure

The authors declare no conflict of interest.