Conflicts of interest in randomised controlled surgical trials: systematic review and qualitative and quantitative analysis.

Conflicts of interest may lead to biased trial designs and unbalanced interpretation of study results. We aimed to evaluate the reporting of potential conflicts of interest in full publications of surgical randomised controlled trials (RCTs). A systematic literature search was performed in CENTRAL, MEDLINE and EMBASE (1985-2014) to find all surgical RCTs of medical devices and perioperative pharmacological or nutritional interventions. The information on conflicts of interest was evaluated both quantitatively and qualitatively, and the development of stated conflicts over time was studied. Of 7934 articles, 444 met the inclusion criteria. In 93 of 444 trials (20.9%), conflicts of interest were disclosed. In half of the cases, the information provided was insufficient to permit conclusions regarding possible influence on the trials. Information about conflicts of interest has increased continuously during the last decades (1985-1994: 0%, 1995-2004: 2.8% and 2005-2014: 33.0%; p<0.001). Among the 115 industry-funded trials, industry participation was considered as a potential conflict of interest in 24 cases (20.9%). Over the past three decades, only every 10th trial has provided appropriate information on conflicts of interest. However, transparency is crucial for the reliability of evidence-based medicine. There is an urgent need for the full disclosure of all conflicts of interest in surgical publishing and for transparency regarding cooperation between academia and industry. ©2016 Markus K. Diener et al., published by De Gruyter.

Background

A conflict of interest exists whenever two or more interests are present that could influence each other. This creates the risk that decisions may not be made in accordance with a major interest [1], [2]. The concept of conflict of interest is not limited to medicine but also exists in various other academic disciplines [3]. The theoretical construct of a conflict of interest is made clear by a practical example. The primary interest of medical research is the identification of the optimal therapy for a given group of patients [4]. In addition, various other interests operate in medical research. One of the major interests of a manufacturer in the healthcare sector is to generate profit. Seen economically, this interest is ethical, and from the company’s perspective, it is a primary interest [5]. However, it becomes critical if focus on the manufacturer’s interest influences the conclusions of medical research inappropriately. In such a case, the welfare of the patients becomes secondary to the for-profit orientation of the company [6]. Clearly, these two interests have the potential to generate a conflict. As early as 1984, the New England Journal of Medicine became the first medical journal to call for disclosure of all potential conflicts by authors [7]. Today, clear guidelines regarding ethical publishing have been developed [8], [9]. Moreover, standardised forms for the disclosure of conflicts of interest exist [10].

Potential conflicts of interest have practical relevance because a relationship between industry research funding and positive study results has been shown for many medical disciplines. Moreover, the presence of such funding is an independent domain of bias and a potential threat to internal validity [11]. Particularly, in surgery, with the frequent use of medical devices, there are strong links to industry [12], [13], [14]. A recently published article has confirmed the relationship between sponsorship and study results for general and abdominal surgery [15].

The aim of this study was to quantitatively and qualitatively evaluate the information on conflicts of interest in surgical randomised controlled trials (RCTs) with medical devices and perioperative pharmacological or nutritional interventions over the past three decades.

Methods

The aim of this systematic review was to assess information on conflicts of interest in surgical RCTs of medical devices and perioperative pharmacological or nutritional interventions quantitatively and qualitatively. This systematic review is a secondary evaluation based on a published protocol [16] and was carried out according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [17]. The primary analysis [15] examined the association between industry sponsorship and positive study findings. This analysis, however, investigates the disclosure of conflicts of interest, with industry funding being just one of many such conflicts.

Systematic literature search

To find relevant trials of medical devices and perioperative pharmacological or nutritional interventions in the field of general and abdominal surgery with potential conflicts of interest, a preliminary search was performed. Hereby, the drugs and nutritional interventions had to be closely related to the surgical procedure. Pharmacological trials (e.g. with oncologicals in surgical patients) were therefore not eligible. The preliminary search identified 14 areas of indication for general and abdominal surgery in which drugs and medical devices with inherent industry interest were used. In each of the 14 indications, a systematic literature search was then performed to find all studies [multi-Participants, Interventions, Comparisons and Outcomes (PICO) search strategy].

In accordance with the recommendations of the Cochrane Collaboration [18], the literature screening was carried out by two independent persons. The literature search was carried out in the Cochrane Library, MEDLINE (via PubMed) and EMBASE. In addition, a hand search was performed in the references from relevant articles. The search was limited to the period from January 1985 to July 2014, because previously disclosure of conflicts of interest had hardly ever been required. No manuscript was excluded on grounds of the language used.

Inclusion and exclusion criteria

All RCTs included were from the field of general and abdominal surgery and examined the effectiveness of a drug, a perioperative nutritional intervention or a medical device. Other types of studies, trials comparing surgical strategies and trials from other medical disciplines were excluded.

Data extraction

For all included trials, the presence or absence of disclosure of conflict of interests was noted. All disclosure statements present were assessed qualitatively. Information about the source of funding and the year and journal of publication was extracted. It was established whether or not the publishing journal was International Committee of Medical Journal Editors (ICMJE) associated [10]. For purposes of quality assurance, data extraction was conducted by two persons independently [19].

Data synthesis

Quantitative analysis:

The proportion of trials disclosing conflicts was analysed over time. The absolute numbers and proportions of studies with conflicts of interest were recorded for the entire period and for the three periods of 1985–1994, 1995–2004 and 2005–2014. In a χ2 test at a level of significance of 5%, it was tested whether the proportion of trials disclosing a potential conflict of interest increased with the passage of time. Further, it was investigated how many industry-funded trials disclosed a potential conflict of interest. This information is given in absolute and relative terms. Moreover, it was examined whether or not trials published by ICMJE-associated journals more frequently provided information on potential conflicts of interest. The association of conflicts of interest with industry funding and ICMJE association was checked in a χ2 test. Furthermore, the proportion of reported conflicts of interest in trials with medical devices, drugs and nutrition were compared to each other in a three-sample χ2 test.

Qualitative analysis:

For qualitative analysis, it was judged whether or not the presented information helped the reader in the critical appraisal of the trial. Simple terms such as “none” or “no conflict of interest” without a clear statement how conflict of interest was defined were considered as unsupportive. Specifically, in the presence of industry funding, the statement had to explicitly define how this might have affected the trial. Statements were considered as helpful when they clearly defined involved secondary interests in study planning, conduct and analysis.

The statistical analysis was performed using R (version 3.1.2) [20].

Results

The systematic literature search found 7934 articles, of which 444 RCTs were included in the quantitative analysis (Figure 1). Most of the articles included were published in English (397 of 444 articles; 89.4%). Other languages were Chinese (19 articles; 4.2%), Italian (8 articles; 1.8%), German (7 articles; 1.6%), Spanish (5 articles; 1.1%), French (3 articles; 0.7%), Russian (2 articles; 0.5%), Portuguese (2 articles; 0.5%) and Turkish (1 article; 0.2%). Of the 444 RCTs included, 294 examined medical devices (66.2%) and 150 analysed perioperative medication or nutrition (33.8%).

Figure 1:

PRISMA flow chart.

Of 444 surgical RCTs, 93 (20.9%) featured conflict of interest disclosure (i.e. a statement whether or not there was a potential conflict of interest). The remaining 351 trials (79.1%) lacked any mention of the presence or otherwise of a conflict of interest.

In the years 1985 to 1994, none of the 33 included RCTs (0%) contained any information on conflicts of interest. Between 1995 and 2004, information about potential conflict of interest was disclosed in 4 of 141 studies (2.8%). Finally, between 2005 and 2014, disclosures of conflicts of interest were found in 89 of 270 studies (33.0%). Figure 2 shows this significant increase (p<0.001).

Figure 2:

Proportional increase of disclosure of conflict of interest in the past 30 years.

A total of 115 of the 444 RCTs (25.9%) were industry funded [15]. Among these, information about potential conflicts of interest was found in 24 trials (20.9%). There was no significant difference (p>0.99) in this respect from the remaining 329 trials, of which 69 (21.0%) provided information about a potential conflict of interest.

Overall, 237 of the 444 trials (53.4%) were published in ICMJE-associated journals. In 47 of these 237 trials (19.8%), disclosures of conflicts of interest were found. Of the 207 trials from journals that were not ICMJE associated, 46 (22.2%) reported conflicts of interest (p=0.50). Trials investigating nutrition published significantly less disclosure statements (9 of 120 trials; 7.5%) than trials investigating medical devices (77 of 294 trials; 26.2%) or drugs (7 of 30 trials; 23.3%; p < 0.01). Table 1 summarises the results.

Table 1:

Disclosure of potential conflicts of interest.

	All trials (n=444)
No statement about conflict of interest	351 (79.1%)
Conflict of interest disclosed	93 (20.9%)
Trials investigating medical devices	77 of 294 trials (26.2%)	p<0.01
Trials investigating drugs	7 of 30 trials (23.3%)
Trials investigating nutrition	9 of 120 trials (7.5%)
Statement adequate to judge about potential influence on trial validity	49 (11.0%)
	Industry-funded trials (n=115)
Industry involvement considered to be a potential conflict of interest	24 (20.9%)	p>0.99a
	Trials from ICMJE-associated journals (n=237)
Conflict of interest disclosed	47 (19.8%)	p=0.50a

aCompared to non-industry-funded trials or non-ICMJE-associated journals, respectively.

Qualitative analysis revealed that some of the disclosure statements did not help the reader to assess the possible conflicts of interest (e.g. “None” or “No conflict of interest”). In contrast, other statements enabled the assessment of potential influence on internal validity. Wordings such as “This study has not received funding of any form and the authors of this article have no commercial interests to disclose” [21] or “The authors declare that they have no competing interests. No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article” [22] in non-industry-funded trials ensure that the financial independence has been maintained.

In industry-supported trials, statements such as the following provided full transparency: “Seamguard was provided by Intermedical Corp. and Quixil by Ethicon gratis for the study with the understanding that the results of the study would be published regardless of outcome. All authors do not have any financial interest or relationship with the two companies” [23]. “Alvaro Sanabaria has a conflict of interest due to consulting activities with Johnson & Johnson. None of the surgeons received a fee to participate in the trial. Johnson & Johnson did not participate in any other step of the trial, including analysis and reporting” [24].

In the 93 of 444 surgical trials (20.9%) that supplied disclosure statements, 49 (49 of 93: 52.7%; 49 of 444: 11.0%) included information that helped the reader to assess potential conflicts of interest.

Discussion

Conflicts of interest are a natural by-product of the fact that people develop interests in different things [1]. In surgery, the principal area of conflict of interest is industry sponsorship of trials [15]; however, countless other secondary interests are conceivable. Therefore, for this systematic review, all information on potential conflicts of interest and not only industrial participation in surgical trials were analysed quantitatively and qualitatively.

The trials covered here examined a variety of interventions, but all had in common that there was an inherent industrial interest in the study result, as positive results could affect the sales of the products. In regard to the existence of a potential conflict of interest, our sample represents a homogeneous population of surgical trials. Despite this, in almost 80% of the trials, the authors did not take into consideration the possibility of conflict between the industrial interest and the research interest. The remaining trials provided disclosure statements, but half of these did not help the reader to draw conclusions about a potential influence of the industry involvement. The situation in surgery is similar to other medical fields. Also, in pharmacological trials, potential conflict of interest is under-reported and financial but also non-material ties lead to the overestimation of treatment effects [11], [15].

Given the significant costs associated with an RCT, it must be assumed that, in most cases, additional funding was available. Surprisingly, not only was information on potential conflicts of interest missing, but also in two thirds of the cases the source of funding was not even specified [15]. An industry involvement always generates a potential conflict of interest. However, the proportion of trials with industry funding disclosing potential conflict of interest was not higher. Besides industry funding, there are other financial or ideal bindings that may create a conflict of interest. Researchers receiving consulting fees and/or travel fees or are tied in non-material ways have also a potential conflict of interest. Furthermore, there are “key opinion leaders” that perform professional lobbying for different industrial partners, which should be transparently disclosed. Therefore, both information (i.e. the funding source and conflict of interest in general) are crucial to transparency. This was also the main reason to assess them in two separate studies [15].

In the subgroup analysis trials investigation nutritional support after surgery disclosed significantly less often a potential conflict of interest than trials investigating medical devices or drugs. One reason may be that regulations for nutritional supplements are even lower than they are for medical devices. Amazingly, not even ICMJE-associated journals showed a higher proportion of disclosure statements. These results agree with those known from the literature [25]. One reason might be that some journals fully applied the requirements of the ICMJE only later. This assumption is supported by the clear increase in the disclosure of conflicts of interest over the past three decades both at the level of individual trials and at the journal and publisher level [26]. The rate of more than 70% of trials with a disclosure statement on conflicts of interest in 2014 suggests that full transparency will soon become a reality.

It would be wrong to place all trials with industry participation under the general suspicion of aiming to achieve higher product sales. On the contrary, industry participation in surgical research is an important engine of innovation and, given the scarcity of public funds, indispensable. However, the top priority of surgical research remains to find the best treatment for patients before all economic interests. One model of cooperation between academia and industry are planning of trials by clinical investigators and financing by industry. In this way, the scientific integrity of the study is preserved. Moreover, the interests of industry go beyond product marketing; manufacturers also have an interest in the best possible treatment of patients. However, the question whether or not investigator-initiated trials are more robust to industry bias than industry-initiated trials cannot be answered yet because a sufficient amount of transparent data on trial funding is lacking. One weakness of the concept of conflict of interest is certainly the varied and largely inconsistent handling of the concept by publishers and journals, even those associated with the ICMJE [25]. In 2014, among 64 journals with scope on general and abdominal surgery, 22% of the journals did not define what constitutes a conflict of interest. Another 36% used standardised definitions, mostly the definition of the ICMJE, and 42% of the journals defined conflict of interest individually in their information for authors. Among the same journals, 12% did not ask authors to provide information on potential conflict of interest. In 27% of the journals, the information was necessary for acceptance of an article, and in 61% of the journals, the information was necessary and was printed with the article. Furthermore, among these, 64 journals, particularly journals from medical societies, had lower editorial demands with regard to the disclosure of conflict of interest [26].

This study is a plea for a pragmatic solution to the problem of industry participation in surgical trials by means of transparency. This starts with the disclosure of all sources of funding. In the presence of industry involvement, it must be described accurately and in detail which aspects of the trial (e.g. planning, conduct and analysis) were performed or financed by an industry partner. The aim is to express the role of the funding source openly and in full. Furthermore, it is necessary to register clinical trials before recruitment starts and to make trial protocols publicly available. Although the main focus should be the influence of the sponsor, it must be stressed that a diversity or variety of other secondary interests may exist. Every single author must therefore disclose his or her conflicts of interest ideally by means of a standardised form such as that of the ICMJE [10]. The individual disclosure should be available for readers.

The observed increase in the proportion of trials that disclose conflicts of interest corresponds to a trend towards the necessary transparency. This trend, however, can only be promoted by publishers, editors and reviewers if they incessantly demand transparency and impose sanctions in the case of non-compliance. Good examples of this development are high-impact journals as the New England Journal of Medicine and The Lancet, in which all ICMJE forms are publicly available, or the source of funding is included as a part of the abstract. Greater transparency is not detrimental to validity as demonstrated by the examples in the results section above. An open approach to potential conflicts of interest not only is transparent but can also strengthen confidence in the validity of the trial and science in general.

Ultimately, it is the responsibility of publishers, editors and reviewers to ensure that all the necessary information for the assessment of a study is available to the reader. This includes not only methodology and standard domains of bias but also all potential conflicts of interest.

Click here for additional data file.