Reporting of conflicts of interest from drug trials in Cochrane reviews: cross sectional study.

OBJECTIVES: To investigate the degree to which Cochrane reviews of drug interventions published in 2010 reported conflicts of interest from included trials and, among reviews that reported this information, where it was located in the review documents.
DESIGN: Cross sectional study. DATA SOURCES: Cochrane Database of Systematic Reviews. SELECTION CRITERIA: Systematic reviews of drug interventions published in 2010 in the Cochrane Database of Systematic Reviews, with review content classified as up to date in 2008 or later and with results from one or more randomised controlled trials.
RESULTS: Of 151 included Cochrane reviews, 46 (30%, 95% confidence interval 24% to 38%) reported information on the funding sources of included trials, including 30 (20%, 14% to 27%) that reported information on trial funding for all included trials and 16 (11%, 7% to 17%) that reported for some, but not all, trials. Only 16 of the 151 Cochrane reviews (11%, 7% to 17%) provided any information on trial author-industry financial ties or trial author-industry employment. Information on trial funding and trial author-industry ties was reported in one to seven locations within each review, with no consistent reporting location observed.
CONCLUSIONS: Most Cochrane reviews of drug trials published in 2010 did not provide information on trial funding sources or trial author-industry financial ties or employment. When this information was reported, location of reporting was inconsistent across reviews.

Introduction

Concerns are ongoing about the influence on the medical evidence base of conflicts of interest stemming from links between researchers and drug manufacturers.1 2 A conflict of interest has been defined by the Institute of Medicine as “a set of circumstances that creates a risk that professional judgment or actions regarding a primary interest will be unduly influenced by a secondary interest.”3 Conflicts of interest due to research funding and other financial relations can influence how drug trials are designed and carried out,1 4 the likelihood that results will support a sponsor’s drug,5 6 7 8 9 whether or not results will be published,10 11 12 and how results will be interpreted in trial reports and articles about drug trials.10 12 13 14 Similarly, industry support of systematic reviews and meta-analyses, including financial ties of the authors of the reviews, has been associated with conclusions more likely to favour a sponsor’s drug compared with systematic reviews and meta-analyses not linked to industry.15 16 As a result, research reporting guidelines now routinely recommend that study funding and author-industry financial ties be disclosed in published research reports, including reports of systematic reviews and meta-analyses.17 18 19 20 Guidelines for systematic reviews and meta-analyses do not, however, require review authors to report conflicts of interest from trials included in the reviews.19 21 In this context, a recent study found that only two of 29 meta-analyses of drug trials published in high impact biomedical journals reported the funding sources of included drug trials, and none reported author-industry financial ties from included trials.22

Systematic reviews and meta-analyses produced by the Cochrane Collaboration are widely recognised as setting the standard for the evaluation of healthcare interventions.23 24 25 26 As evidenced by substantial ongoing funding to the Cochrane Collaboration by major governmental agencies,27 28 29 along with Cochrane’s partnerships with the World Health Organization30 and other key healthcare organisations,31 the Cochrane Collaboration plays an important part in the development of healthcare policies around the world. The Cochrane Collaboration also plays a key part in the training of health researchers27 and influences how both Cochrane and non-Cochrane systematic reviews and meta-analyses are conducted and reported. The Cochrane Handbook for Systematic Reviews of Interventions,21 32 which describes the Cochrane Collaboration’s methodology for review conduct and reporting, is a widely cited source of guidance on systematic review and meta-analysis methodology. The current version of the Cochrane Handbook (March 2011)21 and the previous version (September 2008),32 which guided reviewers until recently, indicate that review authors should extract data on the funding source of included trials and may consider extracting data on trial author-industry financial ties.33 34 Both versions make reporting information on trial funding in reviews optional and indicate that, if reported, it should be included in the table on characteristics of included studies.35 36 Whereas the previous edition (September 2008)32 suggested that potential bias related to study sponsorship could optionally be incorporated into the risk of bias assessment,37 the current version of the handbook (March 2011)21 states that information on conflicts of interest from included trials should not be included in the risk of bias assessment.38

We investigated the extent to which systematic reviews of drug treatments published in 2010 in the Cochrane Database of Systematic Reviews reported funding sources, author-industry financial ties, and author-industry employment from included trials. Among reviews that reported such information, we also determined where it was located in the published Cochrane review document.

Methods

Selection of systematic reviews

We searched the Cochrane Database of Systematic Reviews through the Cochrane Library on 28 March 2011, using the MeSH term “drug therapy,” to identify Cochrane reviews of drug interventions published in 2010. Cochrane reviews are published in the Cochrane Database of Systematic Reviews when new and when updates (that is, a new search) or amendments (that is, edited to make corrections or to reflect changes in methodology) are made to previously published reviews. Because reporting standards are evolving, we restricted the search to this one year period to obtain recent systematic reviews, with or without meta-analyses, that reflected relatively current reporting practices.19 20

Eligibility criteria

Cochrane reviews published in 2010 were eligible if they included a documented systematic review of the literature classified by Cochrane as up to date in 2008 or later,39 included results from at least one randomised controlled trial, and evaluated the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment (for example, placebo, alternative drug). We excluded reviews that only assessed different methods for administering a drug or dosage schedules of that drug. Drugs were defined broadly to include biologicals and vaccines but not nutritional supplements (for example, vitamins) or medical devices without a drug component. We included reviews that investigated a combination of drug and non-drug interventions (for example, psychotherapy), or interventions that may or may not involve a drug (for example, amnioinfusion), if a study group was exclusively given a drug intervention or if the review assessed the addition of a drug to a treatment received by both intervention and control groups. Interventions were classified as having a drug component if any form of the active ingredient (for example, dosage, route, strength, compound) was listed as an approved or discontinued brand name, generic drug, or therapeutic biological product by the US Food and Drug Administration.40 For agents not listed in the Drugs@FDA database,40 we determined drug status on the basis of consensus among investigators, using publically available sources that provided information on a particular agent.

Two investigators independently reviewed Cochrane reviews for eligibility. If either reviewer deemed a review to be potentially eligible based on review of the title and abstract, then we carried out a review of the full text. Two reviewers also independently carried out full text reviews, with any disagreements resolved by consensus. Cohen’s κ statistic was used to assess agreement between reviewers corrected by chance.

Data extraction

Two investigators independently extracted and entered into a standardised spreadsheet data items from the Cochrane reviews, with any discrepancies resolved by consensus. Investigators reviewed all text, tables, figures, appendices, disclosure statements, and acknowledgments from each Cochrane review to record disclosed conflicts of interest from each selected Cochrane review (review funding source and review author-industry financial ties). They also determined whether or not conflicts of interest from included trials (trial funding sources, trial author-industry financial ties, or trial author-industry employment) were reported in the reviews. Data items were extracted only from the included Cochrane reviews and not from any additional sources, such as online Cochrane resources (see supplementary appendix 1 for data extraction forms).

We extracted the funding sources for the Cochrane reviews from the sources of support declaration or acknowledgments and classified them as non-industry (for example, public granting agency, private not for profit granting agency), combined pharmaceutical industry and non-industry, or none reported (review not funded or review funding information not disclosed). Financial ties of review authors to industry were defined per the July 2010 version of the International Committee of Medical Journal Editors uniform disclosure form for potential conflicts of interest20 and included current or former board membership, current or former consultancy work, current or former industry employment, expert testimony, industry grants (issued or pending), payment for lectures including service on speakers bureaus, payment for manuscript preparation, patents (planned, pending, or issued), royalties, payment for development of educational presentations, stock or stock options, travel reimbursement, or other relations with industry, as disclosed in the review. If a review did not contain a disclosure statement, we coded review author-industry financial ties as not reported.

For each Cochrane review we also recorded whether the review reported information on the following types of conflicts of interest from included trials: trial funding sources, trial author-industry financial ties, and trial author-industry employment. For each of these types of conflicts of interest from included trials, we coded the reviews as reporting fully (reporting for all included trials), partially (reporting for some, but not all, included trials), or not reporting. We coded reviews as not reporting trial funding sources if they included data from pharmaceutical industry databases or noted that trial drugs were supplied by the manufacturers for certain trials but did not make any explicit statement of trial funding sources. For Cochrane reviews that reported information on conflicts of interest from included trials either fully or partially, we recorded where the information was reported. Specifically, we recorded whether the information was reported in the context of the risk of bias assessment (text, figure, or risk of bias table attached to the table showing the characteristics of the included studies) or outside of the context of the risk of bias assessment, including the text, the characteristics of included studies table, other table, the abstract, the plain language summary, in a footnote of a summary of findings table, or in the context of sensitivity analyses. In addition to coding whether conflicts of interest from included trials were reported in the Cochrane reviews, we coded whether the review’s reported data extraction protocol included extracting data on trial funding sources and trial author-industry financial ties or employment (yes, no, could not be determined).

A protocol was not published or registered for the present study. However, all methods were determined a priori with two exceptions. Firstly, during data extraction we added the classification of Cochrane reviews as funded by “combined industry and non-industry” sources. This was because, although the Cochrane Handbook states that commercial funding of reviews is prohibited,41 we encountered three Cochrane reviews where industry funding sources, along with non-industry funding sources, were listed. Secondly, our initial review protocol indicated that Cochrane reviews would be coded as either reporting or not reporting conflicts of interest information from included trials. We added the fully and partially reporting classifications because some reviews provided information on some, but not all, included trials. This occurred, for instance, when reviews mentioned a subset of included trials as funded by industry but did not provide information on the funding status of other trials (non-industry funding, no trial funding, not reported).

Results

The electronic database search yielded 272 unique titles and abstracts for review. Of these, 110 were excluded after review of the title and abstract because, although published in 2010, the review content was not classified as being up to date as of 2008 or later, no randomised controlled trials were included in the review, or the review did not assess the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment. Of the 162 Cochrane reviews that underwent review of the full text, 11 were excluded because they were not systematic reviews of the efficacy, effectiveness, or harm of a drug or class of drug against an alternative treatment, leaving 151 eligible systematic reviews (figure). Chance corrected agreement on inclusion and exclusion decisions between reviewers, as assessed with the Cohen’s κ statistic, was 0.95.

Flow chart of selection of Cochrane reviews of drug trials published in 2010 with searches up to date as of 2008 or later

The 151 selected Cochrane reviews evaluated a broad range of drug interventions, including 18 on treatment efficacy or effectiveness, two on harms, and 131 on both efficacy or effectiveness and harms. Between one and 121 trials were included in each systematic review. The content of 27 Cochrane reviews (18%) was classified as up to date in 2008, 59 (39%) in 2009, and 65 (43%) in 2010. The review status of 39 reviews (26%) was listed as “new,” 51 (34%) as “new search” with or without a change to review conclusions, 50 (33%) as “edited” (that is, any modification which does not involve a search for new studies) with or without a change to review conclusions, and 11 (7%) as “stable” (that is, no further changes expected to the review). The 151 selected Cochrane reviews included systematic reviews from 36 of the 53 Cochrane Review Groups that were registered in 2010. (See supplementary appendix 2 for the characteristics of the 151 selected Cochrane reviews.)

Review funding and review author-industry financial ties of Cochrane reviews

Of the 151 selected Cochrane reviews, 125 (83%) reported review funding from non-industry sources, three (2%) reported review funding from both pharmaceutical industry and non-industry sources, three (2%) stated that the review was not funded, and 20 (13%) did not include a sources of support declaration (see supplementary appendix 2). Of the three reviews that reported funding from both industry and non-industry sources, we were able to clarify that for two studies this referred to previous funding to the authors unrelated to the review itself (personal communication, Christopher Eccleston, coordinating editor, Cochrane Pain, Palliative, and Supportive Care Review Group, 10 April 2012); for the third study, this reflected a contribution from Merck for a previous version of a review that had been carried out from 1998 to 2000, before the 2004 Cochrane policy that prohibited industry funding of reviews (personal communication, Jackie Price and Gerry Stansby, coordinating editors, Cochrane Peripheral Vascular Diseases Review Group, 4 May 2012). In 42 of the 151 Cochrane reviews (28%, 95% confidence interval 21% to 35%), at least one review author reported one or more financial ties to the pharmaceutical industry (see supplementary appendix 3).

Reporting in Cochrane reviews of trial funding sources

Forty six of the 151 selected Cochrane reviews (30%, 95% confidence interval 24% to 38%) reported information on the funding source of least some of the included trials. Thirty reviews (20%, 14% to 27%) reported information on trial funding for all included trials and 16 (11%, 7% to 17%) reported for some, but not all, included trials (table 1; also see supplementary appendix 4). Four Cochrane reviews did not report or partially reported trial funding sources, but did state that the trial drug was provided by a pharmaceutical company for at least some trials (see supplementary appendix 5). One hundred and five Cochrane reviews did not report trial funding sources (70%, 62% to 76%), including (based on data extraction protocols) 11 reviews (7%) that recorded, but did not report, trial funding sources, 16 (11%) that provided a data extraction protocol that did not list trial funding source information, and 78 (52%) for which it could not be determined whether or not data on trial funding sources had been collected.

Table 1

 Reporting of trial funding sources, trial author financial ties to the pharmaceutical industry, and trial author employment by the pharmaceutical industry among 151 Cochrane reviews of drug trials published in 2010*

Type of conflict of interest	No of reviews reporting
	Fully (for all included trials)	Partially (for some included trials)	Fully or partially
Trial funding sources	30	16	46
Trial author-industry financial ties	2	9	11
Trial author-industry employment	0	10	10

*See supplementary appendix 5 for coding notes on reporting of included trial funding sources, trial author-industry financial ties, and trial author-industry employment.

Among the 46 Cochrane reviews that reported any trial funding sources, partially or fully, this information was reported in as few as one and as many as seven locations in each review. In all, 22 different reporting patterns (see supplementary appendix 6) were observed. In each of the 46 Cochrane reviews that reported trial funding sources partially or fully, this information was reported in at least one of four locations in the review: in the risk of bias section of the text, in the risk of bias table attached to the characteristics of included studies table, in other text, or in a part of the characteristics of included studies table other than the risk of bias table. Trial funding source was reported in the context of the risk of bias assessment in 28 of the 151 Cochrane reviews (19%), including eight (5%) that reported this information in the risk of bias text only, four (3%) that reported in the risk of bias table only, 14 (9%) that reported in both of these locations, and two (1%) that reported in both of these locations plus the risk of bias figure. Twenty four reviews (16%) reported the sources of trial funding in other text, and 24 (16%) reported the sources of trial funding in the characteristics of included studies table in the “methods” or “notes” fields. Information on trial funding source was reported in other locations less often, including another table (one review, 1%), a footnote in a summary of findings table (five reviews, 3%), the abstract (one review, 1%), the plain language summary (two reviews, 1%), and in the context of sensitivity analyses (six reviews, 4%) (table 2; see also supplementary appendix 4).

Table 2

 Summary of reporting patterns of the 46 of 151 Cochrane reviews of drug trials published in 2010 that reported trial funding sources, trial author financial ties to the pharmaceutical industry, and trial author employment by the pharmaceutical industry*

Type of conflict of interest	Locations in reviews where conflicts of interest information from included trials reported; No of reviews reporting fully or partially†
	No of reviews reporting in risk of bias assessment†			No of reviews reporting in other parts of review†
	Text	Figure	Table‡	Other text	Other table	Characteristics of included studies table	Footnote in summary of findings table	Abstract	Plain language summary	Considered in sensitivity analyses
Trial funding sources (n=46)§	24	2	20	24	1	24	5	1	2	6
Trial author-industry financial ties (n=11)¶	5	0	6	3	0	0	0	0	0	0
Trial author-industry employment (n=10)¶	4	1	8	1	0	1	0	0	0	0

*See supplementary appendix 5 for coding notes on reporting of included trial funding sources, trial author-industry financial ties, and trial author-industry employment.

†Total number of reviews reporting conflicts of interest information fully or partially in each location of 46 Cochrane reviews that reported trial funding sources, 11 reviews that reported trial author financial ties to pharmaceutical industry, and 10 reviews that reported trial author employment by the pharmaceutical industry. Each review could be counted only once per location for each type of conflict of interest information. Numbers in various locations add to more than total number of reviews reporting since some reviews report in more than one location.

‡Risk of bias table is attached to characteristics of included studies table.

§Trial funding sources were reported in between one and seven locations in each review that reported this information.

¶Trial author-industry financial ties or employment were reported in between one and three locations in each review that reported this information.

Partial or full information on trial funding sources was reported in 16 of 39 reviews with “new” status (41%, 95% confidence interval 27% to 57%) and 30 of 112 reviews with an updated or amended status (27%, 19% to 36%), including 17 of 50 reviews with “edited” status (34%, 22% to 48%), 11 of 51 reviews with “new search” status (22%, 12% to 35%), and two of 11 reviews with “stable” status (18%, 5% to 48%). Trial funding source was partially or fully reported in nine of 27 reviews classified as up to date in 2008 (33%, 19% to 52%), 18 of 59 classified as up to date in 2009 (31%, 20% to 43%), and 19 of 65 classified as up to date in 2010 (29%, 20% to 41%).

Reporting in Cochrane reviews of trial author-industry financial ties and trial author-industry employment

Sixteen (11%, 95% confidence interval 7% to 17%) of the 151 Cochrane reviews reported trial author-industry financial ties or employment by industry. Eleven reviews (7%, 4% to 11%) reported information on author-industry financial ties from included trials, including two (1%) that reported for all included trials and nine (6%) that reported for some, but not all, included trials. Ten reviews (7%, 4% to 12%) partially reported trial author employment by industry, and none reported this fully. Of the 10 reviews that reported trial author-industry employment for some included trials, five also partially reported other trial author-industry financial ties (table 1; see also supplementary appendix 4). All of the reviews that reported information on trial author-industry financial ties or trial author-industry employment reported information on trial funding sources. All studies that extracted trial author-industry financial ties or employment data reported it in the review. In addition, in 30 reviews (20%) the data extraction protocols indicated that this information was not extracted and 105 (70%) did not provide enough information to determine if this information had been extracted.

Trial author-industry financial ties or employment by industry were reported in between one and three locations in each review that reported this information. Information on trial author-industry financial ties or employment was reported in the context of the risk of bias assessment in 15 reviews (10%), including five (3%) that reported this information only in the risk of bias section of the text, seven (5%) only in the risk of bias table, two (1%) in both locations, and one (1%) in both locations plus the risk of bias figure. Trial author-industry financial ties or employment were reported in other review text in three reviews (3%) and in the characteristics of included studies table in one review (1%) (table 2; see also supplementary appendix 4).

Reporting in Cochrane reviews of trial funding, trial author-industry financial ties, and trial author-industry employment

Overall, considering either partial or full reporting, 30 Cochrane reviews (20%) reported only information on trial funding sources, six reported on trial funding sources and trial author-industry financial ties (4%), five on trial funding sources and trial author-industry employment (3%), and five on all three (3%). Considering only full reporting, 28 reviews (19%) reported only on trial funding sources and two reviews reported on trial funding sources and trial author-industry financial ties (1%).

Of the 42 Cochrane reviews that had at least one review author with disclosed financial ties to industry, 12 (29%) reported information on trial funding sources compared with 34 of 109 reviews (31%) in which no review authors disclosed financial ties to industry. Two of 42 Cochrane reviews (5%) with review author ties to industry reported trial author-industry financial ties or employment by industry from included trials, compared with 14 of 109 Cochrane reviews (13%) without review authors with disclosed financial ties to industry.

Discussion

Less than a third of 151 Cochrane reviews of drug trials published in 2010 reported the funding source of any included trials, with only a fifth providing funding information for all included trials. Information on the funding source of any included trials was reported at a somewhat higher rate among new reviews (41%) than updated or amended reviews (27%). About 1 in 10 Cochrane reviews reported on trial author-industry financial ties, including employment of the trial author by the pharmaceutical industry, for at least some included trials. When Cochrane reviews did report on conflicts of interest from included trials, the location where this information was reported was inconsistent across reviews. The 46 Cochrane reviews that provided partial or full information on trial funding sources did so in between one and seven locations within each review, with more than 20 different patterns observed for reporting this information. Only one Cochrane review reported information on conflicts of interest from included trials in the review abstract.

Comparison with other studies

Transparent disclosure of conflicts of interest is increasingly emphasised as an important component in the reporting of results from both clinical trials and systematic reviews, including meta-analyses.17 18 19 20 However, a recent study found that only 7% of meta-analyses of drug trials published in high impact biomedical journals included information on trial funding disclosed in original trials, and none reported on trial author-industry financial ties or employment disclosed in the original trial publications.22 The results of the present study show that, although Cochrane reviews reported trial funding sources and trial author-industry financial ties or employment at a higher rate than non-Cochrane reviews published in high impact journals, information on conflicts of interest from included trials was absent from most Cochrane reviews. This gap in the reporting of conflicts of interest from included trials in Cochrane reviews is important because systematic reviews and meta-analyses of drug efficacy or effectiveness and safety are relied on by clinicians and policy makers,42 43 and Cochrane reviews have been found to be a high quality source of evidence on which to base decisions about healthcare interventions.23 24 25 26 Furthermore, the Cochrane Collaboration is an international leader in setting standards for the conduct and reporting of systematic reviews, including meta-analyses, of healthcare evidence.

Policy implications

Authors of systematic reviews and meta-analyses are guided by the preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement, which does not address the reporting of trial funding and author-industry financial ties from included trials.19 44 Authors of Cochrane reviews are additionally expected to adhere to practice recommendations described in the Cochrane Handbook for Systematic Reviews of Interventions,21 32 which is widely used in practice by authors of both Cochrane and non-Cochrane reviews as a primary source of systematic review and meta-analysis methodology.21 32 Both the current version of the handbook (March 2011),21 which presently guides reviewers, and the previous version (September 2008),32 which served as the standard setter at the time the included Cochrane reviews were done, updated, or amended, state that data on trial funding should be collected in all reviews.33 34 However, neither requires that trial funding sources be reported. Both versions of the handbook suggest that the authors of reviews may optionally include trial funding source as an additional field in the characteristics of included studies table.35 36 The 2008 edition of the handbook suggested that potential bias related to the influence of trial sponsors could be considered in an optional “other sources of bias” domain of the risk of bias tool.37 In contrast, the 2011 edition specifies that this information should not be incorporated in the risk of bias assessment.38 Both versions of the handbook mention that review authors may consider extracting data on trial author-industry financial ties but do not specify if and where this information should be reported.33 34

The results of the present study suggest that, without a more explicit reporting policy, conflicts of interest from included trials will not be reported in most Cochrane reviews. Given the well documented influence of industry funding of drug trials on their conduct, interpretation, and reporting,1 4 5 6 7 8 9 10 12 the Cochrane handbook and the PRISMA statement should be updated to require authors of systematic reviews and meta-analyses to report the funding sources of all included trials or to report that trial funding sources were not disclosed. Some Cochrane reviews indicated that a subset of trials in the reviews were funded by industry, but did not report the funding status of other trials (non-industry funding, no trial funding, not reported). As noted previously,22 if the funding source of included trials is only partially reported, readers might assume that the funding sources of other trials were available but not recorded, leaving them unsure as to how to interpret potential bias related to the funding sources of those trials. Alternatively, readers might assume that review authors indeed recorded the funding sources of all included trials, but only reported those with industry funding. This assumption may not be correct, and the potential for conflicts of interest related bias may be different for trials that did not report their funding source compared with trials that reported non-industry funding or trials that were not funded.

Beyond study funding, consumers of research consider conflicts of interest from trial author-industry financial ties and employment as relevant to appraising the likelihood of bias in trials.45 46 47 Authors of the Cochrane handbook and the PRISMA statement should also consider recommending that review authors record and report information on trial author-industry financial ties or employment as disclosed in the original trials (for example, number of trial authors with disclosed industry financial ties or employment, or that there was no disclosure statement).

Cochrane reviews that do report information on conflicts of interest in trials do not consistently do so in the same location of the review document. Thus, along with an explicit recommendation for reporting conflicts of interest from all included trials, greater emphasis should be placed on ensuring that this information can be easily found without readers having to inspect the entirety of the (typically lengthy) review document. The Cochrane handbook suggests that authors of Cochrane reviews may optionally add up to three extra fields to the characteristics of included studies table, including one to report information on trial funding.35 36 When the source of trial funding was reported in the characteristics of included studies table, however, this information was always reported in either the “methods” or “notes” fields of the table, both of which are required fields. The inclusion of “study funding” and “author-industry financial ties or employment” fields as required fields of the characteristics of included studies table would encourage more consistent reporting.

Finally, we recommend that the Cochrane Collaboration reconsider its position that trial funding and trial author-industry financial ties not be included in the risk of bias assessment. The 2008 version of the Cochrane handbook listed “inappropriate influence of funders” (section 8.14.1.6) (for example, data owned by industry sponsor) as a potential source of bias that review authors could optionally incorporate in the “other sources of bias” domain of the Cochrane risk of bias tool.37 The 2011 version of the handbook, however, argues that “vested interests” should not be included in the risk of bias assessment, which “should be used to assess specific aspects of methodology that might be been influenced by vested interests and which may lead directly to a risk of bias” (section 8.15.1.5).38 As previously noted,22 empirical criteria are generally used to select items (for example, sequence generation, blinding) that are included in assessments of risk of bias,38 48 including evidence of a mechanism, direction, and likely magnitude of bias. Empirical data show that trial funding by pharmaceutical companies and trial author-industry financial ties are associated with a bias towards positive results even when controlling for other study characteristics6 8 49 50 and, thus, meet these criteria. One concern might be that including conflicts of interest from included trials in the risk of bias assessment could result in “double counting” of potential sources of bias. However, ratings in the risk of bias table are not summed to a single score, and inclusion of risk of bias from conflicts of interest could reflect mechanisms through which industry involvement can influence study outcomes6 that are not fully captured by the current domains of the risk of bias tool (random sequence generation, allocation concealment, blinding of participants and staff, blinding of outcome assessment, incomplete outcome data, selective reporting, and other sources of bias). Furthermore, even if all relevant mechanisms were to be assessed, the degree of their influence may not be fully captured when reviewers only have access to the relatively brief descriptions of trial methods that are provided in most published reports. Inclusion of conflicts of interest from included trials in the risk of bias assessment would encourage a transparent assessment of whether industry funded trials and independently conducted trials reach similar conclusions. It would also make it explicit when an entire area of research has been funded by industry and would benefit from outside scrutiny.

Coding trial funding sources can be complex, and it may not always be clear to what degree different funders played a part in a given study. There are other examples, however, where risk of bias is coded even though the degree of potential bias may not be easily assessed from information available in trial reports. For instance, the degree to which trial participants and outcome assessors are successfully blinded is not always easily determined by review authors, but is rated as accurately as possible on the basis of available information. A reasonably simple system would be to code trial funding as pharmaceutical industry, non-industry (for example, public granting agency, private not for profit granting agency), combined pharmaceutical industry and non-industry, non-industry with study drug supplied by pharmaceutical industry, no study funding, or not reported.22

Limitations of the study

Limitations should be considered in interpreting results from this study. Firstly, most (74%) of the included Cochrane reviews published in 2010 were either updates or amendments (that is, review status of “new search,” “edited,” or “stable”), for which it is not known which version of the Cochrane handbook review authors might have consulted. However, the 39 reviews with “new” status reported trial funding sources (partially or fully) at only a somewhat higher rate (41%) than updated or amended (for example, status of “new search,” “edited,” or “stable”) reviews (27%), and most of the “new” reviews did not report this information. Secondly, the small number of reviews from each of the Cochrane Review Groups, which support review authors who carry out reviews in a particular content area, did not allow us to assess whether there may be differences across groups in reporting of conflicts of interest from included trials. Thirdly, we did not review the original reports of drug trials included in the Cochrane reviews to determine how many of these included disclosures of trial funding source or trial author-industry financial ties. However, we previously found that 63% of randomised controlled trials included in meta-analyses published in high impact biomedical journals reported the trial funding source in the original published reports, and 26% of the randomised controlled trial reports included financial disclosures by the trial authors.22 Regardless of the actual rate of original disclosure, systematic reviews and meta-analyses should transparently report whether, in each original trial, conflicts of interest are present, absent, or not disclosed. Finally, we searched the Cochrane Database of Systematic Reviews through the Cochrane Library using the MeSH term “drug therapy,” to identify Cochrane reviews of drug trials. It is possible that our search strategy may have missed potentially eligible reviews. However, we have no reason to believe that this would have biased our findings on the proportion of Cochrane reviews reporting conflicts of interest information from included trials.

Conclusions

In summary, the Cochrane Collaboration is a recognised leader in the establishment of methodology for the conduct and reporting of evidence based reviews. This study, however, found that most Cochrane reviews of drug trials did not report information on trial funding sources or trial author-industry financial ties, including employment, from included trials. When this information was reported, patterns of reporting were inconsistent across Cochrane reviews. Cochrane and the PRISMA statement should require the reporting of conflicts of interest from included trials in systematic reviews and meta-analyses. Cochrane should ensure that this information is reported in the same way across reviews, including in the abstract, which would be consistent with the consolidated standards of reporting trials (CONSORT) recommendation that funding information be reported in the abstracts of journal articles.17 18 Cochrane should also give consideration to including conflicts of interest from trial funding and trial author-industry financial ties as part of the risk of bias tool and assessment.

Guidelines for systematic reviews and meta-analyses do not require authors to describe conflicts of interest from included trials

A study found that meta-analyses of drug trials published in high impact biomedical journals rarely reported the funding sources of included trials, and none noted trial author-industry financial ties

It is not known to what degree this information is reported in Cochrane reviews, which set the standard for the conduct and reporting of high quality evidence based reviews

Most Cochrane reviews of drug trials published in 2010 did not report information on trial funding sources or trial author-industry financial ties, including employment, from included trials

When information was reported, it was not consistently reported in the same location across Cochrane reviews

Cochrane and the PRISMA statement should require reviews to report conflicts of interest from included trials in a way that is consistent across reviews, and Cochrane should include this information as part of risk of bias assessment