Financial disclosures of authors involved in spine research: an underestimated source of bias.

BACKGROUND: Reporting financial disclosures has become standard practice in both journal publications and during oral forum at scientific meetings. Despite this, the effect of reporting a financial disclosure of any member of an authorgroup, on the tone of the conclusion of an article has gained little attention. This study was performed to determine what effect reporting a financial disclosure has on the conclusion of an article.
METHODS: A literature search for all articles on interspinous devices and cervical disc prostheses, published from January 1st, 2008 until December 1st, 2010 was performed. Financial disclosures were reported, as were funding by commercially active parties. The tone of the conclusions was graded as positive, neutral or negative.
RESULTS: The odds ratio (OR) for a positive conclusion in cases where a financial disclosure was reported was 16.5 (95% CI: 4.7-58.1). Effect modification occurs with the presence of funding by a commercially active party. In cases where a financial disclosure was reported and funding was available for the study, the OR was 1.0 (95% CI: 0.08-12.6), whilst the OR was 33.3 (95%CI: 4.2-262.3) if funding was not provided. This discrepancy is mainly due to the large number of articles with a neutral/negative conclusion if the authors failed to report any financial disclosure and were not funded by a commercially active party.
CONCLUSIONS: Reporting a financial disclosure is a potential source of bias. Authors with disclosed financial relationships less often publish articles with a neutral/negative conclusion. This source of bias should certainly be taken into account during the critical appraisal of articles, particularly when the quality of the literature is being assessed.

Introduction

Evidence based medicine is generally accepted. Unbiased, high quality research is needed to answer questions regarding clinical effectiveness, cost -utility or cost -effectiveness of new treatments. Collaboration with the industry is essential to initiate time and financially consuming trials. Major sources of biases have been well established, and most of which have been quantitatively been investigated. Others have been recognized and discussed, yet research on them has rarely been performed. The former has been the case for the bias due to financial dependence of the investigator on a company that has developed or sold the product, under investigation [1-4]. We hypothesize that a financial disclosure was related to the conclusion of a study investigating the product for which the disclosure was made. Therefore, this study was performed to investigate whether financial disclosure is related to reporting a favorable result of the product under investigation.

Methods

Because of the background of the first author is mainly in spinal surgery, a literature search in Pubmed was performed on spinal implants. The implants chosen were interspinous distraction devices (ISD) and cervical disc prostheses. The search was restricted from January, 1st 2008 until December 1st, 2010. The search string for ISD was ((interspinous) AND (device OR implant OR spacer)) AND lumbar. The search string for cervical arthroplasty (CAP) included cervical AND (disc OR disk) AND (prosthesis OR prostheses OR arthroplasty).

For both searches, the language was restricted to English, German, French or Dutch. The titles and abstracts were studied. All articles were included except, letters to the editor, editorials and articles not pertaining to ISD or cervical arthroplasty. The remaining articles were completely studied to search for disclosures or funding by commercially active parties related to the Product or Technique under Investigation (PTI).

Since the conclusion is, for most readers, an invitation to read the whole article and many times also the main message readers remember, the abstract was the primary subject of study. The conclusions within the abstracts were examined for remarks on the PTI. The conclusion was then qualified as positive, neutral or negative. A conclusion was graded positive if a statement was included that the PTI exhibited a better or even slightly better performance. It was also deemed positive if the study did not show any superiority but an assumption was made that the use of the PTI would contribute to a better clinical or biomechanical outcome. These statements were characterized by adjectives including promising, encouraging, favorable, and the like. A conclusion was qualified as neutral, if the results of the study did not demonstrate any difference and without any assumptions being made about the possibility of a better outcome. A negative conclusion included a clear statement that the PTI did not carry any benefit above another product or technique, performed more poorly or emphasized mainly complications or adverse events of the POT.

The qualification of the conclusion was subsequently correlated with the presence of any financial disclosures of the authors. If a member of the author group had a disclosure in another article and the article of interest did not explicitly mention disclosures, the article was then included in a separate group as having a financial relationship, without explicit disclosure. For the purpose of analysis, this group was combined with the group that clearly stated disclosures. Moreover, whether the funding to support the work by a commercial party related directly or indirectly to the PTI was noted. The type of financial disclosure or funding was not separately investigated.

Two of the authors (RB, HD) independently investigated the retrieved articles. In cases of discrepancy between the two authors, they deliberated about the findings. If an agreement could not be reached, the opinion of a third investigator (JB) was obtained. The inter- observer agreement for qualifying the conclusion and the relation to funding or disclosures was determined as kappa (κ).

Statistical analysis was performed using a commercially available program (SPSS 17.0). Logistic regression was used to calculate odds ratios (OR). 95% confidence intervals (95%CI) were defined. Statistical significance was assumed when the p value was <0.05.

Results

The search on ISD resulted in the titles and abstracts of 66 articles. Fourteen articles were excluded because they did not relate to the subject of investigation, or did not present the results of a study. Editorials and letters to editors were likewise excluded, as were studies with inadequate abstracts. Therefore, 51 complete articles on ISD were included in this study. 215 articles were retrieved after a search was performed for CAP. 109 complete articles that dealt with the subject of interest were included. In total, 160 articles were studied. The results are depicted in Appendix. A summary of the results is shown in Table 1.

Table 1

Number of articles on interspinous devices and cervical arthroplasty from January 1st, 2008 till December 1st, 2010 related to financial disclosure, funding and the conclusion

	Financial disclosure reported		No financial disclosure reported		No need to report disclosures		Disclosures known but no need to mention them		Total
	Funding	No funding	Funding	No funding	Funding	No funding	Funding	No funding
Positive conclusion	17	24	10	24	–	20	1	8	104
Neutral/negative conclusion	2	1	1	32	–	20	–	–	56
Total	19	25	11	56	–	40	1	8	160
Total	44		67		40		9		160

Financial disclosure reported: clear statement of having a financial disclosure is made in the publication, No financial disclosure reported: a clear statement of not having a financial disclosure is made in the publication; No need to report disclosures: the journal did not ask to publish disclosures relating to the article and they are not clearly stated; Disclosures known but no need to mention them: form other publications it was known that a financial disclosure was present but the journal in which the article of interest was published did not publish any financial relationships

In 21 articles from the first set of articles, the preliminary qualification of the tone of these articles differed between the two investigators. After discussion and deliberation, the qualification was adapted without the need for asking the opinion of the third researcher. Kappa was 0.87.

In 40 articles, no financial disclosure was reported. Therefore, based on 120 articles, it could be estimated that the OR of reporting a positive conclusion while having a financial disclosure was 16.5 (95% CI: 4.7–58.1). If for the author groups of all 40 articles for which the disclosure was not stated explicitly, a financial disclosure was assumed (worst case scenario), the OR of reporting a positive conclusion while having a financial relationship was 3.0 (95% CI: 1.5–5.8), and if it was assumed that they indeed did have not have a financial relationship (best case scenario) the OR was 16.4 (95% CI: 4.8–55.7). The OR for a positive conclusion with a stated financial disclosure was 16.0 (95% CI: 1.8–142.4), and 16.7 (95% CI: 3.6–78.1), for studies on ISD and CAP, respectively.

The effect of funding of a study was also investigated. The OR for a positive conclusion with both a stated financial disclosure and funding by a commercially active party was 1.0 (95% CI: 0.08–12.6), whilst the OR was 33.3 (95%CI: 4.2–262.3) if funding was not provided. Therefore, the presence of funding contributed to effect modification. The addition of an interaction term was statistically significant (p = 0.035). The number of articles with a negative conclusion, where both funding was not present and a financial disclosure was not reported was quite remarkable. The OR for reporting an article with a neutral/negative conclusion in cases in which both financial disclosure and funding existed was 0.03 (95% CI: 0.0–0.2).

Discussion

The effect of financial relationships on the results of trials has been subject of many articles. It has been demonstrated that published funded trials report positive findings more frequently [5]. This effect has been established as “funding-driven bias” [6]. This bias has been classically restricted to articles reporting research related to pharmaceutical therapies.

Financial disclosures or conflict of interest of an author or members of an author groups have also been extensively discussed in the literature. In most instances, the articles dealt with legal or ethical aspects of financial relationships. The epidemiology of financial disclosures has likewise been described [7]. However, the effect of financial disclosures on study outcomes has gained little attention.

This study clearly established the quantitative relationship between financial disclosure and study outcome. The method employed for qualifying the conclusions has not been previously described. However, it is simple and affords high inter-observer agreement, with a kappa of 0.86 being achieved. Although the implants under investigation were only for use in spinal surgery, the conclusions here should not be restricted to this specialty. The ORs for studies on two different implants were essentially the same (i.e. OR 16.0 and 16.7).

To our knowledge, this is the first reported analysis of the effect modification having a financial disclosure has on the nature of study conclusions, in the presence of funding. The demonstrated difference is mainly due to the significantly higher number of studies with a neutral/negative conclusion, in the absence of funding and stated financial disclosure. While this has been described previously, the prior focus has been on the relationship between positive findings and financial disclosure [8]. Considering the nature of commercially active parties, reporting articles with negative conclusions regarding their products, is not in their best interest. Their primary objective lies in promoting their products. Therefore, it can be assumed that it is more likely that they preferentially and actively support any investigation which will result in a positive conclusion.

Publication bias could also contribute to the difference in articles with positive and neutral/negative conclusions. However, a clear distinction exists between the number of articles with a negative conclusion that are funded, and those that are not.

Therefore, the effect financial disclosure has on the outcome of an article certainly exists and should be taken into in account when interpreting the results and conclusions of a study. Financial disclosure bias, as well as, funding-driven bias should be considered when assessing the quality of publications, particularly when performing a meta-analysis. Furthermore, it should be added to the long list of possible biases.

Studying the effect of a financial disclosure in articles relating to interspinous devices, cervical disc prostheses, or any other specific implant, medical device, or drug, within a certain time period has a major advantage: the product of interest is clearly defined and the interest of industry within this time period is apparent. If this study would have included any article within a certain time period, certainly a dilution of the results would have occurred and more bias would have been incorporated. For example, studies on subjects that lack a commercial market, will be less frequently funded and the authors will less often report a financial disclosure. Therefore, it is assumed that the results of this study can be extrapolated to other research related to commercially available medical products. This is substantiated by the nearly equal OR for the two different spinal implants that are otherwise unrelated. Conversely, including all articles on several subjects could contribute to generalization of the findings of this study.

A flaw of this study is that the financial disclosure itself is not specified. Intuitively, stockholders may be more prone to report positive results compared with those authors who received minimal financial support. On the other hand, it is common knowledge that even small gifts will positively influence the opinion of a person towards the gift-related product [9]. Furthermore, it has been established that the description of financial disclosures widely differs and has even been called vague [10, 11]. In 40 articles, the disclosure was not stated. As was demonstrated in calculating the best and worst case scenario, knowledge of the presence of any financial disclosure would not affect the conclusion of this study. Only the magnitude of the results would have differed, although the OR of the forecasted best scenario was probably underestimated. Based on the results of this study, we believe that author groups with a financial disclosure report fewer articles with a neutral/negative conclusion. In our calculation, they were equally divided between the groups with and without known financial disclosure. If this would be corrected, the OR increased dramatically in a best case scenario where none of the authors had a financial disclosure.

Another criticism lies in the lack of qualification of the methodology used in the evaluation of articles. Whether the conclusion was correct based on the validity of the results of each article, has not been verified in this study; we examined only the summarized conclusion of the results and their interpretation as given in the abstract of each article. Therefore, the effect of a financial disclosure on the tone of the conclusion, alone was evaluated.

An ambivalent attitude towards financial relationships and research is currently warranted. Given that the development of new drugs or implants is, in most instances, dependent upon the financial support of the industry, funding of research should be encouraged. On the other hand and for purposes of independent critical appraisal of the results, any financial relationships between industry and researchers should be avoided and rejected. The attempts to influence scientific publications by industry should be minimized. An example is the nation-wide independent research programme on drugs launched by the Italian Medicines Agency (AIFA) in 2005. The challenges, feasibility and future perspectives are well described [12].

Conclusion

Financial disclosures are an underestimated source of bias. In fact, authors having a financial disclosure tend to less often report a study with a neutral or negative conclusion. This should be taken into careful consideration when assessing the quality of a study.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 26 kb)

Table 2

Tables with number of articles for interspinous devices and cervical arthroplasty separately from January 1st, 2008 till December 1st, 2010 related to financial disclosure, funding and the conclusion. References are included

	Financial disclosure reported		No financial disclosure reported		Disclosures not explicitly stated		Disclosures known but no need to mention them		Total
	Funding	No funding	Funding	No funding	Funding	No funding	Funding	No funding
Interspinous devices
Positive conclusion	7[1–7]	6[8–13]	3[14, 15]	8[16–24]	–	5[25–29]	1[30]	2[31, 32]	32
Neutral/negative conclusion	1[33]	–	1[34]	10[35–44]	–	7[45–51]			19
Total	8	6	4	18	0	12	1	2
Total	14		22		12		3		51
Cervical anthroplasty
Positive conclusion	10[52–61]	18[62–79]	7[80–86]	16[87–102]	–	15[103–117]	–	6[118–123]	71
Neutral/negative conclusion	1[124]	1[125]	–	22[126–147]	–	13[148–160]	–	–	36
Total	11	19	7	38	–	28	–	6	109
Total	30		45		28		6		109