Effect of Funding Source on "Spin" in Studies of Ocriplasmin Therapy for Vitreomacular Traction and Macular Hole.

PURPOSE: To examine the relationship between industry funding and "spin" in randomized controlled trials (RCTs) and meta-analyses investigating use of ocriplasmin for patients with vitreomacular traction (VMT) and macular hole (MH).
METHODS: In this study, we examined all PubMed and Ovid MEDLINE RCTs and meta-analyses published in journals with impact factor ≥2 investigating effectiveness of ocriplasmin use for VMT and MH. The main outcome measure was correspondence between the studies' main statistical outcome and their abstract conclusion wording. Each article was reviewed by three independent observers and was evaluated for source of funding, industry co-authorship, study methodology, statistical significance of main outcome measure, correspondence between results of main outcome measure and abstract conclusion, and journal impact factor. Funding was determined by public disclosure. Discrepancies were resolved by consensus.
RESULTS: Twelve studies met inclusion criteria, of which 11 were industry funded and 1 was non-industry funded; 11 (91.67%) showed correspondence between outcome and abstract conclusion, without difference between industry-funded and non-industry funded publications or between publications in journals with high impact factor (≥3) versus low impact factor (≥2 and <3).
CONCLUSION: In RCTs and meta-analyses of ocriplasmin for VMT and MH, our results suggest that neither industry funding nor journal impact factor affected the rate of "spin" in study conclusions. This study helps physicians understand what challenges they face when learning about a newer, less-established drug.

Plain Language Summary

Pharmaceutical industry funding of medical research has been continuously growing over the last decades, which may increase the risk of bias and reporting of results in a pro-industry manner due to conflict of interest. Physicians do not always have time to review financial disclosures or possible bias when reading research articles. They often rely on the abstract portion of peer-reviewed articles, likely in scientific journals with high impact factor, to stay informed about new medications. The goal of this paper was to examine the relationship between industry funding and outcome reporting bias in RCTs and meta-analyses investigating use of ocriplasmin, a novel drug used for patients with vitreomacular traction and macular hole. In order to represent the articles that physicians are most likely to read, we evaluated all high qualities studies about ocriplasmin in journals with high impact factor, and assessed whether the articles’ main outcome results matched their abstract conclusion. Our results suggest that neither industry funding nor journal impact factor affected how study conclusions were “spinned”. This study helps physicians understand the challenges they face when learning about a newer, less-established drug.

Introduction

Over the last few decades, the funding of medical research by pharmaceutical companies has dramatically increased.1,2 Although industry sponsorship of medical research has been found to improve methodologic quality,2 the potential conflict of interest may cause authors to inappropriately display their findings in a positive light and thus cause discordances between study results and conclusions.3 The practice of misleading readers by reporting results in a distorted manner so that findings are viewed more favorably is known as “spin”.4

Because of the large time limitation in daily practice and the overwhelming amount of new information available, many physicians rely on the abstracts of published research and assume that the concluding statements parallel the results of the study.5

Studies investigating the effect of these industry ties have yielded mixed results: many reports have shown that studies funded by private industries were more likely to present results in a way that was in the sponsor’s interest,1 while others found no link between industry support and reporting bias.3 Regardless, the simple possibility of bias requires that physicians carefully review the literature and not blindly trust article conclusions.

When Alasbali et al pioneered the evaluation of bias in the field of ophthalmology and found that industry-funded studies were more likely to positively portray study results compared to their non-industry funded counterpart, they analyzed meta-analyses and randomized controlled trials (RCTs) along with other study methodologies.1 A recent study by our group went further by examining the connection between industry funding and reporting bias in anti-vascular endothelial growth factor (anti-VEGF) studies, but only included RCTs and meta-analyses published in journals with impact factors of 2 or more.6

Our group’s previous study evaluated anti-VEGF therapies, which unlike ocriplasmin, are drugs that have been extensively studied and found to be extremely efficacious in the treatment of macular edema secondary to retinal vein occlusions.6 For a drug as therapeutically successful as anti-VEGF agents, it may be less likely to find discordance between study results and abstract conclusions.6 To the authors’ knowledge, there are no studies investigating potential spin in newer, less established drugs that physicians may not be familiar with.

The safety and efficacy of ocriplasmin (Jetrea, ThromboGenics, Leuven, Belgium), a treatment recently indicated for patients with symptomatic vitreomacular traction (VMT) and vitreomacular adhesion (VMA) including when associated with macular hole (MH), have been evaluated in multiple trials, but the vast majority of these trials were industry funded,7 which could have introduced bias in the results of these studies. Additionally, the use of ocriplasmin still raises some controversies and is the subject of many ongoing and planned clinical trials.8

The goal of this study was to examine the relationship between industry funding and outcome reporting bias in RCTs and meta-analyses investigating use of ocriplasmin for patients with VMT and MH, and assess what challenges physicians face when reading literature about a newer, controversial drug.

Methods

The methods in this study were previously described by our group.6 In this study, we examined published randomized clinical trials or meta-analyses (N=12). Institutional Review Board approval and informed consent were not obtained as there were no human subjects and research involved only analysis of available literature.

Search

We completed PubMed and Ovid MEDLINE searches in April 2018 for studies that evaluated the efficacy of ocriplasmin for patients with VMT and MH (Figure 1). There were no cutoffs imposed on date of publication. Articles chosen were limited to English studies with an IF of 2.0 or more based on information provided by the journal website. In both PubMed and OVID, the search terms were: ocriplasmin AND (macular hole OR vitreomacular traction). The screening for study inclusion, removal of duplicates, and removal of studies because of study type or journal impact factor was performed by one of three independent observers (SH). If there was uncertainty in these steps, consensus was reached between all observers (SH, JS, AEK) on a case-by-case basis.

Figure 1

Selection of randomized clinical trials and meta-analyses investigating ocriplasmin use for patients with vitreomacular traction and macular holes.

Evaluation

SH, AEK, and JS each performed a full-text review and evaluation of included publications using a standardized data sheet detailing study methodology, statistical significance (p < 0.05) of the study’s main outcome measure, abstract conclusion, and correspondence between significance of main outcome measure result and abstract conclusion (Table 1). Study methodology was assessed with the same scoring scale used by Alasbali et al (Table 2).1

Table 1

Summary of Full-Text Study Assessments

Article	Impacta	Significant MOM	Qualityb	Correspondencec	Sample Size	Industry Funding	Industry Co-Author	Industry Sponsor	Comments
HIGHER-IMPACT JOURNALS (Impact Factor ≥ 3.0)
Lescrauwaet et al (2017)9	3.303	YES	1	YES	220	YES	YES	ThromboGenics N.V.	RCT
Birch et al (2018)10	3.7	NO	1	YES	62	YES	YES	ThromboGenics N.V.	RCT
Dugel et al (2016)11	8.2	YES	1	YES	220	YES	YES	ThromboGenics N.V.	RCT
Varma et al (2015)12	5.625	YES	1	YES	652	YES	YES	ThromboGenics N.V.	RCT
Jackson et al (2017)13	3.157	YES	1	YES	652	YES	YES	ThromboGenics N.V.	RCT
Dugel et al (2015)14	5.052	YES	1	YES	652	YES	NO	ThromboGenics N.V.	RCT
Gandorfer et al (2015)15	3.7	YES	1	YES	652	YES	YES	ThromboGenics N.V.	RCT
Haller et al (2015)16	8.2	NO	1	YES	652	YES	YES	ThromboGenics N.V.	RCT
Stalmans et al (2010)17	8.2	NO	1	YES	60	YES	NO	ThromboGenics N.V.	RCT
Benz et al (2010)18	8.2	NO	1	NO	125	YES	NO	ThromboGenics N.V.	RCT
Stalmans et al (2012)19	79.258	YES	1	YES	652	YES	YES	ThromboGenics N.V.	RCT
LOWER-IMPACT JOURNALS (Impact Factor ≥ 2.0 and < 3.0)
Chatziralli et al (2016)20	2.349	NO	1	YES	19	NO	NO		Meta Analysis

Notes: aImpact Factor of publishing journal based on journal website. bScore of 1 given to meta-analyses if search was comprehensive and unbiased; article validity assessed; and conclusions clear and supported. Score of 1 given to RCTs if treatment groups were randomized, double-blind, and correctly analyzed; follow-up was at least 80%; and sample size appropriately large. Score of 2 given to studies failing to meet one or more criteria. cCorrespondence between significance of MOM and wording of abstract conclusion.

Abbreviation: MOM, main outcome measure.

Table 2

Criteria Utilized for Grading of Study Methodology

Quality Score	Criteria
1: Meta-analysis (to assign this level,	Does the paper report a comprehensive search for evidence?
You must answer “yes” to all questions.)	Did the authors avoid bias in selecting articles for inclusion?
	Did the authors assess each article for validity?
	Does the paper report clear conclusions that are supported by the data and appropriate analysis?
1: Large RCT (to assign this level,	Were patients randomly allocated to treatment groups?
You must answer “yes” to all questions.)	Was follow-up at least 80% complete?
	Were both the patients and the investigators blind to the treatment the patient received?
	Were the patients analyzed in the treatment groups to which they were assigned?
	Was the sample size large enough to detect the outcome of interest?
2: RCT	RCT or overview that did not meet level 1

Abbreviation: RCT, randomized controlled trial.

Correspondence between significance of main outcome measure result and abstract conclusion was assessed by surveying: 1) whether the abstract conclusion addressed all aspects of the main outcome measure, 2) whether all results relevant to the main outcome measure were summarized in the conclusion, and 3) whether the wording of the abstract conclusion matched the statistical analysis of the results. The identified main outcome measure was based on the authors’ stated main outcome measure. If there was more than one main outcome measure, we evaluated all outcome measures for correspondence between study results and conclusion. Discrepancies in assessed correspondence were settled based on discussion and unanimous agreement of the three observers, while discrepancies in assessed study methodology were settled based on a simple majority vote. One observer (SH) also collected objective data on each publication including sample size, source of funding (industry versus nonindustry), and whether the publication included an industry co-author. Funding status was based on written disclosure within the article. The main outcome measure was correspondence between the studies’ main statistical outcome and their abstract conclusion wording.

Results

Included Publications

The original search yielded 258 publications between both databases, reduced to 153 after duplicates were removed. Of these, 75 publications were included for full-text review. At this point, sixty-three additional articles were excluded based on main outcome measure, study design, or lack of randomization, and 12 publications were ultimately included in our study (Figure 1). Of the 12 studies included, there were 11 RCTs and 1 meta-analysis.

Correspondence Between Main Outcome Measure and Abstract Conclusion

Statistically significant main outcome measures were present in 7 of 12 (58%) publications, and the wording of the abstract conclusion corresponded with these results in 11 of 12 (92%) publications. Non-correspondence in the Benz et al study was due to treatment superiority claims not supported by study design. Although the study’s abstract states that “patients receiving microplasmin were significantly more likely not to require vitrectomy surgery”, the secondary outcome measure of posterior vitreous detachment (PVD) progression at the highest tested dose of microplasmin was the only significant result.

Funding

All 12 publications included statements regarding industry-funding status. Eleven of 12 (92%) studies received industry funding – all sponsored by Thrombogenics – and 1 of 12 (8%) studies received no industry funding. There was an industry co-author in 8 (73%) of the 11 industry-funded studies. Study characteristics and reviewer assessments are summarized in Table 1.

Comparing Industry Funded versus Non-industry Funded Publications

Statistically significant main outcome measures were reported in 7 of 11 (64%) industry-funded publications and in 0 of 1 (0%) non-industry funded publications. Correspondence between abstract conclusion and significance of main outcome measure was present in 10 of 11 (91%) industry-funded publications and in 1 of 1 (100%) non-industry funded publications.

Comparing Higher-Impact versus Lower-Impact Publications

When publications were stratified by journal IF into a “high impact” group (N=11) with IF ≥ 3 and a “low impact” group (N=1) with IF ≥ 2 and <3, statistically significant main outcome measures were reported in 7 of 11 (64%) high-impact publications and in 0 of 1 (0%) low-impact publications. Correspondence between abstract conclusion and significance of main outcome measure was present in 10 of 11 (91%) high-impact publications and in 1 of 1 (100%) low-impact publication. Eleven of 11 (100%) high-impact publications received industry funding, while 0 of 1 (0%) low-impact publication did so.

Discussion

As biomedical industry funding of biomedical research increases,1 many studies examining the industry’s influence across a wide range of medical specialties and drugs have found that there is significant bias in favor of biomedical companies in publications that are industry-funded.21 More germane to ophthalmology, Alasbali et al found that in industry-funded studies assessing the efficacy of topical prostaglandins for reducing intraocular pressure, the wording of abstracts in industry-funded studies was more likely to not correspond with statistical results.1 On the other hand, other studies found no correlation between industry funding and industry-favoring results, or “spin”, in the biomedical literature.2,3,6 Although only one of the papers in this study showed non-correspondence between abstract conclusion and main outcome measure, there still is an impact on the literature as the paper was cited by 20 PubMed central articles in which authors could be mislead and misrepresent the paper’s results.

Our sample size was limited by the relatively small number of RCTs and meta-analyses dedicated to measuring the efficacy of ocriplasmin. In 2012, ocriplasmin was approved by the Food and Drug Administration for the treatment of vitreomacular adhesions, making it the first injectable drug to treat VMAs.8 Although promising, the use of ocriplasmin is controversial.8 We still lack evidence for use of ocriplasmin in multiple clinical situations, and the reports of its deleterious effects may have deterred some ophthalmologists from pursuing further clinical trials.8

It is crucial for physicians whose priority is to care for patients to inform themselves about advances in their field. When trying to obtain knowledge about a novel, less established drug, it is very difficult for physicians to navigate all potential biases in the medical literature, from impact of industry funding to minimal amount of research about a controversial drug. Due to the lag between FDA approval of a drug and non-industry led clinical trials, most trials available about novel drugs are industry-funded, which explains the low number of non-industry-funded articles in this study. Therefore, physicians learning about a drug in its early life on the market have mostly industry-funded research available to them. Therefore, readers must be aware of these potential biases, and must be cautious about journal quality, research quality, and funding source in order to get the most accurate knowledge of the safety and efficacy of a new treatment.

The aforementioned main limitation of this study is the low number of articles included: 11 articles were industry-funded and found in high IF journals, and only 1 article was not industry-funded and from a lower IF journal. This limited sample size is also partly due to our search criteria, which excluded non-randomized controlled trials, case series, and cohort studies. However, even randomized controlled trials are not without bias. For example, a review of neurology RCTs found 180 inconsistencies in reporting of outcome across 180 RCTs, all of which were biased toward statistically significant results.22 Still, we chose these inclusion criteria because high-quality RCTs are considered to provide the best level of evidence, and meta-analyses play an enormous role in evidence-based medicine by limiting most biases.23

Additionally, we excluded all articles found in journals with impact factors below 2, and thus potentially excluded studies of ocriplasmin use from lower IF journals. Physicians may be more familiar with high impact factor journals, and thus we wanted to focus on manuscripts most likely to be read by physicians. Moreover, Mimouni et al found that impact factor may also be affected by bias, as journals of higher impact factor are more likely to publish articles with statistically significant results rather than “negative” results.24 In the present study, 64% of the studies published in high impact factor journals had significant main outcomes; however, comparisons with lower impact factor articles are limited given our low sample size (n=1).

Conclusion

Although neither funding nor impact factor appeared to affect “spin” in RCTs and meta-analyses of ocriplasmin use, this study helps physicians understand the challenges they face when learning about a newer, less-established drug. Future studies involving new drugs would benefit from expanding search criteria to include articles from lower IF journals, carrying out searches in additional databases (ie, Embase), and with methodologies other than solely RCTs and meta-analyses. This strategy may help increase sample size, and allow researchers to better evaluate whether impact factor or study methodology plays a role in “spin”.