Demonstration of an online tool to assist managed care formulary evidence-based decision making: meta-analysis of topical prostaglandin analog efficacy.

BACKGROUND: The purpose of this paper was to demonstrate the use of an online service for conducting a systematic review and meta-analysis of the efficacy of topical prostaglandin analogs in reducing intraocular pressure (IOP) in glaucoma and ocular hypertension.
METHODS: An online service provider (Doctor Evidence) reviewed and extracted data from the peer-reviewed literature through September 2009. Randomized controlled studies of at least three months' duration assessing at least two prostaglandin analogs in patients with primary open-angle glaucoma, ocular hypertension, or normal-tension glaucoma were included. The primary endpoint was mean IOP. Summary estimates were created using random-effects models. The Q Chi-square test was used to assess statistical heterogeneity.
RESULTS: Sixteen studies satisfied the inclusion criteria and were analyzed. On average, greater IOP-lowering was seen with bimatoprost relative to latanoprost (1 mmHg, P = 0.025) and travoprost (0.8 mmHg, P = 0.033) based on mean IOP after 12-26 weeks of treatment. No statistical difference was observed in IOP-lowering between latanoprost and travoprost (P = 0.841). Findings were similar to previously published meta-analyses of topical prostaglandin analogs.
CONCLUSION: Systematic reviews relying on meta-analytic techniques to create summary statistics are considered to be the "gold standard" for synthesizing evidence to support clinical decision-making. However, the process is time-consuming, labor-intensive, and outside the capability of most formulary managers. We have demonstrated the effectiveness of a commercial service that facilitates the process of conducting such reviews.

Introduction

Evidence-based formulary decisions require systematic review of the peer-reviewed medical literature and meta-analysis to pool data across clinical studies. This type of data review and analysis is time-consuming, labor-intensive, and requires specialized statistical expertise, putting such work outside the capability of most formulary managers. Furthermore, the dynamic nature of the medical literature means that today’s systematic review, which required months and considerable resources to complete, may be outdated within weeks of completion.

Use of a web-based provider of systematic review services and analysis software may ease the process of developing systematic reviews and meta-analyses, and thus facilitate the critical appraisal of current data for evidence-based formulary decisions. One such service provider, Doctor Evidence (Doctor Evidence LLC, Santa Monica, CA), builds evidence databases from clinical studies on a web-based platform. Tools available on the platform allow users to identify and pool individual studies by participant or study characteristics and to select the outcome measures to be analyzed. Details and results of individual studies, as well as results of meta-analysis of pooled data across studies, can be viewed. The data collection and analysis are transparent, and the collated data are presented in a way that is easily readable, as shown in Figure 1.

Figure 1

Forest plots of the differences in intraocular pressure between prostaglandin analogs in individual studies and in the meta-analyses of the pooled data.

Abbreviation: IOP, intraocular pressure.

In collaboration with the Doctor Evidence service provider, we performed a systematic review of the peer-reviewed literature to determine the relative efficacy of the prostaglandin analogs bimatoprost, latanoprost, and travoprost in lowering intraocular pressure (IOP) in primary open-angle glaucoma (POAG), normal-tension glaucoma, and ocular hypertension. We present the results here to demonstrate the utility and value of this web-based tool for conducting a meta-analysis, leading to efficient development of a systematic review.

Methods

Creation of an evidence database using the online platform

Relevant studies were identified by the service provider through an independent search of the Cochrane Library (CDSR, CENTRAL, DARE), Medline, and published systematic reviews. The search used the terms “glaucoma”, “primary”, “open-angle”, “POAG”, “intraocular pressure”, “ocular”, “eye”, “pressure”, “hypertension”, “hypertensive”, “IOP”, “normotensive”, “normal tension”, “low tension”, “bimatoprost”, “Lumigan”, “latanoprost”, “Xalatan”, “travoprost”, “Travatan”, “prostaglandin analogs”, and “visual field”, and was last updated in September 2009. A priori criteria for study inclusion in the database were:

Randomized controlled trial with head-to-head comparison of at least two of the following interventions: bimatoprost 0.03% (Lumigan®, Allergan Inc, Irvine, CA), latanoprost 0.005% (Xalatan®, Pfizer Inc, New York, NY), and travoprost 0.004% (Travatan®, Alcon Laboratories, Fort Worth, TX) used as monotherapy in patients with glaucoma or ocular hypertension

Mean IOP and/or mean IOP fluctuation reported

Minimum follow-up of 12 weeks

At least 50% of the study population comprised of patients diagnosed with POAG, ocular hypertension, or normal-tension glaucoma, with the rest of the study population diagnosed with any type of glaucoma

Study published in English.

Data collection followed a standard procedure used by the service provider. Briefly, two evidence-based medical specialists read each paper completely and extracted data, including the study design, patient characteristics, clinical outcomes, and statistical data. The extracted data were imported into a template that was subsequently evaluated by proprietary technologies and processes for discrepancies, such as mismatches of subgroup and total population data. A third evidence-based medical specialist then performed an additional quality check of the data and reconciled any discrepancies identified prior to final approval of each study imported into the comprehensive database.

Systematic data review and meta-analysis

Application layers on the database platform permitted review of the patient characteristics and outcomes of individual studies as well as results of meta-analyses of pooled data. The primary analyses of mean IOP and mean reduction in IOP from baseline were based on data from the earliest time point of IOP measurement (typically between 8 am and 10 am) on the latest study visit within the timeframe of 12 weeks to six months of follow-up. The reduction in diurnal IOP from baseline was also analyzed based on the latest study visit within the timeframe of 12 weeks to six months of follow-up, with diurnal IOP defined as the average of IOP measurements taken at two or more time points during the day. Other outcomes data, including responder rates and achievement of target pressure levels, were also included in the database in the event that researchers decided to analyze additional endpoints.

Meta-analyses used the inverse variance method of weighting data for pooling.1 Heterogeneity of results across studies was tested with the Cochrane Q Chi-square test.2 Random-effects models were used for analyses because of the heterogeneity of treatment effects across studies.3

Results

The literature search identified 260 studies that were potentially relevant for inclusion in the database. Of these studies, 242 were rejected because they did not meet the inclusion criteria (reasons included not being a clinical study [n = 23], not being a head-to-head randomized controlled trial [n = 32], a crossover rather than parallel-group study design [n = 7], less than half of the population diagnosed with POAG, ocular hypertension, or normal-tension glaucoma [n = 53], did not compare at least two of the following: bimatoprost 0.03%, latanoprost 0.005%, and travoprost 0.004% as monotherapy [n = 110], mean IOP or mean IOP fluctuation not reported [n = 7], inadequate follow-up period [n = 8], not in English [n = 1], and meeting abstract with insufficient information available [n = 1]). The remaining 18 studies were randomized controlled studies comparing the prostaglandin analogs. These studies satisfied the inclusion criteria and were assigned for data extraction. Two of these 18 studies were rejected from the evidence database, one because there were discrepancies in the manuscript and no author clarification was received, and the other because outcomes were stratified by race rather than by treatment. The remaining 16 studies4–19 were included in the evidence database and analyzed.

Details of these 16 studies, including the study design, study duration, study population, and drugs tested, are listed in Table 1. Data were available from more of these studies for the primary outcomes chosen for analysis, ie, mean IOP and mean change in diurnal IOP from baseline, than for other efficacy outcomes that might be of interest, such as responder rates or achievement of target pressure levels, which could also be analyzed using the database. Of the 16 studies, several were not included in either of the meta-analyses presented because they did not report data for the primary outcomes, or data were reported without standard deviations or other error measurements. The specific reasons for studies not being included in the analyses shown are listed in Table 2.

Table 1

Studies analyzed

Study	Study design	Duration of follow-up	Study population	Prostaglandin analog study arms (n = number of patients included in analysis)
Alagoz et al4	Single-site RCT	180 days	Adult patients with newly diagnosed glaucoma, visual field defects, and IOP ≥22 mmHg	Bimatoprost 0.03% QD (n = 36)Travoprost 0.004% QD (n = 46)
Arcieri et al5	Single-site RCT	6 months	Adult patients with POAG, pseudophakic glaucoma, or aphakic glaucoma who needed lower IOP	Bimatoprost 0.03% QD (n = 16)Latanoprost 0.005% QD (n = 15)Travoprost 0.004% QD (n = 17)
Cantor et al6	Multicenter RCT	6 months	Adult patients with POAG or OHT and IOP 21–34 mmHg after washout	Bimatoprost 0.03% QD (n = 76)Travoprost 0.004% QD (n = 81)
Cantor et al7	Single-site RCT	6 months	Adult patients with POAG or OHT and IOP 21–34 mmHg after washout	Bimatoprost 0.03% QD (n = 14)Travoprost 0.004% QD (n = 12)
Cardascia et al8	Single-site RCT	180 days	POAG patients ages 40–60 years with visual field loss and untreated IOP > 20 mmHg	Latanoprost 0.005% QD (n = 9)Travoprost 0.004% QD (n = 9)
Dirks et al9	Multicenter RCT	3 months	Adult patients with normal-tension glaucoma who needed IOP-lowering therapy	Bimatoprost 0.03% QD (n = 33)Latanoprost 0.005% QD (n = 27)
Gandolfi et al10	Multicenter RCT	3 months	Adult patients with glaucoma or OHT and IOP 22–34 mmHg after washout	Bimatoprost 0.03% QD (n = 109)Latanoprost 0.005% QD (n = 105)
aHalpern et al11	Multi-center RCT	48 weeks	Black patients with POAG or glaucoma and IOP 21–36 mmHg	Latanoprost 0.005% QD (n = 43)Travoprost 0.004% QD (n = 49)
Kammer et al12	Multicenter RCT	3 months	Adult patients with glaucoma or OHT who had inadequate IOP control on latanoprost monotherapy	Bimatoprost 0.03% QD (n = 128)Travoprost 0.004% QD (n = 132)
Koz et al13	Single-site RCT	6 months	Previously untreated adult patients with POAG or OHT and IOP 22–36 mmHg	Bimatoprost 0.03% QD (n = 20)Latanoprost 0.005% QD (n = 20)Travoprost 0.004% QD (n = 20)
Netland et al14	Multicenter RCT	1 year	Adults with open-angle glaucoma or OHT and IOP 24–36 mmHg	Latanoprost 0.005% QD (n = 196)Travoprost 0.004% QD (n = 200)
Noecker et al15	Multicenter RCT	6 months	Adults with glaucoma and/or OHT and IOP 22–34 mmHg after washout	Bimatoprost 0.03% QD (n = 133)Latanoprost 0.005% QD (n = 136)
Noecker et al16	Multicenter RCT	3 months	Black adult patients with POAG or OHT and IOP 22–34 mmHg after washout	Bimatoprost 0.03% QD (n = 49)Travoprost 0.004% QD (n = 45)
Noecker et al17	Multicenter RCT	3 months	Black adult patients with POAG or OHT and IOP 22–34 mmHg after washout	Bimatoprost 0.03% QD (n = 16)Travoprost 0.004% QD (n = 15)
Parrish et al18	Multicenter RCT	12 weeks	Adult patients with glaucoma or OHT and IOP ≥ 23 mmHg after washout	Bimatoprost 0.03% QD (n = 136)Latanoprost 0.005% QD (n = 136)Travoprost 0.004% QD (n = 138)
bVarma et al19	Multicenter RCT	12 weeks	Adult patients with glaucoma or OHT and IOP ≥ 23 mmHg after washout	Bimatoprost 0.03% (n = 137)Latanoprost 0.005% (n = 136)Travoprost 0.004% (n = 138)

Notes: Subgroup analysis of Netland 2001 study;

Supplementary analysis of study by Parrish et al.18

Abbreviations: RCT, randomized controlled trial; IOP, intraocular pressure; QD, once daily; POAG, primary open-angle glaucoma; OHT, ocular hypertension.

Table 2

Reasons for exclusion of studies from the analyses shown

Study	Reason
Cantor et al7	Missing error measurement needed for inclusion in meta-analysis of mean IOP Missing change in diurnal IOP data needed for inclusion in analysis of change in diurnal IOP from baseline
Halpern et al11	Missing mean IOP data from a study visit after 12–26 weeks of treatment that would be needed for inclusion in meta-analysis of mean IOP Missing change in diurnal IOP data needed for inclusion in analysis of change in diurnal IOP from baseline
Netland et al14	Missing error measurement needed for inclusion in meta-analysis of mean IOP Missing change in diurnal IOP data needed for inclusion in analysis of change in diurnal IOP from baseline
Noecker et al17	Missing error measurement needed for inclusion in meta-analysis of mean IOP Missing diurnal IOP measurements needed for inclusion in analysis of change in diurnal IOP from baseline
Noecker et al16	Missing error measurement needed for inclusion in meta-analysis of mean IOP Missing diurnal IOP measurements needed for inclusion in analysis of change in diurnal IOP from baseline
Varma et al19	This paper reports the same patients/data as Parrish 2003, so it could not be included as an independent data source for the same analyses

Differences in mean IOP between the prostaglandin analogs were evaluated by meta-analysis (Figure 1). The effects of bimatoprost and latanoprost on mean IOP in the morning after 12–26 weeks of treatment were compared in 826 patients in five studies.5,10,13,15,18 The results demonstrated substantial heterogeneity across studies (Q = 15.6, P = 0.004). Meta-analysis of the data showed that mean IOP was 1 mmHg lower with bimatoprost than with latanoprost (mean difference −0.96 mmHg, 95% confidence interval [CI] −1.8, −0.12, P = 0.025). Six studies that reported mean data with estimates of variance compared the effects of bimatoprost and travoprost on mean IOP in the morning after 12–26 weeks of treatment.4–6,12,13,18 Data were available from a total of 846 patients treated with bimatoprost or travoprost in these studies. Meta-analysis of the data showed that mean IOP was 0.8 mmHg lower with bimatoprost than with travoprost (mean difference −0.82 mmHg, 95% CI −1.58, −0.07, P = 0.033). The effects of latanoprost and travoprost on mean IOP in the morning after 12–26 weeks of treatment were compared in 364 patients in four studies.5,8,13,18 Meta-analysis of the data showed no statistically significant difference in mean IOP between latanoprost and travoprost (mean difference −0.06 mmHg, 95% CI −0.62, 0.50, P = 0.841).

The effects of bimatoprost and latanoprost on the change in diurnal IOP from baseline after 12–26 weeks of treatment were compared in two studies,9,18 but pooling of the data was not possible because no estimate of variance was reported in one of the studies.9 In the study reported by Parrish et al,18 the mean reduction in diurnal IOP from baseline at week 12 was 0.3 mmHg larger with bimatoprost than with latanoprost (n = 273, Figure 2), while in the study reported by Dirks et al,9 the mean reduction in diurnal IOP from baseline at week 13 was 1.1 mmHg larger with bimatoprost than with latanoprost (n = 60, P = 0.035, Figure 2). Differences in results between the studies may reflect differences in the patient populations and their treatment history. Patients in the study reported by Dirks et al9 were diagnosed with normal-tension glaucoma, while patients in the study reported by Parrish et al18 were diagnosed with ocular hypertension or glaucoma associated with elevated IOP. Half of the patients enrolled in the study reported by Parrish et al18 were being treated with latanoprost when they were screened for study entry. The treatment history of patients enrolled in the study by Dirks et al9 was not reported.

Figure 2

Mean change in diurnal intraocular pressure from baseline in individual studies comparing bimatoprost with latanoprost.

Abbreviation: IOP, intraocular pressure.

The effects of bimatoprost and travoprost on the change in diurnal IOP from baseline after 12–26 weeks of treatment were compared in 534 patients in two studies.12,18 Meta-analysis of the data showed that the mean reduction in diurnal IOP from baseline was 0.7 mmHg larger in patients treated with bimatoprost than in patients treated with travoprost (mean difference −0.66 mmHg, 95% CI −1.13, −0.19, P = 0.006, Figure 3).

Figure 3

Forest plot of the difference in the change in diurnal intraocular pressure from baseline between bimatoprost and travoprost in individual studies and in the meta-analysis of the pooled data.

Abbreviation: IOP, intraocular pressure.

Only the study reported by Parrish et al18 compared the effects of latanoprost and travoprost on the change in diurnal IOP from baseline after 12–26 weeks of treatment. In that study, the mean reduction in diurnal IOP from baseline at week 12 was 0.3 mmHg larger with latanoprost than with travoprost (n = 274, Figure 4).

Figure 4

Mean change in diurnal intraocular pressure (IOP) from baseline in a single study comparing latanoprost with travoprost.

Discussion

Clinical and formulary decision-making should be based on the highest quality of evidence available with respect to the effectiveness of treatment and product differentiation. Systematic reviews of drug efficacy are useful for qualitative evaluation of study results and differences in drug efficacy. Quantitative evaluation of differences in drug efficacy requires meta-analysis of the pooled data with associated statistics. There is strong evidence that lower IOP improves outcomes for patients with glaucoma and ocular hypertension.20–22 Conduct of a systematic review and meta-analysis using a web-based platform provided evidence that use of bimatoprost achieved approximately a 1 mmHg lower mean IOP compared with latanoprost or travoprost and that this difference was statistically significant. There was no statistically significant difference between latanoprost and travoprost in mean IOP. These findings are consistent with previously published meta-analyses of the efficacy of the prostaglandin analogs.23,24 Changes in diurnal IOP from baseline were reported in fewer studies, and meta-analysis was possible only for the comparison of bimatoprost and travoprost. Nonetheless, the results were consistent with greater IOP-lowering of bimatoprost compared with latanoprost or travoprost.

The meta-analysis previously reported by Aptel et al24 evaluated change from baseline IOP rather than mean IOP, and at least 90% of the patients in each included study had to be diagnosed with POAG or ocular hypertension. The analysis was similar to ours in that it included only studies comparing at least two prostaglandin analogs, with a total of eight studies involving 1608 patients included. Two of the studies in the meta-analysis by Aptel et al were excluded from our meta-analysis of mean IOP because their duration was only one month, and we required a study duration of at least 12 weeks because latanoprost may not reach full effect after one month of treatment.25 Another two studies included in their meta-analysis14,17 were included in our database, but not our meta-analysis, because no error measurements were reported (Aptel et al calculated estimates of standard deviations for their analysis). In addition to four shared studies, our meta-analysis of mean IOP included five additional recent studies for a total of nine studies, with data analyzed for 1518 patients. Despite these differences, comparable conclusions were drawn. Aptel et al reported larger reductions in IOP from baseline at 8 am in bimatoprost-treated eyes than with latanoprost or travoprost, and we similarly reported lower IOP with bimatoprost than with latanoprost or travoprost at the earliest time point in the day. In addition, both Aptel et al and our group found no significant difference in efficacy between latanoprost and travoprost.

In our systematic review of IOP-lowering with the prostaglandin analogs, the service provider entered all available data from the included studies into individual data templates. For each outcome measure evaluated, the results of each individual study could be visualized, and a summary statement of the results was provided. When meta-analysis was possible, the meta-analysis results could also be visualized, and a summary statement was provided.

A fixed-effect model of meta-analysis is based on a mathematical assumption that every study is evaluating a common treatment effect. The summary treatment effect estimate resulting from this method of meta-analysis is the “true” or “fixed” treatment effect, and the CI describes the uncertainty of the estimate. Often, this underlying assumption may not be correct, because variation in study results is greater than would be expected by chance, indicating important underlying differences in study populations or methods. When this is the case, an alternative approach to meta-analysis is to use a random-effects model. The random-effects model assumes that the treatment effects in the individual studies may be different from each other, and the most common random-effects model also assumes that these different effects are normally distributed. Therefore, the meta-analysis estimates the mean and standard deviation of the different treatment effects. In our study, Chi-square tests indicated heterogeneity of treatment effects across studies, so random-effects models were used for meta-analysis.

Doctor Evidence, the online provider used in this report, uses traditional (and standard) meta-analysis methods and statistics, but the web-based platform offers three key advantages over traditional systematic reviews and meta-analysis. First, the process of performing meta-analysis is standardized, and the software provides an appropriate type of analysis for the data. For example, the meta-analysis of IOP-lowering with the prostaglandin analogs used random-effects rather than fixed-effects meta-analysis models because of the heterogeneity of treatment effects across studies. Second, details of the studies and analysis used are transparent. The calculations used in the statistical models are provided. Third, the results can be rapidly updated with new evidence. The database can be updated and meta-analysis results made available within 48 hours of publication of a new study.

The service is not without cost and may exceed the budget of an individual medical researcher, but could be within a departmental budget. Users such as managed care formulary managers who need to repeat meta-analyses when new clinical data become available may realize cost savings over the long term because of the ability to update the database and meta-analysis rapidly. There is also a learning curve in becoming adept at navigating the database and accessing all the information and analyses contained on the platform. Nonetheless, formulary managers and others may find the online service to be a useful tool, because it facilitates the process of and shortens the timeline for performing systematic reviews and meta-analyses. The software provided can be used for meta-analysis of any endpoint and allows rapid modification of the analysis to facilitate critical appraisal of the evidence. For example, it is possible to select and deselect studies for inclusion in the analysis, which may be useful if, for example, there is a desire to limit the analysis to studies with study populations most similar to the patient population of interest.

In summary, transparent evidence-based decisions require the use of systematic reviews for evidence synthesis. We have demonstrated that collaboration with an online provider of systematic review services and software facilitates the process of developing up-to-date systematic reviews and meta-analysis of clinical evidence. Both the collection and the analysis of the data are transparent, and the analysis can be rapidly updated to include new evidence. The systematic review of prostaglandin analog efficacy in glaucoma and ocular hypertension performed in collaboration with the web-based service provider showed that bimatoprost has greater IOP-lowering efficacy than latanoprost or travoprost, consistent with results of previous meta-analyses of clinical trial data. These results demonstrate that the online service may be a valuable tool for generating systematic reviews and meta-analysis useful in formulary decision-making.