Ian J Saldanha1, Kristina Lindsley1, Diana V Do2, Roy S Chuck3, Catherine Meyerle4, Leslie S Jones5, Anne L Coleman6, Henry D Jampel4, Kay Dickersin1, Gianni Virgili7. 1. Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland. 2. Byers Eye Institute, Stanford University School of Medicine, Palo Alto, California. 3. Department of Ophthalmology and Visual Sciences, Albert Einstein College of Medicine, Montefiore Medical Center, Bronx, New York. 4. Wilmer Eye Institute, Johns Hopkins University School of Medicine, Baltimore, Maryland. 5. Department of Ophthalmology, Howard University Hospital, Washington, DC. 6. Frank and Ray Stark Foundation, David Geffen School of Medicine at UCLA, Los Angeles, California. 7. Eye Clinic, Department of Translational Surgery and Medicine, University of Florence, Careggi Hospital, Florence, Italy.
Abstract
Importance: Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies. Objectives: To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews. Design, Setting, and Participants: This cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease. Main Outcomes and Measures: Proportion of review outcomes also reported in trials and vice versa. Results: This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews. Conclusions and Relevance: Although most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.
Importance: Suboptimal overlap in outcomes reported in clinical trials and systematic reviews compromises efforts to compare and summarize results across these studies. Objectives: To examine the most frequent outcomes used in trials and reviews of the 4 most prevalent eye diseases (age-related macular degeneration [AMD], cataract, diabetic retinopathy [DR], and glaucoma) and the overlap between outcomes in the reviews and the trials included in the reviews. Design, Setting, and Participants: This cross-sectional study examined all Cochrane reviews that addressed AMD, cataract, DR, and glaucoma; were published as of July 20, 2016; and included at least 1 trial and the trials included in the reviews. For each disease, a pair of clinical experts independently classified all outcomes and resolved discrepancies. Outcomes (outcome domains) were then compared separately for each disease. Main Outcomes and Measures: Proportion of review outcomes also reported in trials and vice versa. Results: This study included 56 reviews that comprised 414 trials. Although the median number of outcomes per trial and per review was the same (n = 5) for each disease, the trials included a greater number of outcomes overall than did the reviews, ranging from 2.9 times greater (89 vs 30 outcomes for glaucoma) to 4.9 times greater (107 vs 22 outcomes for AMD). Most review outcomes, ranging from 14 of 19 outcomes (73.7%) (for DR) to 27 of 29 outcomes (93.1%) (for cataract), were also reported in the trials. For trial outcomes, however, the proportion also named in reviews was low, ranging from 19 of 107 outcomes (17.8%) (for AMD) to 24 of 89 outcomes (27.0%) (for glaucoma). Only 1 outcome (visual acuity) was consistently reported in greater than half the trials and greater than half the reviews. Conclusions and Relevance: Although most review outcomes were reported in the trials, most trial outcomes were not reported in the reviews. The current analysis focused on outcome domains, which might underestimate the problem of inconsistent outcomes. Other important elements of an outcome (ie, specific measurement, specific metric, method of aggregation, and time points) might have differed even though the domains overlapped. Inconsistency in trial outcomes may impede research synthesis and indicates the need for disease-specific core outcome sets in ophthalmology.
Authors: Ian J Saldanha; Tianjing Li; Cui Yang; Jill Owczarzak; Paula R Williamson; Kay Dickersin Journal: J Clin Epidemiol Date: 2017-02-27 Impact factor: 6.437
Authors: Päivi H Miskala; Barbara S Hawkins; Carol M Mangione; Eric B Bass; Neil M Bressler; Li Ming Dong; Marta J Marsh; Lee D McCaffrey Journal: Arch Ophthalmol Date: 2003-04
Authors: Bronagh Blackwood; Mike Clarke; Danny F McAuley; Peter J McGuigan; John C Marshall; Louise Rose Journal: Am J Respir Crit Care Med Date: 2014-04-15 Impact factor: 21.405
Authors: Paula R Williamson; Douglas G Altman; Jane M Blazeby; Mike Clarke; Declan Devane; Elizabeth Gargon; Peter Tugwell Journal: Trials Date: 2012-08-06 Impact factor: 2.279
Authors: Ian J Saldanha; Jimmy T Le; Sharon D Solomon; Michael X Repka; Esen K Akpek; Tianjing Li Journal: Ophthalmology Date: 2019-01 Impact factor: 12.079
Authors: Kristina B Lindsley; Susan Hutfless; Barbara S Hawkins; Jill F Blim; Dan Roberts; Timothy W Olsen; Flora Lum; Kay Dickersin Journal: JAMA Ophthalmol Date: 2018-11-01 Impact factor: 7.389
Authors: Ian J Saldanha; Kristina B Lindsley; Sarah Money; Hannah J Kimmel; Bryant T Smith; Kay Dickersin Journal: BMC Med Res Methodol Date: 2020-02-11 Impact factor: 4.615
Authors: Paula R Williamson; Ricardo de Ávila Oliveira; Mike Clarke; Sarah L Gorst; Karen Hughes; Jamie J Kirkham; Tianjing Li; Ian J Saldanha; Jochen Schmitt Journal: BMJ Open Date: 2020-09-06 Impact factor: 2.692