Douglas A Jabs1, Andrew Dick2, John T Doucette3, Amod Gupta4, Susan Lightman5, Peter McCluskey6, Annabelle A Okada7, Alan G Palestine8, James T Rosenbaum9, Sophia M Saleem10, Jennifer Thorne11, Brett Trusko12. 1. Department of Ophthalmology, the Icahn School of Medicine at Mount Sinai, New York, New York; Department of Medicine, the Icahn School of Medicine at Mount Sinai, New York, New York; Department of Epidemiology, The Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland. Electronic address: douglas.jabs@mssm.edu. 2. Department of Ophthalmology, School of Clinical Sciences, University of Bristol - University College London Institute of Ophthalmology Medicine, Bristol, United Kingdom; National Institute for Health Research Biomedical Research Centre at Moorfields Eye Hospital, London, United Kingdom; University College London Institute of Ophthalmology, London, United Kingdom. 3. Department of Environmental Medicine and Public Health, the Icahn School of Medicine at Mount Sinai, New York, New York. 4. Department of Ophthalmology, Post Graduate Institute of Medical Education and Research, Chandigarh, India. 5. University College London Institute of Ophthalmology, London, United Kingdom; Moorfields Eye Hospital, London, United Kingdom. 6. Save Sight Institute, Discipline of Ophthalmology, Sydney Medical School, University of Sydney, Sydney, Australia. 7. Department of Ophthalmology, Kyorin University School of Medicine, Tokyo, Japan. 8. Department of Ophthalmology, University of Colorado School of Medicine, Aurora, Colorado. 9. Departments of Ophthalmology and Medicine, Oregon Health and Sciences University, Portland, Oregon; Legacy Devers Eye Institute, Portland, Oregon. 10. Department of Ophthalmology, the Icahn School of Medicine at Mount Sinai, New York, New York. 11. Department of Epidemiology, The Johns Hopkins University Bloomberg School of Public Health, Baltimore, Maryland; Department of Ophthalmology, The Johns Hopkins University School of Medicine, Baltimore, Maryland. 12. Department of Medicine, Texas A&M University College of Medicine, College Station, Texas.
Abstract
PURPOSE: To evaluate the interobserver agreement among uveitis experts on the diagnosis of the specific uveitic disease. DESIGN: Interobserver agreement analysis. METHODS: Five committees, each comprised of 9 individuals and working in parallel, reviewed cases from a preliminary database of 25 uveitic diseases, collected by disease, and voted independently online whether the case was the disease in question or not. The agreement statistic, κ, was calculated for the 36 pairwise comparisons for each disease, and a mean κ was calculated for each disease. After the independent online voting, committee consensus conference calls, using nominal group techniques, reviewed all cases not achieving supermajority agreement (>75%) on the diagnosis in the online voting to attempt to arrive at a supermajority agreement. RESULTS: A total of 5766 cases for the 25 diseases were evaluated. The overall mean κ for the entire project was 0.39, with disease-specific variation ranging from 0.23 to 0.79. After the formalized consensus conference calls to address cases that did not achieve supermajority agreement in the online voting, supermajority agreement overall was reached on approximately 99% of cases, with disease-specific variation ranging from 96% to 100%. CONCLUSIONS: Agreement among uveitis experts on diagnosis is moderate at best but can be improved by discussion among them. These data suggest the need for validated and widely used classification criteria in the field of uveitis.
PURPOSE: To evaluate the interobserver agreement among uveitis experts on the diagnosis of the specific uveitic disease. DESIGN: Interobserver agreement analysis. METHODS: Five committees, each comprised of 9 individuals and working in parallel, reviewed cases from a preliminary database of 25 uveitic diseases, collected by disease, and voted independently online whether the case was the disease in question or not. The agreement statistic, κ, was calculated for the 36 pairwise comparisons for each disease, and a mean κ was calculated for each disease. After the independent online voting, committee consensus conference calls, using nominal group techniques, reviewed all cases not achieving supermajority agreement (>75%) on the diagnosis in the online voting to attempt to arrive at a supermajority agreement. RESULTS: A total of 5766 cases for the 25 diseases were evaluated. The overall mean κ for the entire project was 0.39, with disease-specific variation ranging from 0.23 to 0.79. After the formalized consensus conference calls to address cases that did not achieve supermajority agreement in the online voting, supermajority agreement overall was reached on approximately 99% of cases, with disease-specific variation ranging from 96% to 100%. CONCLUSIONS: Agreement among uveitis experts on diagnosis is moderate at best but can be improved by discussion among them. These data suggest the need for validated and widely used classification criteria in the field of uveitis.
Authors: Alan G Palestine; Pauline T Merrill; Sophia M Saleem; Douglas A Jabs; Jennifer E Thorne Journal: JAMA Ophthalmol Date: 2018-10-01 Impact factor: 7.389