Agreement of Optic Nerve Head Evaluation of Primary Open-Angle Glaucoma Between General Ophthalmologists and Glaucoma Specialists.

PURPOSE: To investigate the agreement of optic nerve head evaluations and initial diagnoses of primary open-angle glaucoma (POAG) by general ophthalmologists and glaucoma specialists in Shanghai, China.
METHODS: This multicenter, cross-sectional study involved the outpatients from the general ophthalmology departments of four top eye hospitals in Shanghai. The participants underwent ocular examinations, including intraocular pressure, fundus photography, corneal thickness, refractometry, visual acuity, visual field and gonioscopy. General ophthalmologists and glaucoma specialists performed the diagnoses and classified them as non-glaucoma, POAG suspects, and POAG. The consistency of initial diagnosis between general ophthalmologists and glaucoma specialists was measured using the weighted kappa coefficient. Logistic regression analysis was used to detect the risk factors for the reliability of POAG diagnosis.
RESULTS: In 922 participants, the initial diagnosis rates of POAG and non-glaucoma were much higher in the glaucoma specialist group than in the general ophthalmologist group, while the initial diagnosis rates of POAG suspects were higher in the general ophthalmologist group. The weighted kappa coefficient between the two groups was 0.831±0.027 (95% confidence interval, 0.779-0.884). Logistic regression analysis of the risk factors for the reliability of POAG diagnosis showed that the independent risk factors were intraocular pressure (OR 8.363, 95% CI: 4.27-16.37) and vertical cup-to-disc ratio (OR 3.459, 95% CI: 1.54-7.76).
CONCLUSION: The diagnosis consistency between the general ophthalmologists and the glaucoma specialists was similar among outpatients in the area of Shanghai. However, general ophthalmologists tended to classify the indefinite subjects as POAG suspects, and their accuracy in diagnosing POAG was low. By paying more attention to the risk factors of POAG diagnosis, general ophthalmologists could improve the diagnosis accuracy.

Introduction

Glaucoma is the leading cause of visual impairment worldwide. It is an irreversible disease, usually asymptomatic at an early phase, which causes vision loss and blindness by damaging the optic nerve. Given the large current population and expected future population growth, treatment of glaucoma (which is only partly effective and may alleviate symptoms and its progression) is an expensive disease that imposes a great burden on patients and society, placing enormous demands on the healthcare system.1–3

Primary open-angle glaucoma (POAG) is the most common type of glaucoma. The prevalence of POAG in China ranges from 0.7% to 2.1%.4–8 In addition, data on the initial diagnosis of POAG in outpatients are still very limited. POAG tends to progress slowly; patients are often asymptomatic until the disease reaches an advanced stage. For example, an Indian study reported that 98.5% of cases are diagnosed at an advanced stage.9 Similar data (93% of cases) were reported by a Japanese study.10 Thus, early detection and treatment are crucial for preventing visual impairment caused by glaucoma.11,12

POAG patients usually exhibit a relatively normal appearance with regard to the anterior ocular segments and do not develop an acute attack, which represents an additional challenge to ophthalmologists, especially for those who were not specialized in glaucoma. According to the Preferred Practice Pattern guidelines of the American Academy of Ophthalmology,13 early diagnosis of POAG requires detailed ophthalmic examinations, including applanation tonometry, gonioscopy, optic disc evaluation, and automated perimetry. However, different ophthalmologists might give different diagnoses at the early stage of POAG, even if provided with the same examination results. Therefore, the diagnostic reliability of general ophthalmologists and the factors influencing the diagnostic reliability need to be investigated.

In the present study, we examined participants from four top eye hospitals in Shanghai, China’s economic center. We investigated the agreement of optic nerve head evaluations and the initial diagnoses of POAG in outpatients by general ophthalmologists and glaucoma specialists and analyzed the risk factors for the reliability of POAG diagnosis.

Methods

Participants

This was a multicenter, cross-sectional, observational, hospital-based study. The outpatients from general ophthalmology departments of four eye hospitals, including A) Ruijin Hospital of Shanghai Jiaotong University; B) Shanghai Tenth People’s Hospital of Tongji University; C) Shanghai General Hospital of Shanghai Jiaotong University; D) Eye and ENT Hospital of Fudan University, were recruited between 2016 and 2018. Exclusion criteria were following: 1) previous diagnosis of POAG; 2) age < 30 years old; 3) severe opacity of refractive media; 4) narrow or closed-angle; 5) the previous history of intraocular surgery; 6) any of the factors (such as uveitis) causing secondary glaucoma; 7) anomalous discs, including tilted discs; 8) corneal diseases influencing the measurement of intraocular pressure (IOP). In this study, the general ophthalmologists were resident doctors from these four-eye hospitals, and the panel consisted of five glaucoma specialists from these four eye hospitals who were all the professors of ophthalmology. All 27 ophthalmologists received training in general ophthalmology lasting from 2 years to 6 years, while the glaucoma specialists received training on glaucoma for at least 5 years, except for the initial training of general ophthalmology.

The study was approved by the ethics committee of Eye and ENT Hospital of Fudan University and was conducted according to the tenets of the Declaration of Helsinki for the use of human subjects in biomedical research. Informed consent was obtained from all individual participants included in the study. The data were labeled with serial numbers and analyzed in a manner that protected patient privacy.

Examination Protocol

The participants underwent a comprehensive ophthalmic examination. Uncorrected visual acuity was measured (E charts) at a distance of 5 m. Automatic refractometry (Auto Refractometer AR-610; Nidek Co, Ltd., Tokyo, Japan) was performed if uncorrected visual acuity was < 1.0. Slit-lamp examinations were also conducted. Gonioscopy was performed with a Goldmann one-mirror lens (Haag Streit, Bern, Switzerland) at 25x magnification with low ambient illumination. Central corneal thickness was measured by A-scan (Nidek Co., Ltd.). Digital stereoscopic photographs of the optic nerve and macula were taken using a fundus camera (CR-DGi non-mydriatic retinal camera; Canon Inc., Kyoto, Japan). VCDR, cup notch, narrowed optic disc rim, optic disc margin hemorrhage, and retinal nerve fiber layer defect were used as indicators of structural glaucomatous change. The grading process used VCDR from 0.1 to 1.0 in 0.1 increments. IOP was measured using a Goldmann applanation tonometer (Haag Streit AG, Bern, Switzerland). Visual field (VF) was then tested with the Humphrey Field Analyzer (Carl Zeiss Meditec Inc., Dublin, CA) Central 24–2 Swedish Interactive Threshold Algorithm standard program. Tests were considered reliable and eligible for analysis if there were < 33% false-positives, 33% false-negatives, and 20% fixation losses. Demographic information, including age, gender, ethnicity, family history of glaucoma, history of diabetes, and history of hypertension were also collected.

Definition of Non-Glaucoma, POAG Suspect and POAG

The ophthalmologist reviewed all data available for each eye of the participant, including optic nerve images, results of the Humphrey Field Analyzer VF testing, and clinical records. The doctor then determined whether the participant was of non-glaucoma, POAG suspect, or POAG. Finally, a panel consisting of five glaucoma specialists openly discussed all the cases and gave the final diagnosis according to the Preferred Practice Pattern guidelines of the American Academy of Ophthalmology.13,14

Non-glaucoma individuals should meet all the criteria listed below: 1) healthy subjects with no history or presence of glaucoma; 2) IOP ≤21mmHg adjusted by central corneal thickness on several visits; 3) open angles by gonioscopy; 4) normal-appearing optic nerve head; 5) normal Humphrey 24–2 visual field test. POAG suspects should meet all the criteria listed below: 1) IOP ≤30mmHg; 2) normal visual field test; 3) asymmetric optic nerve head cupping (VCDR difference between both eyes ≥0.2 in the presence of a similar optic disc size) or increased cupping (VCDR >0.6). POAG patients should meet all the criteria listed below: 1) an abnormal visual field defined as the presence of at least two of the following: a) a glaucoma hemifield test outside normal limits, b) P < 5% for corrected pattern standard deviation, or c) a cluster of at least three contiguous points with P < 5%, including at least one of these with P < 1% in the pattern deviation plot; 2) one or more optic disc sign as follows: a) presence of a localized thinning or loss of the neuroretinal rim, b) optic disc excavation, c) VCDR >0.6, and d) VCDR asymmetry between both eyes ≥0.2. POAG suspects and POAG patients were then defined as glaucoma suspects and glaucoma with open-angle, excluding other possible secondary causes.

Statistical Analysis

All analyses were performed in SPSS software, version 25.0 (IBM, Chicago, IL, USA). Continuous variables were presented as mean ± standard deviation, while categorical variables were presented as frequency and percentage. The initial diagnosis rate of POAG suspects and POAG patients was calculated. For the comparison of variables among the non-glaucoma group, the POAG suspect group, and the POAG group, one-way analysis of variance was used to compare the means of continuous variables. The chi-square test was used to identify the difference in categorical variables. The weighted kappa coefficient was used to measure the consistency between the two kinds of diagnoses, and the results were shown as point estimates and 95% confidence intervals (CI). Univariate and multivariate logistic regression analyses were used to detect the risk factors for the reliability of POAG diagnosis. The dependent variable was the diagnosis agreement, which was consistent between the general ophthalmologists and glaucoma specialists. All p values were two-sided and were considered statistically significant when the values were < 0.05.

Results

Demographic and Clinical Characteristics of the Participants

A total of 971 participants were enrolled in this study; 49 were excluded because of anomalous discs or low-quality fundus images. The proportion of valid data was 94.95%. The numbers of participants from hospitals A, B, C, and D were 318 (34.49%), 88 (9.54%), 111 (12.04%), and 405 (43.93%), respectively. The mean number of participants for 27 ophthalmologists was 33.82±28.22 (ranging from 13 to 85). The sociodemographic and clinical data are shown in Table 1. Most of the participants were female (555, 60.20%), and the sample covered an age spectrum from 30 to 90 years, with a mean age of 51.03 years. A total of 342 (37.09%) participants had a history of hypertension, 149 (16.16%) had a history of diabetes, and 39 (4.23%) had a family history of glaucoma. In addition, 324 (35.14%) participants had myopia with refraction ≤-3.0 Diopter, 63 (6.83%) had IOP >21mmHg, and 337 (36.55%) had VCDR > 0.3.

Table 1

General Information of the Included Subjects

Variables	Total Subjects
Number, n (%)	922 (100)
Gender, n (%)
Male	367 (39.80)
Female	555 (60.20)
Age (Mean±SD, range)	51.03±16.71 (30–90)
Hypertension history, n (%)	342 (37.09)
Diabetes history, n (%)	149 (16.16)
Family history of glaucoma, n (%)	39 (4.23)
Refraction (Diopter, Mean±SD, range)
Right eye	−2.07±3.37 (−23.00 to 3.00)
Left eye	−2.11±3.44 (−25.00 to 3.00)
Refraction ≤-3.0D, n(%)	324 (35.14)
IOP (Mean±SD, range)
Right eye	16.17±4.39 (8.00 to 53.00)
Left eye	16.46±4.29 (7.00 to 44.00)
IOP >21mmHg, n (%)	126 (13.67)
VCDR (Mean±SD, range)
Right eye	0.37±0.15 (0.10 to 1.00)
Left eye	0.38±0.15 (0.10 to 1.00)
VCDR difference of both eyes (Mean±SD, range)	0.03±0.08 (0 to 0.6)
VCDR > 0.3, n (%)	337 (36.55)
Hospitals, n (%)
Hospital A	318 (34.49)
Hospital B	88 (9.54)
Hospital C	111 (12.04)
Hospital D	405 (43.93)

Abbreviations: SD, standard deviation; IOP, intraocular pressure; VCDR, vertical cup-to-disc ratio.

General Participant Information with Different Diagnoses

According to the diagnoses made by the panel of glaucoma specialists, 833 (90.35%) cases were non-glaucoma, 41 (4.45%) were POAG suspects, and 48 (5.21%) had POAG. The percentage of male participants in the non-glaucoma group was 38.06%, while it showed a significant increase in the POAG suspect group and the POAG group (53.66% and 60.42%, respectively). The male to female ratio was 0.61, 1.16, and 1.53 in the non-glaucoma group, the POAG suspect group, and the POAG group, respectively. The participants’ age in the three groups was comparable, ranging from 48.46 to 51.27 years old. There was a significant difference in hypertension history and diabetes history among the three groups, showing the obvious reduction in the POAG suspect group and the POAG group (p<0.001). The percentages of the family history of glaucoma showed a significant increase in the POAG group (p<0.001). The percentages of myopia with refraction ≤ −3.0 diopters in the three groups were comparable, ranging from 34.69% to 41.46%. The percentage of participants with IOP≥21mmHg significantly increased from 1.08%, 56.09% to 62.50%, and the difference of VCDR between both eyes increased from 0.02, 0.07 to 0.12 in the non-glaucoma group, the POAG suspect group, and the POAG group, respectively (p<0.001) (Table 2).

Table 2

Characteristics of the Participants Diagnosed as Non-Glaucoma, POAG Suspect, and POAG by the Panel of Glaucoma Specialists

Variables	Non-Glaucoma	POAG Suspect	POAG	P-value
Number	833	41	48	–
Gender, n(%)
Male	317 (38.06)	22 (53.66)	29 (60.42)	<0.001
Female	516 (61.94)	19 (46.34)	19 (39.58)	<0.001
Ratio of male to female	0.61	1.16	1.53	<0.001
Age (Mean±SD)	51.27±16.75	48.46±15.89	48.94±16.86	0.892
Hypertension history, n(%)	326 (39.14)	9 (21.95)	7 (14.58)	<0.001
Diabetes history, n(%)	137 (16.45)	6 (14.63)	6 (12.50)	<0.001
Family history of glaucoma, n(%)	34 (4.08)	2 (4.88)	3 (6.25)	<0.001
Refraction (Diopter, Mean±SD, range)
Righteye	−2.06±3.34 (−21.00 to3.00)	−1.93±2.42 (−7.00 to 3.00)	−2.44±4.45 (−23.00 to3.00)	0.317
Left eye	−2.08±3.40 (−21.00to3.00)	−2.00±2.71 (−9.50to 3.00)	−2.69±4.58 (−25.00 to3.00)	0.368
Refraction ≤-3.0D, n(%)	289 (34.69)	17 (41.46)	18 (37.50)	0.131
IOP (Mean±SD, range)
Right eye	15.57±3.46 (8.00 to32.00)	20.32±4.67 (11.00 to29.60)	23.14±8.66 (12.00 to53.00)	<0.001
Left eye	15.92±3.56 (8.00 to31.00)	20.54±4.93 (11.00 to33.00)	22.45±7.69 (12.00 to44.00)	<0.001
IOP >21mmHg, n(%)	70 (8.40)	23 (56.09)	30 (62.50)	<0.001

Abbreviations: POAG, primary open-angle glaucoma; SD, standard deviation; IOP, intraocular pressure; VCDR, vertical cup-to-disc ratio; NS, not significant.

The Initial Diagnosis Rates of POAG Suspects and POAG Patients in This Hospital-Based Study

Using the diagnosis of the panel of glaucoma specialists as the golden standard, of all the 922 participants involved in our study, the initial diagnosis rate of POAG suspects and POAG patients was 4.45% and 5.21%, respectively. Of the 48 POAG patients, there were 29 males and 19 females. According to the general ophthalmologists, the initial diagnosis rate of POAG suspects and POAG patients was 7.48% and 4.34%, respectively. The consistency rate between these two diagnoses was measured by the weighted kappa coefficient, and the value was 0.831±0.027 (95% CI, 0.779–0.884) (Table 3).

Table 3

Comparison of the Diagnostic Information of General Ophthalmologists and Glaucoma Specialists

Diagnosis	General Ophthalmologists	Glaucoma Specialists
Non-glaucoma	813 (88.18%)	833 (90.35%)
POAG suspect	69 (7.48%)	41 (4.45%)
POAG	40 (4.34%)	48 (5.21%)
Kappa coefficient	0.831±0.02 (95% Confidence Interval: 0.779–0.884)

Abbreviation: POAG, primary open-angle glaucoma.

Information Regarding Participants with Different Diagnoses According to General Ophthalmologists and Glaucoma Specialists

In total, 40 (4.34%) participants received a different diagnosis. According to the specialist panel, among those who were diagnosed as non-glaucoma individuals, 23 were misdiagnosed as POAG suspects and 2 with POAG patients by the general ophthalmologists. That means that many non-glaucoma individuals might be overly diagnosed as POAG suspects by general ophthalmologists. Of those who were diagnosed as POAG patients according to the specialist panel, 9 were misdiagnosed as POAG suspects and 2 as non-glaucoma individuals, thus suggesting that the accuracy of POAG diagnosis by general ophthalmologists need to be improved (Table 4).

Table 4

Information of Those Subjects Who Got Different Diagnosis Between General Ophthalmologists and Glaucoma Specialists

Variables	General Ophthalmologists → Glaucoma Specialists
Diagnosis	Suspect→Non-Glaucoma	POAG→Non-Glaucoma	Suspect→POAG	Non-Glaucoma→POAG	Non-Glaucoma→Suspect	POAG→Suspect
Number	23	2	9	2	3	1
Refraction, Right eye	−1.86	−1.50	−2.55	0.75	−4.50	0.00
Refraction, Left eye	−2.00	−1.13	−2.30	0.50	−4.67	0.00
IOP, Right eye	20.3	19.65	21.1	20.9	16.3	22
IOP, Left eye	20.9	20.15	19.8	22.6	16.9	16
VCDR, Right eye	0.46	0.45	0.64	0.55	0.43	0.5
VCDR, Left eye	0.53	0.45	0.55	0.4	0.43	0.4

Abbreviations: POAG, primary open-angle glaucoma; IOP, intraocular pressure; VCDR, vertical cup-to-disc ratio.

Independent Risk Factors for the Reliability of POAG Diagnosis

Univariate and multivariate logistic regression analysis identified larger VCDR (p=0.003) and higher IOP (p<0.001) as the significant independent risk factors for the reliable diagnosis of POAG, while the history of hypertension, history of diabetes, family history of glaucoma and myopia with refraction ≤ −3.0 diopter showed no significant relationship (p>0.05). The odds ratio was 8.363 (95% CI: 4.27–16.37) for IOP and 3.459 (95% CI: 1.54–7.76) for VCDR (Table 5).

Table 5

Multivariate Logistic Regression Analysis of Risk Factors for POAG Accurate Diagnosis

Variable	Univariable Analysis		Multivariable Analysis
	OR (95% CI)	P	OR (95% CI)	P
Gender	0.894 (0.471–1.698)	0.733
Age	0.646 (0.338–1.233)	0.185
Hypertension history	0.410 (0.187–0.901)	0.026
Diabetes history	0.565 (0.198–1.611)	0.285
Family history of glaucoma	0.570 (0.076–4.256)	0.583
Myopia	1.232 (0.653–2.325)	0.518
IOP	10.580 (5.465–20.481)	<0.0001	9.248 (4.663–18.340)	<0.0001
VCDR	4.842 (2.206–10.627)	<0.0001	3.261 (1.445–7.361)	0.004

Abbreviations: POAG, primary open-angle glaucoma; IOP, intraocular pressure; VCDR, vertical cup-to-disc ratio; OR, odds ratio; CI, confidence interval.

Discussion

This multicenter study showed that the diagnosis consistency between general ophthalmologists and glaucoma specialists was similar. However, general ophthalmologists tended to diagnose non-glaucoma or POAG as POAG suspects, and their accuracy in diagnosing POAG was low. In addition, we found that larger VCDR and higher IOP were the independent risk factors for the reliable diagnosis of POAG. By paying more attention to the risk factors of POAG diagnosis, general ophthalmologists could improve the accuracy of glaucoma diagnosis.

In the present study, we found that the initial diagnosis rate of POAG was 5.21%, which is higher than the prevalence of POAG described in other population-based studies.15–21 The major reason was that participants were recruited among outpatients, which would make the rate higher with a smaller screening population. However, this proved to be a cost-effective way to screen POAG. On the other hand, the diagnostic criteria used were similar to those of the Korean Study.22,23 We did not follow the criteria of the International Society of Geographic and Epidemiologic Ophthalmology (ISGEO),24 as the former was more easily adapted for outpatient studies and more closely adheres to clinical practice. In summary, this study provided data on the initial diagnosis rate of POAG in outpatients in the area of Shanghai, which might provide some insight into clinical practice.

The diagnoses made by the general ophthalmologists were in accordance with those made by the panel of glaucoma specialists with the Kappa coefficient of 0.831±0.027 [95% CI: 0.779–0.884]. This suggested that the general ophthalmologists in the study were qualified to diagnose POAG, but there were still some shortages in the diagnostic accuracy. Importantly, the initial diagnosis of POAG suspects by general ophthalmologists was rather higher than that by glaucoma specialists for approximately 3%. The general ophthalmologists tended to classify the indefinite subjects as POAG suspects, leading to misdiagnosis of 23 non-glaucoma individuals and 9 POAG patients. For such subjects, referral to glaucoma specialists or long-term follow up is necessary. Moreover, gathering more experience from glaucoma specialists according to the risk factors for accurate diagnosis of POAG was a shortcut for general ophthalmologists to improve the level of diagnosis. One study in Korea also investigated the interobserver agreement between a glaucoma specialist and residents, indicating the weighted kappa values varied from 0.28 to 0.70.23 The subspecialists in their study had less experience (junior grade) compared to those in our study. The disagreement also suggested that the clinical skill and awareness related to glaucoma detection needed to be improved in general ophthalmologists.

Many risk factors for POAG have been reported over the years, including older age, male gender, family history, diabetes mellitus, myopia, elevated IOP, larger sup-to-disc ratio, thin central cornea.25–31 However, risk factors for accurate diagnosis of POAG are still missing. In the present study, we demonstrated that IOP and VCDR were the independent risk factor for reliable diagnosis of POAG. The odds ratio was 8.363 (95% CI: 4.27–16.37) for IOP and 3.459 (95% CI: 1.54–7.76) for VCDR. This suggested that participants with high IOP or large VCDR were at a higher risk of being misdiagnosed with POAG by general ophthalmologists, thus implying that the diagnosis of POAG should not just rely on the high IOP or large VCDR but on the optic disc sign and abnormal visual field as well. Thus, more clinical training should focus on understanding the optic disc sign and the visual field.

Our study has some limitations. It was a hospital-based cross-sectional study involving four top eye hospitals in the area of Shanghai. Still, among the participants, who were outpatients, those previously diagnosed with glaucoma were excluded, and therefore the initial diagnosis rate in outpatients was lower than the reality. Moreover, the findings of this hospital-based study could not be generalized to other population-based studies. Finally, the study focused only on POAG because primary angle-closure glaucoma and secondary glaucoma have different diagnostic criteria and might be more complicated, which needs further study.

In conclusion, our data suggest that general ophthalmologists’ level and awareness for glaucoma diagnosis need to be improved. There is a need for public health measures and research into appropriate and cost-effective screening strategies for detecting glaucoma.