Clinical effectiveness of the Manchester Glaucoma Enhanced Referral Scheme.

BACKGROUND: Glaucoma referral filtering schemes have operated in the UK for many years. However, there is a paucity of data on the false-negative (FN) rate. This study evaluated the clinical effectiveness of the Manchester Glaucoma Enhanced Referral Scheme (GERS), estimating both the false-positive (FP) and FN rates.
METHOD: Outcome data were collected for patients newly referred through GERS and assessed in 'usual-care' clinics to determine the FP rate (referred patients subsequently discharged at their first visit). For the FN rate, glaucoma suspects deemed not requiring referral following GERS assessment were invited to attend for a 'reference standard' examination including all elements of assessment recommended by National Institute for Health and Care Excellence (NICE) by a glaucoma specialist optometrist. A separate 33 cases comprising randomly selected referred and non-referred cases were reviewed independently by two glaucoma specialist consultant ophthalmologists to validate the reference standard assessment.
RESULTS: 1404 patients were evaluated in GERS during the study period; 651 (46.3%) were referred to the Hospital Eye Service (HES) and 753 (53.6%) were discharged. The FP rate in 307 assessable patients referred to the HES was 15.5%. This study reviewed 131 (17.4%) of those patients not referred to the HES through the GERS scheme; 117 (89.3%) were confirmed as not requiring hospital follow-up; 14 (10.7%) required follow-up, including 5 (3.8%) offered treatment. Only one patient (0.8%) in this sample met the GERS referral criteria and was not referred (true FN). There were no cases of missed glaucoma or non-glaucomatous pathology identified within our sample.
CONCLUSION: The Manchester GERS is an effective glaucoma filtering scheme with a low FP and FN rate. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background

Community optometrists identify ~95% of suspect glaucoma and ocular hypertension (OHT) cases in the UK through General Ophthalmic Services (GOS) sight testing.1 Without ‘filtering’ the false-positive (FP) rate for suspect glaucoma referrals is high (~40%).2 3 Glaucoma referral filtering schemes have been in operation in the UK for more than a decade, and Manchester Glaucoma Referral Refinement Scheme (GRRS), now termed Glaucoma Enhanced Referral Scheme (GERS), was the first such scheme to be established.4 An early evaluation of the Manchester scheme observed a reduction in FP rate from ~40% to~10%; however, the false-negative (FN) rate was not evaluated. Indeed, there is a paucity of data on FN rates within community referral filtering schemes.4–6 One study7 included a retrospective analysis of 100 sets of notes and optic nerve images of patients not referred through the scheme. The authors quoted a 3%–10% FN rate and concluded there was ‘no compromise on patient safety’.7 Ratnarajan et al 8 included consultant clinical review of non-referred patients, although on a limited sample of 34 discharged subjects willing to attend the hospital for review, and concluded the FN rate of optometrists was 15%, although no cases of glaucoma were missed. We aimed to evaluate the clinical effectiveness of the Manchester GERS by estimating both the FP and FN rates. Companion qualitative work including the establishment and running of the Manchester GERS has previously been published.9 10 This paper is the first to report on the revised GERS’s clinical effectiveness and includes prospective evaluation of a relatively large sample of both referred and non-referred subjects.

Method

A favourable ethical opinion was granted by the National Research Ethics Service (NRES) Committee North West (IRAS reference 128578, REC reference 14/NW/0024) and added to the National Institute for Health Research (NIHR) portfolio with Manchester University NHS Foundation Trust as sponsor.

Manchester GERS pathway and patient recruitment for the false-negative study

The Manchester glaucoma referral pathway is summarised in figure 1. All Manchester patients identified by their optometrist as a glaucoma suspect requiring referral should be directed to GERS for referral filtering, unless emergency referral is required. The GERS criteria for onward HES referral are summarised in online supplementary appendix 1. Although the 2009 National Institute for Health and Care Excellence (NICE) treatment recommendation11 was not a referral recommendation, this guideline was used as a basis for GERS referral guidelines encompassing age, intraocular pressure (IOP) and central corneal thickness (CCT), facilitating alignment to risk profiles. Following GERS assessment, patients are directly referred to the HES glaucoma service, discharged to community optometry care or referred to the HES via a GOS 18 referral form where non-glaucomatous pathology is identified.

Figure 1

Manchester pathways for non-GERS (A) and GERS ((B) for patients identified as glaucoma suspects following a routine primary care eye examination. GERS, Glaucoma Enhanced Referral Scheme; GP, general practitioner.

Recruitment to the study ran from October 2014 to August 2016. If an optometrist identified possible glaucoma following a routine sight test in Manchester, their patient was referred to a GERS-accredited optometrist for evaluation. The GERS optometrist, postexamination, either made a direct referral to Manchester Royal Eye Hospital (MREH) for specialist assessment or discharged the patient from the scheme.

In order to evaluate the FN rate, 15 accredited GERS clinicians (14 optometrists and 1 ophthalmologist) were asked to provide all glaucoma suspects deemed to not require referral following examination with a patient information sheet informing them of the study and explaining that although their examination was satisfactory (and that in the opinion of the optometrist the patient had no need for further assessment), the hospital has invited them to participate in a study evaluating GERS. If the patient expressed unwillingness to be contacted, a form was completed to confirm their decision. For patients agreeing to be contacted, their details were passed to the research team via a contact request form. Adult patients ≥18 years of age consenting to participate and who post-GERS assessment were deemed to not require referral were eligible for inclusion. We excluded children, those unable to communicate in English and other patients unable to provide fully informed consent. All non-referred patients willing to attend MREH for assessment were contacted via telephone to offer a ‘reference standard’ examination at a research clinic.

Hospital research clinic ‘reference standard’ examination

To define an FN within GERS, a reference standard was required, and this assessment comprised all standard tests and examinations undertaken for a newly referred patient with suspect glaucoma. Reference examinations took place in hospital clinics undertaken by three experienced glaucoma specialist optometrists (ie, working in glaucoma specialist clinics for over 10 years) holding both the College of Optometrists’ Diploma in Glaucoma and Independent Prescribing qualifications. All three optometrists work both independently and also alongside consultant ophthalmologists in specialist glaucoma clinics. A comprehensive new patient examination was performed, including all elements of assessment recommended by NICE.11 Assessments in both GERS and hospital reference examination included: symptoms and history; general health and medications; previous eye history; family history; evaluation of glaucoma risk factors; visual acuities; anterior segment assessment; van Herick assessment; Goldmann tonometry; pachymetry; visual field assessment; and a dilated fundus examination incorporating clinical optic disc assessment using slit-lamp biomicroscopy (with either a 78D, 60D or 66D lens). The hospital reference standard examination also included gonioscopy, and a technician obtained colour optic disc photographs and OCT RNFL imaging. The care pathway for patients attending the reference standard study clinic was aligned to the remit of usual MREH care, and patients not requiring follow-up were discharged. More specifically, if glaucoma, OHT or suspected glaucoma (or any comorbidity) necessitated treatment or hospital review, treatment would be initiated if appropriate and a referral made to the relevant hospital clinic for follow-up.

Categorisation of patients

The HES optometrist reviewed clinical assessment data and classified patients into diagnostic groups: normal (or suspect where no HES follow-up needed), OHT, glaucoma suspect or glaucoma. Non-glaucomatous ocular comorbidities were noted, and appropriate advice, treatment and/or referral were arranged. An FN case was deemed to be a case of missed glaucoma (or any other comorbidity necessitating HES referral) or a glaucoma suspect or OHT case where GERS criteria for referral were triggered on reference standard examination. Specifically, we used IOP, visual fields, optic nerve head status and anterior chamber angle status by van Herick assessment. Parameters were assessed for adherence to single or combined GERS referral criteria (online supplementary appendix 1). A final FN classification was assigned to each participant as follows:

Unlikely: no evidence from any element of reference standard assessment of suspicion of glaucoma or OHT necessitating hospital review. Patients falling into this group did not, on reference standard assessment, meet any GERS referral criterion.

Possible: some element of assessment (eg, IOP, visual fields and optic disc status) provided evidence of possible requirement for review but potentially a requirement that may not have been met on community GERS examination (eg, due to variation in IOP).

Likely: much clearer evidence of a requirement at GERS examination for the participant to have required referral (eg, optic disc examination shows abnormality and confirmed visual field loss consistent with glaucoma).

Validation study

To assess the validity of reference standard examination by glaucoma specialist optometrists, 33 cases (comprising 30 randomly selected non-referred cases reviewed in the FN study supplemented by three GERS referred cases) were reviewed independently by two glaucoma specialist consultant ophthalmologists. Here, data from the clinical episode including history and symptoms, IOP, CCT, anterior segment assessment including van Herick’s grading, gonioscopy, visual field assessment and stereoscopic optic disc images were presented. The research optometrists’ decision making was masked, and the consultants were categorised patients into the same diagnostic groupings, that is, normal (or suspect where no HES follow-up needed), OHT, glaucoma suspect or glaucoma. They were also asked, according to GERS criteria, whether they would have discharged the patient or referred them to the HES.

False positives

Outcome data were collected for patients newly referred through GERS during the timeline of the study and assessed in usual care glaucoma clinics. In the absence of an electronic patient record, all outcome data were manually retrieved from the case notes, whereby clinicians were asked to retain patients’ records after clinics for GERS referred patients seen in HES new patient glaucoma clinics. Patients examined and subsequently discharged at their first visit were defined as FP cases for the purpose of this study.

Results

Recruitment

There were 1404 patients evaluated in GERS during the study period. The flow chart for patients seen in the scheme and study during this period is detailed in figure 2, facilitating assessment of risk of bias.12

Figure 2

Patient flow chart for recruitment of patients. GERS, Glaucoma Enhanced Referral Scheme.

False-positive study

For patients referred from the GERS and examined, we managed to retain case records for 307 patients and review the outcomes of the 283 patients who attended, as detailed in table 1.

Table 1

Outcomes of patients referred through GERS for MREH glaucoma assessment and where the outcome is known

	N	% (95% CI)
Monitor without treatment	153/283	54.1 (47.9 to 59.4)%
		OHT	18	11.8%
		PACS	13	8.5%
		Suspect	122	79.7%
Monitor with treatment	78/283	27.6 (22.6 to 33.0)%
		Glaucoma	42	53.8%
		OHT	13	16.7%
		PAC/S	13	24.4%
		Suspect	4	5.1%
Further investigations required (eg, day phasing)	9/283	3.2 (1.7 to 5.9)%
		Glaucoma	3	33.3%
		Suspect	6	66.7%
Did not attend	24/307	7.8%
Discharge	44/283	15.5 (11.8 to 20.2)%

There was no significant difference (p=0.31, χ2 test) across optometric practices when comparing the FP rate. Percentages with exact binomial CIs exclude non-attenders.

GERS, Glaucoma Enhanced Referral Scheme;MREH, Manchester Royal Eye Hospital;OHT, ocular hypertension;PAC, primary angle closure;PAC/S, primary angle closure suspect.

False-negative study

The overall return rate for contact request forms was 40.9%, although there was a large variation in return rate between community optometry practices (median (IQR) between practices 50 (15-67)%, p<0.001, χ2 test) with seemingly differential compliance with study recruitment requirements. Only one practitioner failed to return any forms, potentially relating to low overall activity in GERS, seeing just eight patients over the study period. The 72.1% consent rate in agreement to participate was, however, more consistent within forms returned from practices (median (IQR) between practices 75 (67-88)%, p=0.034), although there was still a statistically significant difference between practices. The mean time lag from GERS assessment to reference standard examination was~6.5 weeks (45.0 days, SD 38.7, range 3–208 days).

FN rate

The vast majority of patients were subsequently discharged at their HES study visit (N=117, 89.3%). The outcomes for those assessed in the FN study are summarised in table 2.

Table 2

Classification of patients seen in the FN study using the protocol definitions

	N	% (95% CI)		N	% (95% CI)
Glaucoma unlikely (did not meet referral criteria at GERS or require follow-up)	117	89.3 (82.9 to 93.5)%
Glaucoma possible (did not meet referral criteria at GERS, but required follow-up)	13	9.9 (5.9 to 16.2)%	Treated	5	3.8 (1.6 to 8.6)%
					OHT treated with drops	3	60.0%
					PACS treated with laser	2	40.0%
			Untreated	8	6.1 (3.1 to 11.6)%
					OHT/glaucoma suspect	4	50.0%
					PACS	4	50.0%
Glaucoma likely (did meet referral criteria at GERS but was not referred)	1	0.8 (0 to 4.2)%	Treated	0	0 (0.0 to 2.8)%
			Untreated	1	8.0 (0.0 to 4.2)%
					Glaucoma suspect	1	100%
Overall FN	14	10.7 (6.4 to 17.1)%	Treated	5	3.8 (1.6 to 8.6)%
			Untreated	9	6.9 (3.7 to 12.5)%

FN, false negative;GERS, Glaucoma Enhanced Referral Scheme;OHT, ocular hypertension;PACS, primary angle closure suspect.

Fisher’s exact test shows there was no significant difference between practices in the overall FN rate (HES vs discharge), p=0.27. Within our FN study sample evaluated in the HES, there were no incidents of missed non-glaucomatous pathology requiring referral post-GERS assessment.

When comparing consultant to consultant decision making, there was agreement between consultants on management decisions in 23 of 30 non-referred cases versus 25 of 30 and 24 of 30 cases where the glaucoma specialist optometrist and each individual consultant were in agreement. This finding suggests that the glaucoma specialist optometrists’ decision making demonstrated non-inferiority when compared with consultant–consultant agreement. There were only two cases (7%) where both consultants agreed, and their decision was different to that of the specialist optometrists’ decision, and in each case, the consultant would have discharged the patient, whereas the glaucoma specialist optometrist considered HES follow-up or treatment was required.

Discussion

A range of glaucoma filtering schemes has been developed in the UK with the aim to improve accuracy of referrals to the HES,4 6 7 including repeat measures, enhanced case findingand referral refinement schemes. The aim of this service is to provide a safe and efficient pathway for patients to be reviewed promptly after referral for suspect glaucoma, closer to home, by accredited optometrists, while reducing the FP referral rate to the HES. A previous evaluation has shown the FP rate to reduce from ~40% to ~10%.4 A subsequent study comparing a number of ‘referral refinement’ schemes demonstrated that specialist-trained optometrists can reduce the first visit discharge rate of patients subsequently reviewed in secondary care compared with direct referrals from non-specialist optometrists by 22%.5

In the present study, there was a low referral rate to the HES, with 53.6% patients being discharged following GERS community assessment, thereby potentially providing a financial saving to the NHS. The FP rate within the scheme was found to be 15.5%, much lower than that found in Ratnarajan et al’s5 study looking at referrals from non-specialist optometrists to the HES, where the mean FP rate was up to 43.9%. Indeed, another study showed the positive predictive value of referrals to be as low as 25%.13

Some previous research has examined the FN rate in referral filtering schemes, with the Carmarthenshire study showing an FN rate of 3%–10%.7 There are, however, difficulties associated with estimating the FN rate from retrospective review of case notes and optic disc photos, including the possibility that other ocular parameters may have been abnormal (eg, status of the anterior chamber angle). A recent prospective study looking at 82 patients seen in the Moorfields ‘virtual triage’ scheme showed an FN rate of 4%–20%, dependent on FN definition.14 The FN rate in Ratnarajan et al’s study8 was 15%; however, only a limited number of patients were examined in their study, and the results were assessed virtually. Our study is therefore the largest prospective FN study within a glaucoma filtering scheme, and uniquely the assessment of patients replicated usual care for referred patients, encompassing all NICE recommended investigations sufficient for diagnosis.

Only one patient meeting GERS criteria for referral was not actually referred. The GERS optometrist labelled visual fields as ‘normal’ on the GERS assessment pro forma, despite the attached visual field plot showing field changes where glaucoma could not be excluded. Reference standard examination indicated no definite glaucoma but deemed requirement for HES monitoring without treatment (ie, a chronic open angle glaucoma suspect). This single case indicates an FN rate of 0.8%, with 95% CI excluding values above 4.2%, where an FN is defined as a patient who was not referred but who ought to triggered referral at the point of GERS assessment. Following additional assessment a further 13 patients (9.9%) were followed up, 5 of whom (3.8%) received prophylactic treatment with topical medication or laser. Three of these five patients (2.3% of patients seen in the study) were patients diagnosed with OHT, but none had IOPs meeting the referral threshold at GERS assessment. There were no patients with narrow angles and raised IOPs deemed to require both medical and laser treatment. The discrepancy in IOP measurement between GERS and reviewer consultations may be explained by many factors, including interobserver and/or diurnal variation in IOP measurements.15 The remaining two treated patients (1.5%) had asymptomatic narrow angles who met the European Glaucoma Society Guidelines’ recommendation for treatment with YAG laser peripheral iridotomy.16 Both patients had a van Herick assessment of grade 1 in each eye but did not meet the then GERS referral criteria, since they were both asymptomatic. The remaining eight patients followed up without treatment either had narrow angles not requiring treatment but judged to benefit from monitoring (N=4, 3.1%) or labelled as glaucoma suspects (N=4, 3.1%) in accordance with NICE definitions.11 The one patient not referred, but actually meeting the referral criteria at the point of GERS, did not require treatment at follow-up, indicating there were no incidents of missed glaucoma requiring treatment within our study sample.

One potential limitation of the current research is the failure to secure outcome data on all patients seen within the HES following GERS referral. In the absence of an electronic patient record, data retrieval relied on clinicians to retain notes after clinic. The outcome data for those where patients’ notes could be reviewed is, however, comparable with an earlier analysis of the then GRRS scheme.4 All patients examined and subsequently discharged at their first visit were defined as FP cases for the purpose of this study. However, a proportion of these cases might arguably have been reasonably referred in, necessitating decision making by a more specialist clinician. A further limitation is arguably the use of experienced glaucoma specialist optometrists to provide the hospital-based reference standard assessment versus specialist glaucoma consultants. However, set against this potential limitation is the optometrists’ apparent caution in assessment in the validation study. While this validation study is limited in not affording consultants with an opportunity for direct patient examination (including gonioscopy) and they relied on the available reference standard examination findings, they did review subjects’ optic disc and visual field status. The optometrist had the same judgement in all cases where either consultant would refer plus an additional two (7%) cases, and the FN rate would not be expected to be underestimated compared with consultant assessment. Agreement in glaucoma management between glaucoma specialist optometrists and glaucoma specialist consultant ophthalmologists has been studied in the past, demonstrating good agreement in our study centre.17 Apparent disagreement between glaucoma specialists highlights the subjective nature of glaucoma assessment and the impossibility of a perfect referral filtering scheme, given measurement imprecision and the implications of balancing sensitivity and specificity in referral criteria. A further limitation is that of the delay (mean of~6.5 weeks) in HES reference standard assessment following community GERS examination, with possibility for real change in status in a minority of subjects included in our evaluation.

Reflecting on the results of this study, as well as on feedback from GERS accredited optometrists,10 GERS referral criteria have since been modified to specify that any patient with a van Herick grade 1 may be referred through GERS, regardless of symptoms. Following publication of updated NICE guidance,18 GERS referral criteria are being revised, taking into consideration referral criteria within the updated guideline, and these changes could potentially alter the outcomes. For example, in the present FN study sample, had an IOP criterion of ≥24 mm Hg been the referral threshold, nine referrals would have been made.

It is clear that GERS is highly successful in reducing the FP rate from community referrals for suspect glaucoma. The FN evaluation in this study shows that GERS is clinically effective and very safe, offering reassurance to commissioners wishing to implement community pathways.