Comparison of Peristat Online Perimetry with the Humphrey Perimetry in a Clinic-Based Setting.

PURPOSE: We determined the receiver operating characteristic (ROC) curves for Peristat online perimetry at detecting varying degrees of glaucoma and the correlation between Peristat online perimetry and Humphrey visual field.
METHODS: A prospective, comparative study of Peristat online perimetry (an achromatic static computer threshold testing program) and Humphrey visual field (HVF) 24-2 SITA standard testing was performed by 63 glaucoma patients and 30 healthy controls in random order. The number of total adjacent abnormal test points were identified for each test, and compared with Spearman correlation. Receive operating characteristic curves were generated for Peristat online perimetry detection of mild and moderate-severe glaucoma patients using contrast sensitivity thresholds of -16.7, -21.7, and -26.7 dB.
RESULTS: The area under the ROC curve for glaucoma detection ranged from 0.77 to 0.81 for mild disease (mean deviation [MD], >-6 dB on HVF) and 0.85 to 0.87 for moderate to severe disease (MD, <-6 dB on HVF) depending on contrast threshold. Peristat online perimetry and Humphrey visual field abnormal points were highly correlated with Spearman rank correlations ranging from 0.55 to 0.77 (all P < 0.001).
CONCLUSIONS: Peristat online perimetry exhibits a reasonable ROC curve without specialized equipment and exhibited significant correlation with the conventional 24° Humphrey visual field test. TRANSLATIONAL RELEVANCE: Low cost widely available internet-based visual fields may complement traditional office-based visual field testing.

In the United States, open angle glaucoma affects 2.5 to 3 million people with projections for over 7 million affected by 2050.[1,2] The detection and management of glaucoma provide unique challenges. Glaucoma detection is hampered by delayed diagnosis due to its prolonged asymptomatic clinical phase. It is estimated that only 34% and 8% of all open angle glaucoma cases are diagnosed in the developed and developing world, respectively.[1]

Current guidelines recommend screening for glaucoma with comprehensive eye examinations in high-risk populations.[3] These exams rely on eye care professionals, who are expensive, and only accessible to a small number of patients.[4] Perimetry is an essential aspect of the comprehensive exam for higher risk patients, with one survey showing that more than 90% of undetected glaucoma cases have significant visual field defects that would be apparent with standard perimetry testing.[5] This adds more time and cost to the initial evaluation with a typical Humphrey visual field (HCPCS code 92083) billing Medicare $65.04.[6] Once diagnosed, attending routine follow-up for glaucoma remains a challenge. Patient surveys have repeatedly shown that lack of access to transportation, scheduling conflicts, and appointment costs contribute to missing comprehensive eye examinations.[7-9]

The ability to perform online visual field testing is attractive for its potential to assist with detection and progression monitoring of glaucoma at low cost and with fewer barriers to patient access. Visual field tests show major variation with more than 80% of abnormal fields returning to normal values on repeat testing with the corresponding recommendation that progression be verified on three visual fields.[10,11] Such repetition of visual fields may be impractical and is frequently not achieved in routine clinical practice, but multiple in-home visual field tests with results relayed to providers could help to average out some of the variability seen among less frequent in-office visual field tests.[12] Such a home-based monitoring approach has shown improved outcomes in other fields of medicine, such as systemic blood pressure control.[13] While glaucoma screening remains an unsolved challenge, several studies have shown that screening algorithms that combine structural and functional metrics may outperform single method screening protocols.[14-16]An online, low-cost perimetry test, such as Peristat, could be useful under such a combined approach.

Peristat is a novel web-based virtual supra-threshold perimetry system that allows self-testing on any 17-inch or larger computer monitor. Peristat online perimetry (POP) sequentially tests a visual field of 24° from fixation horizontally and 20° vertically using four levels of standardized threshold stimuli, allowing patients to be tested for characteristic visual irregularities in less than 5 minutes per eye.[17] In contrast to Humphrey visual field (HVF), POP is a freely available, online program at the KeepYourSightFoundation website (available in the public domain at http://www.keepyoursight.org) that provides unlimited home access. A previous pilot study showed POP could be useful when performed in the office with a sensitivity of 84% to 86% and specificity of 94% to 97% compared to HVF for moderate or worse visual field defects.[17] However, in this study three masked graders subjectively evaluated correlations between POP and HVF.[17] In the current study, POP is compared to HVF using a quantitative and automatable approach to compare the correlation of the two testing methodologies in a new set of patients with and without glaucoma, including analysis of ROC curves for glaucoma detection.

Methods

Permission was obtained from the University of California, San Francisco (UCSF) and Stanford Institutional Review Boards to perform the study. Written, informed consent was obtained from all participating patients, and the study was conducted in compliance with the Declaration of Helsinki and Health Insurance Portability and Accountability Act (HIPAA) requirements.

Study Population

A convenience sample of patients was enrolled prospectively from glaucoma clinics at UCSF and Stanford. In addition, the spouses, friends, and caretakers of glaucoma subjects as well as patients from the comprehensive ophthalmology clinic were invited to participate as the normal controls based on fundus exam and best-corrected visual acuity. Participants underwent a comprehensive examination that included SITA standard Humphrey visual field (Zeiss, Oberkochen, Germany) testing and Peristat online perimetry (Keep Your Sight Foundation, San Francisco, CA). Inclusion criteria were: (1) a best-corrected visual acuity in the tested eye of 20/60 or better, (2) no additional ocular or neurologic causes of visual field deficits in the tested eye, and (3) able to reliably perform HVF and Peristat in the tested eye with reliability defined as less than 33% of fixation losses, 25% of false-positive responses, and 25% of false-negative responses within the first two attempts for both visual field tests. Only one eye per participant was included in the data analysis; the right eye was used unless it met exclusion criteria, in which case the left eye was used.

Diagnostic Tests

Humphrey visual field and Peristat online perimetry were performed by all participants in random order based on Excel random number generation from August 2013 to February 2014. Humphrey visual field was performed using the 24-2 SITA-standard algorithm as described previously.[18] Peristat online perimetry testing was conducted within 3 months of HVF testing on a 17-inch monitor in a darkened room in the clinic with guidance by a trained investigator. Stimuli were displayed sequentially in a preset algorithm, interrogating a visual field of 24° from fixation horizontally and 20° vertically (Fig. 1A). The target size is 3 mm in diameter and presents within 6° intervals. Desired stimulus intensity is achieved after background monitor calibration to lowest intensity and a preset empiric algorithm that includes the respective red-green-blue (RGB) color model values on the specific monitor and corresponding light intensity for each threshold level. This model was designed to standardize otherwise significant variability across computer monitors, surpassing the variability of 95% of computer monitors. The RGB range for the target was between 64 and 255 potential color values with a differential light intensity between 30 and 300 lux. Each target is presented for 0.2 seconds, and the patient is allowed an additional grace period of 1.8 seconds to respond. Points were tested at one of three stimulus thresholds with −16.7, −21.7, and −26.7 dB change from background illumination based on empiric measurement of average background monitors, which corresponded to light grey, dark grey, and black squares mapped as visual field defects, respectively (Fig. 1B). The patient was tested in the right eye first and then the left eye with the untested eye occluded. Working distance was approximately 18 inches but specifically determined by having patients fixate on a central white circle and then adjust their distance until a blinking indicator in the periphery disappeared as it entered the blind spot. The blind spot stimulus was used for head position and fixation. Fixation losses were recorded for responses to stimuli in the blind spot, false-positives were recorded for responses without stimuli, and false-negatives were recorded for failure to respond to stimuli that had previously generated a correct positive response at the same or weaker contrast. Unreliable Peristat testing was repeated up to three times. A unique account was created for each patient to store Peristat data via the Keepyoursight foundation (available: http://www.keepyoursight.org).

Figure 1

(A, B) Peristat online perimetry. Patients were instructed to fix their vision on a central white circle at such a distance that the blinking green circles would disappear in their blind spot. High contrast stimuli were initially presented in the central visual area to establish a baseline response pattern followed by peripheral stimuli of varying intensity (left). Patients were instructed to respond to each stimulus by pressing the computer space bar. The errors were recorded to create a plot of visual field defects at −16.7, −21.7, and −26.7 dB contrast sensitivities (right).

Outcomes

Presence of glaucoma was defined by two authors' clinical assessment (YH and RC) on the basis of neuroretinal rim thinning, excavation, or retinal nerve fiber layer defects in conjunction with characteristic HVF abnormality confirmed on at least two HVF. Severity of glaucoma was defined by HVF mean deviation (MD) using thresholds of the Hodapp-Parrish-Anderson criteria where MD values of mild glaucoma are >−6 dB, moderate between −6 and −12 dB, and severe is <−12 dB.[19]

Statistical Considerations

We compared the first reliable HVF to the first reliable POP visual field test. We counted the total number of adjacent abnormal points in pattern deviation (PD) plot and total deviation (TD) plot for HVF (e.g., number of points at <5%, <2%, <1%, and <0.5%) and POP (e.g., number of points at −16.7 dB or worse, −21.7 dB or worse, and −26.7 dB or worse) in the entire visual field. We defined adjacent points as 2 or more decreased responses next to each other including diagonal in the visual field grid as an isolated abnormal response may not be meaningful. When counting points at a given threshold, any point that was at or more severe than the threshold being analyzed was counted as a failure to respond. We assessed the correlation of the total number of abnormal adjacent points between the two tests using Spearman's rank correlation. Because POP is a novel test and there is no previous reference to determine which level of POP sensitivity has the best correlation with HVF, correlation was determined across all potential sensitivity levels for POP (−16.7, −21.7, and −26.7 dB) and HVF (5%, 2%, 1%, and 0.5% for TD and PD). Bonferroni correction was used to adjust P values for multiple comparisons. Receiver operating characteristic (ROC) curves were generated to assess the diagnostic accuracy of each of the Peristat sensitivities (<−16.7, <−21.7, and <−26.7 dB), separately for a reference standard of mild glaucoma or worse (versus no glaucoma), and moderate glaucoma or worse (versus mild or no glaucoma). The area under the curve (AUC) was calculated for each ROC curve with 95% confidence intervals (CI) using modified Walden intervals with continuity correction and the maximum Youden index (sensitivity + specificity −1) and its associated sensitivity and specificity were calculated.

Results

We assessed 77 participants from the glaucoma clinics and 30 individuals not attending these clinics. Of these, 93 (86.9%) participants were enrolled. Excluded participants were all from the glaucoma clinic (four subjects failed to meet visual acuity criteria, six were unable to perform reliable HVF, and four were unable to perform Peristat). After ophthalmologist assessment for enrollment criteria, 63 participants were identified with glaucoma, all from the glaucoma clinic: 35 had mild, 16 had moderate, and 12 had severe glaucoma. The demographic characteristics of participants with glaucoma was similar to those without glaucoma; in contrast, the eyes of patients with glaucoma had significantly greater cup-to-disc ratios and slightly worsened visual acuity compared to those without glaucoma (Table 1).

Table 1

Continuous Variables are Shown with Standard Deviation (SD) and P Values are Calculated with Wilcoxon Rank-Sum Testing

Diagnostic attributes of Peristat and HVF were evaluated. Figure 2 shows the scatterplot of the total number of abnormal adjacent test spots on Peristat at <−16.7, <−21.7, and <−26.7 dB level versus the total number of abnormal adjacent test spots on HVF at 5%, <2%, <1%, and <0.5% level counted on TD plot (Fig. 2A) and on PD plot (Fig. 2B). When compared to different abnormality levels of HVF, the correlations between POP and HVF for PD and TD plots were the strongest when the total number of abnormal adjacent spots on POP was counted at −16.7 dB level. The correlation between the total numbers of abnormal adjacent test spots between the two tests did not show a simple linear relationship, so Spearman rank correlation between POP and HVF was calculated. The values of Spearman's rho ranged from 0.55 to 0.77 for POP compared to HVF TD and 0.60 to 0.75 for POP compared to HVF PD (all P <0.001; Figs. 2A, 2B).

Figure 2

(A, B) Scatterplots showing the number of missed adjacent Peristat points against missed adjacent Humphrey visual field points for total deviation (left) and pattern deviation (right). Peristat missed points are shown at −26.7, −21.7, and −16.7 dB while HVF points are shown at 5%, 2%, 1%, and 0.5%. Spearman correlation coefficients (ρ) are shown in right lower corner for each scatterplot; all were significant at the P < 0.001 level after Bonferroni adjustment.

The ROC curve was plotted to test the ability of POP to identify mild or worse glaucoma (Fig. 3A) and moderate or worse glaucoma (Fig. 3B). The area under the ROC curve for detecting mild or worse glaucoma was 0.81 (95% CI, 0.71–0.90), 0.77 (95% CI, 0.67–0.87), and 0.77 (95% CI, 0.67–0.87) for −16.7, −21.7, and −26.7 dB thresholds, respectively. The area under the ROC curve for detecting moderate or worse glaucoma was 0.87 (95% CI, 0.80–0.95), 0.85 (95% CI, 0.77–0.94), and 0.85 (95% CI, 0.76–0.93) for −16.7, −21.7, and −26.7 dB thresholds, respectively. Criteria for labeling a case positive that maximize the Youdon index along with corresponding sensitivity and specificity are shown in Table 2. At these cutoffs, specificity ranged from 85% to 100%, sensitivity ranged from 54% to 86%, the lowest positive likelihood ratio was 5.7 and the highest negative likelihood ratio was 0.46. There was no significant difference between the AUC of the three ROC curves (i.e., −16.7, −21.7, or −26.7 dB) for distinguishing mild or worse glaucoma versus controls (P = 0.36) or moderate or worse glaucoma from the combined controls and mild glaucoma patients (P = 0.75, χ2 test).

Figure 3

(A, B) Receiver operating characteristic (ROC) curves for Peristat detection of patients with glaucoma severity equal or greater than mild glaucoma (0.81, 0.77, 0.77) or moderate glaucoma (0.87, 0.85, 0.85) at low (−16.7 dB), medium (−21.7 dB), and high (−26.7 dB) contrast sensitivity, respectively.

Table 2

The Number of Points Missed Shows the Criteria for Categorizing a Study as a Positive Case with Corresponding Sensitivity and Specificity at That Threshold

Discussion

This study investigated the ROC curve of POP to discriminate patients with glaucoma from controls and its correlation with HVF testing. We piloted a quantitative approach towards evaluating points missed to categorize POP results that does not depend on the subjective evaluation by ophthalmologists as done in a previous study.[17] Among patients who were able to complete both tests successfully, there was a significant correlation of adjacent test points missed between the results of POP and HVF, ranging from 0.55 to 0.77. Peristat online perimetry testing had a good ability to discern patients with moderate to severe glaucoma from those with mild glaucoma and healthy controls, with an AUC ranging from 0.85 to 0.87 and high positive likelihood ratios.

Rarebit is a similar computer-based visual field testing program requiring only a 15-inch personal computer. Rarebit investigates 30 test spots each occupying 5° of central and up to 30° of peripheral vision using single and double simultaneous stimulation presented in pseudo-random order in which patients must identify whether there were zero, one, or two stimuli. We are aware of only one study looking at the ROC for Rarebit detection of patients with mild glaucoma versus healthy controls.[20] This study yielded an AUC of 0.89 to 0.95 for distinguishing glaucoma patients versus healthy controls. However, a Pearson's correlation coefficient between HVF MD and Rarebit hit rate was relatively low at 0.38 among patients with glaucoma. This may be partially due to the fact that in the Rarebit study, patients with ocular hypertension were excluded from analysis of ROC curves with possible spectrum bias.

Frequency doubling technology (FDT) is a well-studied and relatively portable method for visual field testing, though at higher cost than online software. Comparing Peristat with FDT studies is difficult as gold standard definitions of glaucoma vary to include optic nerve head changes alone,[21-23] nerve changes with visual field abnormalities,[24-26] nerve head changes with elevated IOP,[27] or a combination of nerve changes, elevated IOP, and visual field changes.[28] Thresholds for healthy controls similarly vary.[21-24,27,28] Our results may be most comparable with those FDT studies with the same definition of glaucoma based on optic nerve head appearance and visual field.[24,26] These studies found an AUC for FDT of 0.88 to 0.95 for distinguishing early glaucoma patients from healthy controls and 0.98 to 1.00 for distinguishing moderate glaucoma.[24,26] The linear correlation coefficients between FDT and standard automated perimetry ranged from 0.75 to 0.84.[24,26] This compares with AUCs of 0.78 to 1.00 in the wider literature of visual field tests.[21-23,27-29] Expanding to nonvisual field–based glaucoma screening methods, the AUC for distinguishing glaucoma versus health controls is in the 0.70 to 0.95 range.[14,22,25,30-33]

Accordingly, our finding of an ROC curve that ranged from 0.77 to 0.87 and Spearman correlation varying from 0.55 to −0.77 is comparable to some of the estimations of previously studied technologies. Peristat online perimetry has the distinct advantage that the basic equipment required is a 17-inch computer with internet connection. This advantage allows patient-administered, home-based evaluations for more frequent visual field monitoring in diagnosed patients and opens the possibility that POP could test functional perimetry as part of a multimodal glaucoma screening program. Unfortunately, as a single screening test the Peristat would miss at least 14% of moderate or worse glaucomas and up to 46% of early glaucomas. Additionally, given the low prevalence of glaucoma at under 2% in the United States, even with a sensitivity of 71% and specificity of 92% the POP has a low positive predictive value of approximately 15% when used as a single screening test.

This study has several limitations. The patients and controls examined were enrolled as a convenience sample of patients and their spouses, friends, and caretakers. While glaucoma patients were enrolled consecutively, participant without glaucoma were not. Glaucoma patients also were already established in clinics and familiar with performing HVF testing. This familiarity with HVF testing would be true if POP were used to supplement information from HVF in patients with known glaucoma, but limits applicability to a perimetry naïve population that may undergo multimodal glaucoma screening. In this study, visual field testing was conducted with technical supervision and support to establish whether it may be feasible in any environment. The room was dimly lit during this testing but not controlled at a specific level of luminance and the monitor screen size was held constant for all Peristat testing. Moving forward, there is a need for future studies of POP performed in home without support and with likely even larger variations in background luminance and monitor sizes to determine whether results can be replicated out of the office. We excluded patients who had other causes of ocular pathology, though in a diverse clinical population patients may have nonglaucomatous etiologies for visual field defects. The reference standard for glaucoma diagnosis varies in the literature. Our definition, which incorporates HVF as part of the case definition, may unduly favor the ROC of perimetry-based testing, such as the Peristat, that measure a similar functional outcome.

In conclusion, for patients who were able to complete POP and HVF successfully in a glaucoma clinic, the number of abnormal points on the POP was significantly correlated with the number of points missed on the HVF, and the AUCs of the POP generally were comparable to other visual field testing modalities. Additional studies are warranted to determine whether visual field evaluation remains reliable in larger populations and when the tests are performed at home for HVF-naïve patients.