Accuracy of noncycloplegic photorefraction using Spot photoscreener in detecting amblyopia risk factors in preschool children in an Indian eye clinic.

Purpose: To evaluate the accuracy of Spot photoscreener (PS) as a noncycloplegic photorefractor in detecting amblyopia risk factors (ARFs) in preschool children in an Indian eye clinic setting. Also, to derive appropriate cutoff values for screening to obtain maximum sensitivity and specificity of the device in detecting ARF.
Methods: This was a cross-sectional study conducted in the outpatient pediatric eye clinic at a tertiary eye care institute. A Spot PS was used to screen all the children between the ages of 6 months and 5 years that presented to the eye clinic from August 2018 to October 2018. This screening was followed by a complete eye examination, including cycloplegic refraction by a masked examiner. The 2013 American Association for Pediatric Ophthalmology and Strabismus (AAPOS) guidelines were considered the standard cutoff values for clinically significant refractive error in children younger than 5 years of age.
Results: The study comprised of 219 children. The Spot PS diagnosed 135 (61.64%) children with ARF as compared with 124 (56.62%) children detected by clinic examination. For ARF detection, the Spot photoscreeneer had 85.48% sensitivity, 69.47% specificity, 78.52% positive predictive value and 78.57% negative predictive value. The sensitivity for detection of strabismus and hypermetropia was very low (42% and 36%, respectively). The 95% limits of agreement ranged from -5.48 to +5.59 diopters (D) with a bias of 0.06 D for spherical equivalent between noncycloplegic photorefraction and cycloplegic refraction.
Conclusion: The Spot PS may be used as a screening tool to detect ARF in children younger than 5 years of age keeping its limitations in consideration. However, the performance can be improved by modifying the cutoff values for the referral.

Amblyopia is a unilateral or, less often, bilateral reduction of best-corrected visual acuity (BCVA) caused by form vision deprivation and/or abnormal binocular interaction. For this, there is no identifiable pathology of the eye or a visual pathway and it is reversible when treated appropriately.[1] The prevalence of amblyopia in childhood is approximately 2.5%. However, the prevalence of amblyopia risk factors (ARFs) is much greater, around 21%.[2] The most common risk factors for unilateral amblyopia include strabismus and significant refractive error and for bilateral amblyopia are bilateral astigmatism and bilateral hypermetropia.[3] Early screening and treatment are associated with a 70% lower prevalence of amblyopia.[4]

A comprehensive clinical evaluation of every child below five years of age by an ophthalmologist requires a large volume of resources. Additionally, it is difficult to assess vision and refraction in small children in non eye clinic settings. Thus, screening devices with high sensitivity and specificity can be an effective alternative in early detection and accurate referral.

Photoscreeners (PSs) are newer tools that aid in the assessment of refractive error based on the light reflex test. Spot PS is an easily portable, handheld, wireless, battery-powered device, available with a computer interface.[5] It is based on the phenomenon of photoscreening, which uses optical images of the eye's red reflex to estimate refractive error, media opacity, ocular alignment, and other factors, such as ocular adnexal deformities (e.g., ptosis). All the above mentioned factors put a child at a risk for developing amblyopia.[1] While adnexal problems and media opacities are readily visible, uncorrected refractive errors and strabismus are often missed by a simple torchlight examination. The Spot PS reports refraction within the range of −7.50 diopters (D) of myopia to +7.50 D hypermetropia.[5] The American Academy of Pediatrics (AAP) has issued a policy statement supporting the use of PSs for screening in children between 6 months and 3 years of age, in older children who are unable or unwilling to cooperate with routine acuity screening, and as an alternative to visual acuity screening with vision charts from 3 through 5 years of age.[6] This has not found popularity in Indian childhood eye screening programs and most of the vision screening programs for children are aimed at detecting uncorrected refractive errors in children older than 5 years of age.[78] This is, definitely, a barrier in early detection of ARF in preschool age, where it is the most amenable to treatment. There is also concern regarding the use of cycloplegic agents during the screening process in the absence of ophthalmologist and guardians, owing to their potential side effects. Therefore, the purpose of this study was to evaluate the accuracy of the Spot PS as a noncycloplegic photorefractor in detecting ARFs in preschool children. We also aimed to derive appropriate cutoff values for refractive error, which can be applied in community eye screening of preschool children for early detection of ARF.

Methods

This was a cross-sectional study conducted in the outpatient pediatric eye clinic at a tertiary eye care institute. The study received approval from the institutional review board and ethics committee of the hospital. All children between ages of 6 months and 5 years presenting to the eye clinic from August 2018 to October 2018 were screened using the Spot™ Vision Screener (Welch Allyn, Skaneateles Falls, NY, USA) after obtaining appropriate consent.

All children with previous intraocular surgery or trauma were excluded. Additionally, those children who did not cooperate for photoscreening or cyclorefraction (CR) and whose parents refused to give consent for the same were excluded from the study. The image acquisition by the PS was done at a distance of 3 feet (approximately 1 meter), keeping the device at the child's eye level and with the child fixating on the display of light and sound with a straight-ahead head posture. The examination room was slightly illuminated without allowing any direct daylight. Information was displayed on a 4.5-inch touch screen and stored for printing and generating Excel database for analysis. When the device was unable to evaluate a subject, it notified with a reason like “pupils too small” or “pupils not found” or “out of range” or continued attempting to obtain a reading till a result was obtained after three attempts.[6]

The Spot PS reports detailed refraction within the range of −7.50 D of myopia to +7.50 D hypermetropia. Values beyond this range are reported as greater than 7.5 D and a detailed refraction value is not provided by the machine. Thus, refractive error values more than +7.5 D or less than −7.5 D were not included in the analysis for correlation of the absolute values of refraction by the PS and cycloplegic retinoscopy. However, they were included in sensitivity and specificity analysis. It detects strabismus based on the measurement of gaze calculated from the corneal light reflex.[6] The criteria for gaze cut off are based on the degrees of displacement of the corneal light reflex from the pupillary center. The device software considers a 3.3-degree temporal displacement as normal adjustment. Thus, referral criteria for either eye include displacement of corneal light reflex 5° nasal, 8° temporal, 8° vertical, and asymmetry of 8°. The asymmetry is calculated as the sum of differences in horizontal and vertical deviation in comparison of both eyes.[6]

Spot provides a result of screening in the form of either “all measurements within range” or “complete eye exam recommended” based on predetermined referral criteria that can be entered in the device. The former was considered as a screening negative while the latter was considered as screening positive in the analysis. For the purpose of this study, the referral criteria of Spot PS for refractive error was modified as per the 2013 American Association for Pediatric Ophthalmology and Strabismus (AAPOS) guidelines in different age groups.[9] According to the AAPOS guidelines, the amblyogenic refractive error is considered when astigmatism in children aged 12–30 months is >2.0 D, myopia is >-3.5 D, hypermetropia is >4.5 D, and anisometropia is >2.5 D; when astigmatism in children aged 31–48 months is >2.0 D, myopia is >-3 D, hypermetropia is >4.5 D, and anisometropia is >2.0 D; and when the astigmatism in children aged 49–60 months >1.5 D, anisometropia is >1.5 D, myopia is >-1.5 D and hypermetropia is >3.5 D.[9] According to these criteria, visually significant media opacities (>1 mm) and manifest strabismus (>8 prism diopters [PD] in primaryposition) should be detected for all ages.[9]

All children eligible for the study underwent screening by a PS under undilated state. Thereafter, retinoscopy was done after administration of appropriate cycloplegic agent. Atropine eye ointment 1% twice a day for 3 days was used for cycloplegia in the presence of esotropia or age less than 2 years while cyclopentolate 1% eye drop was used twice to thrice on the same day in other children. Both the examinations (photoscreening and cycloplegic refraction) were performed by different optometrists, who were masked to each other's findings. Additionally, another masked ophthalmologist performed the clinical examination for the evaluation of strabismus before cycloplegic agents were instilled.

Statistical analysis

With a 95% confidence interval (CI) of ±0.10 and assuming the maximum possible variance (0.25) for the estimators of sensitivity and specificity, the minimum required sample size is estimated to be 97 individuals in each category of cycloplegic refraction positive and negative as diagnosed in the clinic. The total minimum sample size for the study is thus 97 + 97 = 194.

The minimum required sample size was calculated using the following formula:

Sample size, n = (Z1−α/2/d)2 × p(1 − p)

Where, α = probability of error, in other words (1− α) is the level of significance.

Z1−α/2= Z-score, corresponding to the level of significance equal to (1− α).

d = acceptable margin of error, i.e. length of the CI = 2d.

Assumptions: 1− α = 95%, d = 0.10 and p(1 − p) = 0.25.

Readings from both the eyes were considered independently in the analysis for comparing refractive error between PS and CR. Anisometropia was defined as a difference in spherical equivalent of >1.5 D. All cylindrical values were converted to myopic depiction for ease of comparison.

To assess the validity of the PS, sensitivity, and specificity, positive predictive value and negative predictive value were calculated for spherical equivalent (SE), myopic sphere, hypermetropic sphere, astigmatism, anisometropia, and strabismus. This was performed using the screening positive and screening negative results of the PS and comparing with cycloplegic retinoscopy for refraction (after application of same AAPOS 2013 cutoff values) and clinical evaluation for strabismus. Additionally, paired t-test and curve estimation regression analysis were performed to assess the difference and quantitative relationship between the absolute measurements obtained from the Spot PS and those from CR. The Bland–Altman (B and A) plot was used to document the agreement of the measurements of the PS and CR. The receiver operating characteristic (ROC) curve was used to select the best cutoff points related to appropriate sensitivity and specificity of the Spot PS.

Results

A total of 222 children between the age range of 6 months and 5 years presenting to the outpatient clinic of the pediatric ophthalmology and strabismus department were screened in the 3-month period. Three children did not cooperate for photoscreening and were excluded with 219 (438 eyes) children completing both examinations successfully [Fig. 1]. Out of these, 126 (57.53%) were males and 93 (42.47%) were females. The mean age of these children was 41.8 ± 5.83 months. There were 23 eyes with out-of-range hypermetropia >+7.5 D (six eyes) or myopia <−7.5 D (17 eyes) measured by Spot PS. Hence, these were also excluded and, finally, 412 eyes were included in the analysis.

Figure 1

Flow of the process of inclusion and examination of children in the study

ARFs detection by Spot PS

The Spot PS diagnosed 135 (61.64%) patients with ARF, which was depicted as “screening positive” or “complete eye examination recommended,” and 84 (38.36%) patients had no ARF, which was labeled as “all measurements in range” or “screening negative.” Examination at the clinic, including CR found ARFs in 124 (56.62%) children. Ptosis or any media opacity was not detected in the children participating in the study. Table 1 shows the 2 × 2 contingency table of the PS in detecting the ARF when compared with the complete examination at the clinic, including CR.

Table 1

2 × 2 contingency table of PS vs clinical examination for detection of ARFs

	Clinic examination including CR		Total	P
	Positive	Negative
PS result
Positive	106 (48.40%)	29 (13.24%)	135 (61.64%)	<.0001
Negative	18 (8.22%)	66 (30.14%)	84 (38.36%)
Total	124 (56.62%)	95 (43.38%)	219 (100.00%)

PS=Photoscreener, ARFs=Amblyopia risk factors, CR=Cyclorefraction

Refractive ARFs measured by CR

The range of refractive errors (SE) detected in the study cohort via CR was from −10.6 D to +10.0 D. Hypermetropia more than the cutoff was found in 61 eyes (13.93%), of which 58 (13.24%) eyes had values greater than +3.5 D. Myopia more than the cutoff as an ARF was found in 62 (13.93%) eyes, of which 52 (11.87%) eyes had values lesser than −3 D. Anisometropia was found in 11 children (5.02%). Astigmatism was present in 137 eyes (31.27%).

Validity of Spot PS

The sensitivity of the Spot PS was detected as 85.48%, specificity as 69.47%, positive predictive value as 78.52%, and negative predictive value as 78.57% for the detection of ARF. Table 2a shows the sensitivity, specificity, positive predictive value, and negative predictive value of PS for individual refractive error and strabismus when compared with cycloplegic retinoscopy and detailed clinical evaluation. There was a low sensitivity for the detection of hypermetropia and strabismus by the PS.

Table 2a

Sensitivity, specificity, positive predictive value, and negative predictive value of PS for individual ARFs

	Sensitivity	Specificity	Positive Predictive Value	Negative Predictive Value
Myopia	73.02%	94.40%	68.66%	95.42%
Astigmatism	57.86%	84.90%	64.29%	81.09%
Hypermetropia	36.07%	97.08%	66.67%	90.37%
Strabismus	42.22%	91.95%	57.58%	86.02%
Anisometropia	53.85%	92.72%	31.82%	96.95%

PS=Photoscreener, ARFs=Amblyopia risk factors

Comparison of refractive error between CR and PS

Paired t-test was used to compare the actual refractive error detected by PS with the CR. The mean of the differences in the amount of SE, astigmatism, and anisometropia were statistically similar while hypermetropia (P = 0.021) and myopia (P = 0.000) showed a statistically significant difference between both the groups [Table 2b]. The mean SE obtained from Spot PS was 0.06 ± 2.82 D lower than that of cycloplegic retinoscopy.

Table 2b

Comparison of individual refractive risk factors between CR and PS

	Mean of the Difference (CR-PS)	SD	P (paired t-test)
CR SE-PS SE	0.06	2.82	0.676
CR Myopia-PS Myopia	-2.23	3.61	0.000
CR Hypermetropia-PS Hypermetropia	0.26	1.95	0.021
CR Astigmatism-PS Astigmatism	0.14	1.19	0.054
CR Anisometropia-PS Anisometropia	-0.14	1.06	0.059

CR=Cyclorefraction, PS=Photoscreener, SD=Standard deviation

Factors affecting the error in measurement

There was no correlation found between the error of SE between the two groups when compared with age of the patients (P = 0.82), gender (P = 0.36), or magnitude of refractive error (P = 0.51).

Agreement analysis

The difference of SE (CR − PS) was plotted against the mean of CR and PS values to obtain a B and A agreement graph [Fig. 2a]. Similar graphs were constructed for hypermetropia [Fig. 2b], myopia [Fig. 2c], astigmatism [Fig. 2d], and anisometropia [Fig. 2e]. The B and A analysis for SE revealed that the 95% limits of agreement ranged from −5.48 to +5.59 D with a bias of 0.06 D [Fig. 2]. The difference in SE was within 1D in 52.4% and within 2D in 76.7% of patients.

Figure 2

Bland–Altman assessment of agreement between the cyclorefraction and photoscreener for spherical equivalent (a), hypermetropia (b), myopia (c), astigmatism (d), and anisometropia (e). The dashed line depicts the mean deviation and the solid lines depict the 95% limits of agreement

Fig. 3 shows the ROC for detection of hypermetropia, myopia, astigmatism, and anisometropia. According to the ROC curves, the best sensitivity and specificity were obtained when referral criteria for hypermetropia, myopia, astigmatism, and anisometropia were adjusted to +1.38 D, -0.63 D, 1.36D, and 0.63 D, respectively.

Figure 3

ROC curve analysis for detection of refractive amblyopia risk factors (a) Hypermetropia, (b) myopia, (c) astigmatism, and (d) anisometropia

Strabismus

Strabismus was missed by the PS in 21 (51.21%) out of 41 squints diagnosed by the ophthalmologist in the clinic (P < 0.001). Out of these, the PS failed to diagnose 16 children with esotropia (10 PD to 50 PD) and 5 children with exotropia (25 PD to 45 PD).

Discussion

The use of PS as a tool for early detection of ARFs in preschool children, especially under 5 years of age, has not been studied in India. The only prior study in India was conducted by Panda et al. in a hospital-based setting in a tribal region of Odisha.[10] They included 177 children aged 4–16 years and obtained a sensitivity of 93% in detecting ARFs. This is high in comparison with our study. Also, in their study, the difference in the SE lied within the 1 D in 87% of children, which is higher than that found in our study (52%).[10] These differences can be explained by the differences in population area and the age groups included in the two studies. We conducted our study in an urban setting while they conducted it in a rural setting. This could explain the difference in sensitivities. Also, younger children can have highly variable accommodation during noncycloplegic photorefraction, which may account for the wide limits of agreements found in our patients.[11] The present study wanted to evaluate ARFs in preschool children, especially under 5 years of age, in the eye clinic, before planning an extensive study in the community Anganwadi program.

In our study, the Spot PS showed a sensitivity of 85.48%, specificity 69.47%, and positive predictive value 78.52% in detecting the ARFs in preschool children, which is comparable with several other studies performed in children of similar age groups.[12131415] Silbert et al. obtained a sensitivity of 87% and specificity of 74% in children aged 1–6 years.[12] Similarly, Forcina et al. evaluated the use of PS in 0.5 to 3 years aged children and found the sensitivity to be 89.8% and specificity to be 70.4%.[14] Both these studies were also clinic-based studies and used the AAPOS 2013 guidelines.[9]

The prevalence of ARF detected by CR was quite high (56.62%) in our study population. This is comparable with other studies involving children presenting to an eye clinic, like Qain et al. (57.5% prevalence of ARF) and Mu et al. (74.2% ARF prevalence)[1516] Higher prevalence of ARF in these studies, including ours, is probably because the study population comprised of children visiting the eye hospital with visual complaints. Thus, the prevalence of an ARFs would be higher in this sample than in the population. Asare et al. found prevalence of ARF, detected by the PS as 6.1%, in their community-based study in children between 18 to 59 months of age.[17] The selection of study population from eye clinic may have also overestimated the sensitivity of amblyopia screening.

Pearson's correlation coefficient between SE obtained by PS and CR showed a weak positive correlation (r2 = 0.214). Paired t-test showed that Spot PS tended to measure spherical equivalent with a small myopic shift (0.06 ± 2.82D). Mu et al. and Qain et al. also showed a similar myopic shift of −0.49 D and −0.17D, respectively.[1516] However, on comparing individual refractive errors, myopia detected by PS was found to be less than the actual value as assessed by CR and the difference was statistically significant. In children with high myopia, the absolute values of refractive error showed a large difference between PS and CR. For high myopia (<−6D), 4.49 D of myopia was underestimated by the PS. This was not found in other studies and we could not find a plausible explanation for it. Hypermetropia was also underestimated. This can be explained by the effect of induced accommodation, which is not fully overcome at 1-meter distance, especially in younger age group. It has been reported that preschool children can have variable levels of accommodation during photorefraction and it can go up to 4D.[11]

Strabismus was missed by the PS in 51.21% cases. Although, 13 (61.9%) children in whom strabismus was missed, were screened positive for refractive errors. The Spot PS sensitivity for detecting strabismus was 42.22% and specificity was 91.95%. Large esotropias (10 to 50PD) and exotropias were also missed (25 to 45PD). Peterseim et al. found the sensitivity and specificity of the Spot PS to detect strabismus as 77.17% and 93.73%.[18] The children included in their study had a mean age of 6 years (age range 11–221 months), which might have lead to higher sensitivity. They also suggested that the child's head position could affect the measurement of strabismus.[18] We made attempts to take measurements while the child was looking at the PS display in straight-ahead gaze; however, the examiner did not stabilize the head by holding it. This could have been another factor for under-detection of strabismus.

In B&A analysis, a moderate agreement was noticed on comparing spherical equivalents of Spot and CR. The proportion of children with >4 D difference in SE between CR and photorefraction was 9.95%. That implies that around 10% of refractive error estimates have a very large difference between both groups. The mean bias in the difference of absolute value of myopia between the two groups was very large (−2.23D) with large range of limits of agreement. These differences are much higher than those reported by Mu et al. and Panda et al.[1015] As mentioned before, both these studies have included children above 4 years of age. The absolute value of refractive error obtained by the PS depends on several factors like the attention span of the child, direction of gaze, and accommodative status, especially when measured in a noncycloplegic state. All these factors can account for the large variations in the refractive error values in our cohort of 0.5–5-year-old children.

The performance of the Spot PS can be improved by optimizing referral criteria based on the ROC analysis. ROC curves showed best cutoff values for hypermetropia, myopia, astigmatism, and anisometropia as 1.38 D, 0.63 D, 1.36 D, and 0.63 D. Using this revised referral criteria, the sensitivity and specificity for hypermetropia increased to 68% and 84% and sensitivity and specificity for myopia increased to 83% and 87%. Several other authors have previously described the use of optimized criteria to increase the sensitivity and specificity of the device suitable for a particular demographic profile of patients.[131518]

Additionally, we used the AAPOS 2013 guidelines for the prescription of glasses in children as the cutoffs for screening by PS. These guidelines are based on cycloplegic retinoscopy values. As we are comparing noncycloplegic refraction done by the PS with cycloplegic retinoscopy in children less than 5 years of age, the same referral criteria may not hold true for detection of ARF. Therefore, we need to revise the criteria for cutoff values of ARFs to get higher sensitivity and specificity, especially if the device is used for general population. Another limitation of our study was that the children included were selected from an eye clinic and most of them were expected to have preexisting eye problems. Thus, the prevalence of ARFs was higher as compared with the community, and it might have overestimated the sensitivity of amblyopia screening. Therefore, community-based studies are needed to see the applicability of our results in the general population.

Conclusion

The Spot PS was found to have an overall sensitivity of 85% and specificity of 70% for a screening of ARFs in preschool children presenting to the eye clinic. However, the sensitivity to detect strabismus and hypermetropia was very low. The absolute values of the refractive errors deduced by Spot PS in noncycloplegic state show huge variations when compared to cycloplegic retinoscopy. Thus, the cutoff values for referral may have to be modified to obtain maximum sensitivity and specificity in our population of children aged 0–5 years.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.