Choosing appropriate tools and referral criteria for vision screening of children aged 4-5 years in Canada: a quantitative analysis.

OBJECTIVES: To assess the diagnostic accuracy of five vision screening tools used in a school setting using sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV).
DESIGN: We compared the results of the five best evidence-based screening tools available in 2014 to the results of a comprehensive eye exam with cycloplegic refraction by a licenced optometrist. Screening included Cambridge Crowded Acuity Cards, Plusoptix S12 and Spot photoscreeners, Preschool Randot Stereoacuity Test and the Pediatric Vision Scanner (PVS). Referral criteria followed AAPOS (2013) guidelines and published norms.
SETTING: A large school in Toronto, Canada, with 25 split classrooms of junior kindergarten (JK: 4 year olds) and senior kindergarten (SK: 5 year olds) children. PARTICIPANTS: Over 2 years, 1132 eligible children were enrolled at the school. After obtaining parental consent, 832 children were screened. Subsequently, 709 children had complete screening and optometry exam data. MAIN OUTCOME MEASURES: The presence/absence of a visual problem based on optometrist's assessment: amblyopia, amblyopia risk factors (reduced stereoacuity, strabismus and clinically significant refractive errors) and any other ocular problem (eg, nystagmus).
RESULTS: Overall, 26.5% of the screened children had a visual problem, including 5.9% with amblyopia. Using all five tools, screening sensitivity=84% (95% CI 78 to 89), specificity=49% (95% CI 44 to 53), PPV=37% (95% CI 33 to 42), and NPV=90% (95% CI 86 to 93). The odds of having a correct screening result in SK (mean age=68.2 months) was 1.5 times those in JK (mean age=55.6 months; 95% CI 1.1 to 2.1), with sensitivity improved to 89% (95% CI 80 to 96) and specificity improved to 57% (95% CI 50 to 64) among SK children.
CONCLUSIONS: A school-based screening programme correctly identified 84% of those kindergarten children who were found to have a visual problem by a cyclopleged optometry exam. Additional analyses revealed how accuracy varies with different combinations of screening tools and referral criteria. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Over 700 typically developing children underwent screening and cyclopleged optometry exams by a licenced optometrist.

Comparative advantages (sensitivity, specificity, positive predictive value and negative predictive value) calculated for five screening tools and all possible combinations.

Diagnostic accuracy calculated for different referral criteria for each screening tool.

Results compared for children aged 4–5 years.

Background

The early detection of amblyopia is an important public health issue because, if left untreated past about age 7 years, it leads to a lifetime of reduced vision in a structurally normal eye secondary to abnormal visual experience early in life.1–3 Amblyopia is the most common cause of blindness in one eye and increases the risk of losing the other ‘good’ eye and concomitant risk of job loss.4–7 It also leads to impaired binocular vision and reading8 9: by 8–12 years of age, children with amblyopia read more slowly than those with normal vision.10 11 The prevalence of amblyopia is estimated to be 3%–5% in children aged 3–6 years.12–16 Risk factors for amblyopia include clinically significant, untreated refractive errors, which occur in roughly 10% of children aged 3–6 years.17–22 Timely prescription of glasses can be all that is necessary to prevent amblyopia and the learning difficulties.23–29 For example, in 4-year-old children with hyperopia (far-sightedness) who wore glasses for as little as 6 weeks, IQ scores improved to a level comparable with the control group, with a significant improvement in visual motor subtest scores.30

Early visual screening is associated with a decrease in the prevalence of amblyopia and improved acuity.31–33 The US Preventive Services Task Force systematically reviewed the evidence on vision screening in 2011 and again in 2017 and recommended that children age 3–5 years be screened for vision problems.34–36 However, systematic reviews of the literature question the effectiveness of vision screening programmes.37 38The seminal study by the Vision In Preschoolers (VIP) Group in 2004 tested 11 preschool vision screening tests administered by optometrists/ophthalmologists and compared their results with those from standardised comprehensive eye examinations.19 With specificity set at 90% (in an attempt to prevent false referrals), sensitivity ranged from 16% (cover–uncover test) to 64% (non-cyloplegic retinosocopy). A follow-up study found that lay screeners were just as effective as nurse screeners in administering the screening tests, with the four best tests being non-cyloplegic retinoscopy, Retinomax Photoscreener, SureSight Vision Screener and crowded Linear Lea Symbols.39 However, since the publication of these studies, newer photoscreeners have become available (Plusoptix; Spot by Welch Allyn) that have high agreement with cyclopleged eye examinations.40 In addition, a new electronic hand-held device, the Pediatric Vision Scanner (PVS), has been developed that detects strabismus and amblyopia by measuring binocular fixation.41 It is unclear how accurately these newer screening tools can detect visual problems in a non-clinical population of children aged 3–6 years, when used in combination with traditional tests of visual acuity and stereoacuity.

The primary purpose of the present study was to conduct an updated assessment of paediatric vision screening tools in a normative sample of children. Our goal was to determine the sensitivity and specificity of the five best evidence-based screening tools available at the start of the study in 2014, by evaluating their results against a gold standard, comprehensive eye examination including cycloplegic retinoscopy by licenced optometrists. We chose to target children in junior kindergarten (JK) and senior kindergarten (SK) (aged 4–5 years) because amblyopia can be treated effectively in this age range.1–3 42

Research design and methodology

Study design

This is a prospective cohort study in which we chose the screening tools and referral criteria before participants were recruited and data collection began. The target sample size of 600 children was chosen based on reported prevalence of amblyopia of 3%–5% for children in this age range, which would yield 18–30 cases, as well as more cases of children with amblyopia risk factors.

Participants

Participants were children aged 3–6 years enrolled in a large school in Toronto, Ontario, from 2014 to 16, with 25 kindergarten classes. Many students were from immigrant families: the October 2016 student census showed that 95% of children spoke a primary language other than English at home and 37% of families had lived in Canada for fewer than 5 years. In year 1, there were 722 JK and SK children who were eligible to participate. The 529 children for whom there was informed parental consent (73%) were screened and 469 of those children subsequently received a complete gold standard eye examination with a parent or guardian present (90% of screened children). When needed, glasses were dispensed at no cost to parents, typically about 1 month following the eye examination. In year 2, we increased the sample size by inviting parents of JK and SK children who had not participated in year 1 to participate (n=410). The 283 children for whom there was informed parental consent (69%) were screened, and 243 of these children received a complete gold standard eye examination with a parent/guardian present (87% of children screened in year 2). Data for three children from year 2 were excluded because the experimenter did not record the child’s age. The final sample was 709 children (mean age=60.4 months, range=46–76 months) for whom we had complete data. For 80% of the children, the parents reported that this was their child’s first eye exam (17% reported having received an eye exam previously and 3% of parents could not remember). An additional 26 children were examined by the optometrist, but the child/parent refused cycloplegia and their data are not included in the analyses. The parents of 74 additional children consented to screening and the follow-up eye exam initially but did not attend the exams because they changed their minds (38), moved away (15), failed to arrive at the appointment (8), had scheduling conflicts (7) or were on extended vacation (6).

Screening tools

We identified five screening tools shown to be effective in detecting amblyopia and amblyopia risk factors in young children. Previous research suggests that using some combination of an acuity test, a test of binocular function (eg, stereoacuity) and an autorefractor or photoscreener is effective in detecting eye problems in young children.19 35 39 43 44 Eye charts (eg, acuity and stereoacuity tests) are not reliable until age 4 years, at which point they can identify children at risk for amblyopia but miss many children with hyperopia.45 46 Photoscreeners and autorefractors such as the Spot (formerly Pediavision, now Welch Allyn), Plusoptix, Retinomax and SureSight (Welch Allyn) have been shown to effectively detect refractive errors in children under 6 years of age.40 47 A recently developed electronic device, the PVS, is designed to directly detect strabismus and amblyopia in children.41 The order of tests was randomised. To avoid any bias, the screeners were masked to the results of other tests.

Cambridge Crowded Acuity Cards

Because amblyopia is defined as a difference in acuity between the two eyes,1 48 almost any type of vision screening includes a test of monocular visual acuity.19 35 39 Various acuity tests (eg, Lea symbols, HOTV and Cambridge Crowded Acuity cards) have been used with young children, but their sensitivity and specificity differ depending on the study sample. Nevertheless, there is a consensus that crowded acuity charts are more accurate than those that use single optotypes,43 49 and some evidence suggests that crowded pictures may not be as effective in identifying amblyopia as crowded letters.50 All tests designed for young children use matching as a response measure because some children cannot yet name letters (or even some symbols). Therefore, we chose the Cambridge Crowded Acuity Cards, which use letters that are crowded by encircling letters and a matching card, tested at a distance of 3 metres. It has been used successfully with children aged 3–7 years in previous studies.51 52 The screener began by playing the game binocularly at near, then covered the randomly chosen eye with glasses and moved back to 3 m. Each eye was tested in turn while the fellow eye was occluded by glasses. Testing both eyes typically took 5–10 min for each child.

Plusoptix S12 (software version 6.1.10.0) and Spot (Welch Allyn; software version 2.1.4) photoscreeners

Sensitivity and specificity differ across studies, but photoscreeners have high agreement with cycloplegic retinoscopy, although they perform better in detecting astigmatism, myopia and anisometropia than hyperopia.40 47 The Plusoptix S12 and Spot (Welch Allyn) photoscreeners, both of which were commercially available at the start of this study in 2014, have been shown to be particularly effective.40 47 53–56 In one study, conducted in a paediatric ophthalmology office, Plusoptix S04 showed a sensitivity of 99% and specificity of 82%.54 In another study, the Spot had a sensitivity of 87% and specificity of 74% for detecting clinically significant refractive errors.56 One study suggested that the Plusoptix was more accurate, but the Spot provided a reading for more children than the Plusoptix (ie, better testability),40 so we chose to include both in our screening battery. Testing with each photoscreener took 1–2 min per child.

Randot preschool stereoacuity test

Binocular function is degraded by amblyopia or eye misalignment (a risk factor for amblyopia)1 and that deficit can be manifested as reduced stereopsis. The Randot Preschool Stereoacuity Test is a widely used test that is adapted for testing children. A large-scale study of over 5000 children aged 3–18 years has provided good normative data,57 and interobserver test–retest reliability is high.58 A VIP study with lay screeners found that including a test of stereoacuity increases the ability to detect amblyopia risk factors such as strabismus.39 Testing took 2–4 min per child.

The PVS

The PVS is a new device that detects strabismus and amblyopia directly. It uses binocular retinal birefringence scanning to automatically detect strabismus as small as 1 prism dioptre (D). When tested against the presence of amblyopia and/or strabismus, the PVS demonstrated sensitivity and specificity 90% or better in previous published studies.41 59 PVS holds promise in detecting strabismus, amblyopia or any other medical cause of reduced three-dimensional depth perception, while minimising false referrals from moderate levels of refractive errors that do not require treatment. The PVS has not been evaluated in any large-scale study for use in a vision screening programme aimed at detecting any vision problem in a population-based sample or in combination with other tests. Testing took 1–2 min per child.

Classification of vision screening test results

Children who met the referral criteria for any test (table 1) were considered a ‘refer’, whereas only children who passed all five tests were considered a ‘pass’. Referral criteria for visual acuity and refractive errors were based on the 2013 American Association for Pediatric Ophthalmology and Strabismus (AAPOS) vision screening guidelines.60 The preprogrammed referral thresholds on the Plusoptix did not match these AAPOS guidelines, and thus screeners were trained to manually make the decision of pass/refer based on the refractive error values. The referral criterion for visual acuity for children <48 months were made more lenient based on the recommended guidelines and normative data in the literature.44 51 61 The referral criterion for the Randot Preschool Stereoacuity Test was based on previously published normative data.57 For the PVS, we followed the manufacturers’ guidelines (failure to fixate binocularly on at least 60% of 10 attempted measurements).41 Children who did not understand a test or who were uncooperative were considered a ‘refer’, as previous research suggests that these children are at higher risk of having a vision disorder.62

Table 1

Screening referral criteria

Screening tool	Referral criterion
	31–48 months	>48 months
Far acuity	<0.3 logMAR*	<0.2 logMAR
Stereoacuity	>100 arcsec	>100 arcsec
PVS	<0.6 Binocularity	<0.6 Binocularity
Photoscreeners	Hyperopia >4.0D in any meridianMyopia <−3.0D in any meridianAstigmatism >2.0DAnisometropia >2.0D	Hyperopia >3.5D in any meridianMyopia <−1.5 in any meridianAstigmatism >1.5DAnisometropia >1.5D

*Children aged 48 months were tested against the <0.2 logMAR criterion for acuity.

PVS, Pediatric Vision Scanner.

Gold standard eye examinations

Each child received a gold standard comprehensive eye examination, including cycloplegic refraction, by a licensed optometrist. The exams were typically conducted 1 month after screening, and the optometrists were masked to the screening results of each child. An examination lane was set up at the school, and examinations occurred with parents/guardians present. The following tests were performed: monocular visual acuity at near and far (HOTV or Snellen letters, or Lea picture symbols), colour vision (Hardy-Rand-Rittler Test), ocular motility (versions and ductions), ocular alignment (cover test), binocular function (Preschool Randot or Titmus stereo test, motor fusion), anterior segment (using a slit lamp), cycloplegic refraction (cyclopentolate 1% and phenylephrine 2.5%) and dilated fundus examination. Glasses were provided at no cost to the family when prescribed.

Definition of amblyopia and clinically significant refractive errors

The definition of amblyopia and its risk factors, including clinically significant refractive errors, are shown in table 2, and are based on the guidelines of the American Academy of Ophthalmology and AAPOS for children aged 31–72 months.1 60 Optometrists made clinical judgments about whether a child should receive treatment (eg, glasses) independently from these research definitions. Prescribing practices vary among clinicians, and thus ‘number of glasses prescribed’ was not an a priori variable of interest in our study.

Table 2

Definitions of amblyopia and amblyopia risk factors

Disorder	Definition
Amblyopia
1.Unilateral	≥2 line difference in best corrected acuity and worse than 20/40 in any eye.
2.Bilateral	Worse than 20/40 in the better-seeing eye.
Amblyopia risk factors
1.Strabismus	Tropias >10D
2.Reduced stereoacuity	>100 arcsec
3.Refractive errors	As defined below
a.Hyperopia	>3.5D
b.Myopia	<−1.5D
c.Astigmatism	>1.5D
d.Anisometropia	>1.5D

D, dioptre.

Data analyses

Data from 709 children for whom there were complete gold standard eye examination results were used to determine the prevalence of vision problems, as well as the diagnostic capability of the five screening tools. Sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) were computed by comparing the screening diagnosis (‘pass’ vs ‘refer’) to the findings from the comprehensive eye examination (ie, problem present vs not present). These measures were computed for each screening tool independently as well as for all possible combinations, along with 95% CIs, using a bootstrapping method that randomly sampled two-thirds of the data 2000 times. We also conducted an exploratory analysis by splitting the outcomes into JK versus SK children. In addition, we conducted ROC analyses using SPSS software (V.25) for each screening tool as an added measure of diagnostic accuracy, as well as to evaluate the appropriateness of the referral criteria.

Patient and public involvement

A researcher from the Toronto District School Board (TDSB), the principal and vice principal at the study site and the central principals of the Model Schools for Inner Cities (MSIC) programme of the TDSB (a programme that services low-income neighbourhoods) advised on the design of the study, recruitment and consent material for parents and logistics of running the study. Parents of eligible kindergarten children were consulted at a beginning-of-school-year event. A summary of the findings was shared with the principal and vice principal at the study site, lead principals of the MSIC and the TDSB research ethics board. The full research report, as well as general information relevant to parents, have been made available to the public via our website: www.visionscreening.ca

Results

Of the 709 children for whom there were complete screening and gold standard eye exam data, 26.5% had a vision problem as defined in table 2. The prevalence of each type of problem is shown in table 3, with 5.9% of children having amblyopia and 24.0% having an amblyopia risk factor, the most common of which was astigmatism (14.2%). The ‘other’ problems (0.1%) consisted of four cases of nystagmus and a single case of optic nerve abnormality. table 4 shows the screening results (from all five tools) against the results of the comprehensive eye examination.

Table 3

Prevalence of eye problems

Disorder	n (% screened)
Any problem	188 (26.5)
Amblyopia	42 (5.9)
Unilateral	24 (3.4)
Bilateral	18 (2.5)
Amblyopia risk factors	170 (24.0)
Strabismus	28 (3.9)
Reduced stereoacuity	35 (4.9)
Hyperopia	38 (5.4)
Myopia	10 (1.4)
Astigmatism	101 (14.2)
Anisometropia	8 (1.1)
Other problems	5 (0.1)

If a child had more than one problem, he/she was counted repeatedly in each applicable category (26 children had both amblyopia and a risk factor or other problem).

Table 4

Results obtained from the five screening tools compared to the results of the comprehensive eye examination with cycloplegic refraction by a licenced optometrist

Screening	Comprehensive eye examination						Grand total
	Junior kindergarten			Senior kindergarten
	Any problem	No problem	Total	Any problem	No problem	Total
Refer	100	179	279	58	87	145	285
Pass	23	140	163	7	115	122	424
Total	123	319	442	65	202	267	709

Table 5 shows the sensitivity, specificity, PPV and NPV for detecting a vision problem for all five tools combined, as well as for each tool separately. Because a child was considered ‘refer’ if he or she failed any one of the tests, sensitivity was highest (84%, 95% CI 78 to 89) but specificity was lowest (49%, 95% CI 44 to 53) when all five tools were combined. Individually, the Plusoptix had the highest sensitivity (64%, 95% CI 57 to 71) and the Spot had the highest specificity (93%, 95% CI 91 to 95).

Table 5

Diagnostic capabilities of the screening tools for detecting any vision problem (95% CIs based on bootstrapping are shown in parentheses)

Screening tool	Sensitivity	Specificity	PPV	NPV
All five tools	0.84 (0.78 to 0.89)	0.50 (0.44 to 0.53)	0.37 (0.33 to 0.42)	0.90 (0.86 to 0.93)
Cambridge crowded acuity	0.59 (0.51 to 0.65)	0.73 (0.69 to 0.76)	0.44 (0.38 to 0.50)	0.83 (0.79 to 0.86)
Randot preschool	0.33 (0.27 to 0.40)	0.90 (0.88 to 0.93)	0.55 (0.46 to 0.64)	0.79 (0.75 to 0.82)
Plusoptix photoscreener	0.64 (0.57 to 0.71)	0.88 (0.84 to 0.90)	0.65 (0.58 to 0.71)	0.87 (0.84 to 0.90)
Spot photoscreener	0.60 (0.53 to 0.66)	0.93 (0.91 to 0.95)	0.76 (0.68 to 0.83)	0.86 (0.84 to 0.89)
Paediatric vision scanner	0.41 (0.33 to 0.47)	0.77 (0.73 to 0.80)	0.38 (0.32 to 0.45)	0.78 (0.74 to 0.81)

NPV, negative predictive value; PPV, positive predictive value.

As an exploratory analysis, we also conducted logistic regression analyses to determine whether the accuracy of the screening tools differed when used with children in JK (mean age=55.6 months; range=46–64 months) versus SK (mean age=68.2 months; range=53–76 months). The estimated odds of having a correct screening result in SK were 1.52 times (95% CI 1.11 to 2.08) those in JK, suggesting that screening was more accurate with SK children. Sensitivity and specificity for the two age groups are shown separately in table 6.

Table 6

Diagnostic capabilities of the screening tools for detecting any vision problem for children in junior kindergarten (JK) and senior kindergarten (SK)

Screening tool	Sensitivity		Specificity
Grade	JK	SK	JK	SK
All five tools	0.81 (0.73 to 0.88)	0.89 (0.80 to 0.96)	0.44 (0.39 to 0.50)	0.57 (0.50 to 0.64)
Cambridge Crowded Acuity	0.59 (0.50 to 0.68)	0.57 (0.45 to 0.69)	0.68 (0.63 to 0.73)	0.81 (0.75 to 0.86)
Randot Preschool	0.37 (0.28 to 0.45)	0.26 (0.16 to 0.37)	0.88 (0.84 to 0.92)	0.93 (0.89 to 0.96)
Plusoptix Photoscreener	0.60 (0.51 to 0.69)	0.71 (0.58 to 0.81)	0.86 (0.82 to 0.90)	0.90 (0.84 to 0.94)
Spot Photoscreener	0.55 (0.46 to 0.64)	0.68 (0.55 to 0.78)	0.93 (0.89 to 0.95)	0.95 (0.90 to 0.97)
Pediatric Vision Scanner	0.37 (0.29 to 0.46)	0.48 (0.36 to 0.60)	0.74 (0.70 to 0.79)	0.80 (0.74 to 0.85)

95% CIs based on bootstrapping are shown in parentheses.

Additionally, we compared bootstrapped estimates of sensitivity, specificity, PPV and NPV for all possible combinations of the screening tools. The results are summarised in table 7. Understanding how sensitivity and specificity differ with the addition or deletion of a screening tool has important consequences for policy makers, because using fewer tests without significantly lowering sensitivity and specificity makes most economic sense. Our analyses reveal that the combination of the three tools of Cambridge Crowded Acuity Cards, Randot Preschool stereotest and Plusoptix had a sensitivity of 81% and specificity of 64%, which is comparable with the results using four or five tools, suggesting that such a combination would be ideal to keep costs down. As would be expected, using only the PVS and Randot had the worst sensitivity of 56% because neither test is designed to effectively detect refractive errors. The Plusoptix and Spot seem to perform similarly, although using both improved sensitivity from 60% (Spot only) and 64% (Plusoptix only) to 68% while maintaining specificity at 85%, indicating that the two photoscreeners do not always detect the same problems. A direct comparison of the performance of the Plusoptix and Spot revealed that the two photoscreeners agreed in 89% (633/709) of cases. Of the 57 cases in which the Plusoptix referred but the Spot passed a child, 16 had a problem (2.3% of screened children had a problem missed by the Spot), whereas 41 did not have a problem (Plusoptix had over-referred). Of the 19 cases in which the Spot had referred but the Plusoptix passed a child, 6 had a problem (0.8% of children screened had a problem missed by the Plusoptix) and 13 did not have a problem (Spot had over-referred).

Table 7

Diagnostic capabilities of different combinations of screening tools for detecting any vision problem (95% CIs are shown in parentheses)

Combinations of screening tools	Sensitivity	Specificity	PPV	NPV
2 tools
Acuity and Plusoptix	0.76 (0.70 to 0.82)	0.68 (0.64 to 0.72)	0.46 (0.41 to 0.52)	0.89 (0.85 to 0.92)
Acuity and Spot	0.73 (0.67 to 0.79)	0.71 (0.67 to 0.74)	0.48 (0.41 to 0.53)	0.88 (0.85 to 0.91)
Acuity and Randot	0.67 (0.60 to 0.72)	0.69 (0.64 to 0.72)	0.43 (0.37 to 0.49)	0.85 (0.81 to 0.88)
Acuity and PVS	0.70 (0.63 to 0.77)	0.56 (0.52 to 0.61)	0.37 (0.32 to 0.42)	0.84 (0.80 to 0.88)
Plusoptix and PVS	0.73 (0.67 to 0.80)	0.67 (0.63 to 0.71)	0.45 (0.39 to 0.51)	0.88 (0.84 to 0.91)
Plusoptix and Randot	0.72 (0.65 to 0.78)	0.80 (0.77 to 0.84)	0.57 (0.50 to 0.63)	0.89 (0.85 to 0.92)
Plusoptix and Spot	0.68 (0.61 to 0.74)	0.85 (0.82 to 0.88)	0.63 (0.56 to 0.69)	0.88 (0.85 to 0.91)
Spot and Randot	0.68 (0.61 to 0.74)	0.85 (0.82 to 0.88)	0.63 (0.55 to 0.69)	0.88 (0.85 to 0.91)
Spot and PVS	0.69 (0.63 to 0.76)	0.71 (0.67 to 0.75)	0.47 (0.40 to 0.52)	0.87 (0.83 to 0.90)
Randot and PVS	0.56 (0.49 to 0.63)	0.69 (0.65 to 0.73)	0.40 (0.34 to 0.46)	0.81 (0.78 to 0.85)
3 tools
Acuity and Plusoptix and Spot	0.78 (0.71 to 0.83)	0.67 (0.63 to 0.71)	0.46 (0.41 to 0.51)	0.89 (0.86 to 0.92)
Acuity and Plusoptix and Randot	0.81 (0.75 to 0.86)	0.64 (0.60 to 0.68)	0.45 (0.39 to 0.50)	0.90 (0.87 to 0.93)
Acuity and Plusoptix and PVS	0.81 (0.75 to 0.87)	0.53 (0.48 to 0.57)	0.38 (0.34 to 0.43)	0.88 (0.84 to 0.92)
Acuity and Spot and Randot	0.77 (0.70 to 0.83)	0.66 (0.62 to 0.70)	0.45 (0.40 to 0.51)	0.89 (0.86 to 0.92)
Acuity and Spot and PVS	0.79 (0.73 to 0.84)	0.55 (0.50 to 0.59)	0.39 (0.34 to 0.43)	0.88 (0.84 to 0.91)
Acuity and Randot and PVS	0.74 (0.68 to 0.80)	0.53 (0.48 to 0.57)	0.36 (0.31 to 0.41)	0.85 (0.81 to 0.89)
Plusoptix and Spot and Randot	0.75 (0.67 to 0.80)	0.79 (0.75 to 0.82)	0.55 (0.49 to 0.62)	0.90 (0.86 to 0.92)
Plusoptix and Spot and PVS	0.76 (0.68 to 0.81)	0.65 (0.61 to 0.69)	0.44 (0.39 to 0.50)	0.88 (0.84 to 0.91)
Plusoptix and Randot and PVS	0.79 (0.72 to 0.84)	0.61 (0.57 to 0.65)	0.42 (0.36 to 0.47)	0.89 (0.85 to 0.92)
Spot and Randot and PVS	0.75 (0.68 to 0.81)	0.65 (0.61 to 0.69)	0.44 (0.38 to 0.50)	0.88 (0.84 to 0.91)
4 tools
Acuity and Plusoptix and Randot and PVS	0.83 (0.78 to 0.88)	0.49 (0.45 to 0.53)	0.37 (0.33 to 0.42)	0.89 (0.85 to 0.92)
Acuity and Plusoptix and Spot and Randot	0.81 (0.74 to 0.86)	0.64 (0.59 to 0.67)	0.45 (0.39 to 0.50)	0.91 (0.87 to 0.93)
Acuity and Plusoptix and Spot and PVS	0.82 (0.76 to 0.87)	0.52 (0.48 to 0.57)	0.38 (0.34 to 0.43)	0.89 (0.85 to 0.92)
Acuity and Spot and Randot and PVS	0.82 (0.75 to 0.87)	0.51 (0.47 to 0.55)	0.37 (0.33 to 0.43)	0.88 (0.84 to 0.91)
Plusoptix and Spot and Randot and PVS	0.80 (0.74 to 0.85)	0.60 (0.56 to 0.64)	0.42 (0.36 to 0.46)	0.89 (0.86 to 0.92)

PVS, pediatric vision scanner.

Testability was high for all tools: Cambridge Crowded Acuity 97.5%, Plusoptix 94.8%, Spot 99.5%, Randot 97.9% and PVS 98.6%. While the overall prevalence of having a visual problem was 26.5%, there was a higher proportion of children with a visual problem among the children who were unable to complete a test: Cambridge Crowded Acuity 33.3%, Randot 40.0%, Plusoptix 64.9%, Spot 50.0% and PVS 40.0%. These results suggest that screening programmes should refer any child who is unable to complete a test for a comprehensive eye examination.

We conducted ROC analyses based on the results for each screening tool tested against the presence/absence of any visual problem and also for specific target disorders that the screening tool was designed to detect. The area under the curve (AUC) provides a measure for comparing the diagnostic accuracies of the screening tools, and these values are summarised in table 8 (children who were not able to complete a test were excluded from these analyses). The highest AUC value was found for the spherical equivalent (SE) values from the Spot tested against the presence of hyperopia and the ROC is shown in figure 1. The AUC values provide converging evidence that overall, the two photoscreeners had the highest accuracy, although this is likely because they are designed to detect astigmatism (by the cylinder values), the most common disorder found in our sample. Acuity had better diagnostic power than the Randot and the PVS, likely because they were designed to detect amblyopia and binocular vision disorders, not refractive errors such as astigmatism. This also explains the low AUC value for the SE from the photoscreeners. When tested against the target disorder for which they were designed (SE from photoscreeners against hyperopia and PVS against strabismus), the AUC values are higher. There were too few cases of myopia and anisometropia to be considered separately.

Table 8

Area under the curve (AUC) values from ROC analyses of each screening tool

Screening tool	Target disorder	n	AUC	95% CI	Significance
Acuity	Any problem	691	0.685	0.637 to 0.733	p<0.001
Plusoptix – cylinder	Any problem	672	0.822	0.784 to 0.860	p<0.001
Plusoptix – SE	Any problem	672	0.507	0.448 to 0.565	p=0.80
Spot – cylinder	Any problem	707	0.836	0.799 to 0.874	p<0.001
Spot – SE	Any problem	707	0.537	0.482 to 0.592	p=0.13
Randot	Any problem	694	0.615	0.564 to 0.666	p<0.001
PVS	Any problem	699	0.610	0.562 to 0.658	p<0.001
Plusoptix – SE	Hyperopia	672	0.856	0.790 to 0.923	p<0.001
Spot – SE	Hyperopia	707	0.886	0.817 to 0.954	p<0.001
PVS	Strabismus	699	0.654	0.554 to 0.754	p<0.01

PVS, pediatric vision scanner; SE, spherical equivalent.

Figure 1

ROC of spot SE values tested against the presence of hyperopia as determined in a gold standard eye exam by a licenced optometrist (AUC=0.89). The curved line shows how sensitivity and specificity varies with different SE values (specificity is plotted as 1 – specificity on the x-axis). The diagonal line represents a test with no diagnostic accuracy (ie, 50% chance that the test correctly diagnosed the disease); the extent to which the curve deviates from the diagonal indicates the accuracy of the test and is quantified as area under the curve (AUC). SE, spherical equivalent.

Based on these ROC analyses, table 9 provides sensitivity and specificity values for detecting any visual disorder using different referral cut-offs for each tool. Together with tables 6 and 7, these tables provide a reference for choosing the appropriate screening tool(s) and referral criteria depending on the availability of resources (ie, whether the emphasis is on not missing visual problems or not over-referring for unnecessary eye exams). We also provide sensitivity and specificity values for the photoscreeners and PVS for detecting the specific disorders that they were designed to detect in online supplementary table 1. In regions/countries where there is universal healthcare, it may be beneficial to optimise sensitivity, even at the cost of lower specificity, so that more children enter into optometry care and few problems are missed. In contrast, in regions where resources are scarce, it may be more cost effective to target high specificity to minimise false referrals.

Table 9

Sensitivity and specificity values at different referral cut-offs for each screening tool

Test	Refer if worse than or equal to	Sensitivity	Specificity
Acuity	6/9	1.0	0
	6/12	0.60	0.72
	6/18	0.33	0.92
	6/24	0.08	0.99
	<6/24	0.02	1.0
Plusoptix (cylinder)	0.50	0.94	0.27
	0.75	0.89	0.49
	1.0	0.80	0.68
	1.25	0.74	0.77
	1.5	0.69	0.85
	1.75	0.59	0.90
	2.0	0.54	0.92
Spot (cylinder)	0.50	0.97	0.16
	0.75	0.90	0.44
	1.0	0.82	0.68
	1.25	0.72	0.81
	1.5	0.64	0.89
	1.75	0.56	0.93
	2.0	0.48	0.97
Randot	100	1.0	0
	200	0.31	0.92
	400	0.15	0.97
	800	0	1.0
PVS	1.0	1.0	0
	0.8	0.73	0.42
	0.6	0.55	0.60
	0.4	0.39	0.78
	0.2	0.35	0.82

PVS, pediatric vision scanner.

Discussion

The goal of the current study was to assess the diagnostic capabilities of different screening tools that are appropriate for typically developing kindergarten children. We found that 5.9% of children had amblyopia and 24.0% had amblyopia risk factors. These rates are similar to those from a previous screening programme conducted in Toronto that included over 10 000 students in the early grades (although no examinations were conducted on children who passed screening),63 as well as other studies conducted in Newfoundland (Canada), USA, Australia, Singapore and China.12-22 Overall, the current battery revealed a sensitivity of 84% but a specificity of 49%. The results indicate that by using all five tools, vision problems are not likely to be missed but roughly half of the children referred for a full eye examination would not have an eye problem. Importantly, parents reported that for 80% of children in our sample, this was their first optometry exam. Without such a school-based screening programme, the children with amblyopia and its risk factors might not have been identified until it was too late for treatment to be effective.

The optimal set of screening tools will depend on available resources as well as the social and political climate of a given jurisdiction. Adding more tests will result in fewer missed problems (higher sensitivity) but more over-referrals (lower specificity), which can overburden clinicians unnecessarily where access to eye care may be difficult. For example, the VIP study in the USA examined how sensitivity of the screening tests changed with specificity targeted at 90%19 39 and 94%64 to prevent unnecessary referrals. In our data, both the Plusoptix and Spot photoscreeners had high specificity, above 90%, for SK children and reasonable sensitivity (around 70%). This finding, in part, may be because our sample had a high prevalence of astigmatism, which photoscreeners detect more accurately than they do hyperopia or myopia.40 47 53 Consistent with the findings from the autorefractor/photoscreener used in phase 1 of the VIP study,19 39 there appears to be little advantage to using both the Plusoptix and the Spot. Adding the PVS to detect amblyopia and strabismus appears to improve sensitivity. We used the Cambridge Crowded Acuity Cards in our current study, whereas other preschool screening studies have used HOTV or Lea symbols. We do not expect the results to differ depending on the type of acuity test used, as long as an age-appropriate crowded acuity test is used that uses matching and presents each letter/symbol individually (eg, Cambridge Crowding Cards, HOTV Hand-Held 50% Crowded Book and Lea Symbols Crowded Symbol Book), because single-letter acuity tests without crowding are less sensitive in identifying individuals with amblyopia. The bootstrapping analyses indicate that the combined tools are more accurate in detecting vision problems in senior kindergarten children (5 year olds) than in JK children (4 year olds). Importantly, we provide sensitivity and specificity values for varying referral cut-offs from the ROC analyses so that policy makers can make their own decisions for their screening programme about the age at which to implement, which tools to use and which cutoffs to use for referrals.

In pilot work, we found that most children failed the two behavioural tests (visual acuity and stereoacuity) if we started testing at the referral threshold. Thus, we chose to begin with the easiest levels (largest letters of 6/60 for acuity testing and largest discrepancy of 800 arcsec for stereoacuity) and to work towards more difficult levels until children made mistakes. We believe that this is an important strategy for school-based vision screening so that the screeners have time to build rapport with the children and to engage the child with the ‘games’. Although only two of the five screening tests required a verbal response, it is possible that some children failed more from fatigue than an actual visual problem as they went through the later tests. The order of tests was counterbalanced, and there was no systematic effect of order on referral rate for any test, with referral rates ranging from 2% (children who completed Randot second) to 8% (children who completed acuity as their first test and as their fourth test). However, fatigue may account for the lower accuracy of the screening tools in JK compared with SK children, a result suggesting that fewer tests may be advantageous for that age group and not just from a cost-savings perspective.

One limitation of our study is that we may have overestimated the prevalence of amblyopia (5.9%) because it sometimes resolves quickly after spectacle correction. Although acuity was remeasured at the time the child received the glasses, follow-up assessments were not made. Had they been possible, the prevalence of amblyopia might have been lower. Nevertheless, those cases would still have been included in the count of children identified as having eye problems requiring treatment. Another limitation is the lack of demographic data that might help explain the high percentage of screened children (26.5%) found to have a visual problem. The school serves an immigrant community, and it is known that race and ethnicity affect prevalence rates of refractive errors. For example, astigmatism is more common among Native American, African American and Hispanic children compared with non-Hispanic white children.65 66 Finally, as one would expect from a sample of children aged 4 and 5 years, we had too few cases of myopia and anisometropia to conduct meaningful ROC analyses.

In summary, the current study showed that conducting vision screening and follow-up gold standard eye examinations can effectively detect eye problems in children before they enter grade 1. Our screening tools consisting of crowded visual acuity, photoscreeners, stereoacuity and a device measuring eye alignment, correctly referred 84% of children aged 3–6 who had a vision problem (89% for those 5–6). These results support implementation of a similar vision screening programme in or before senior kindergarten, in communities where none exists. Future research should include a health economic analysis of the costs and benefits associated with such a screening programme.