Economic evaluations of vision screening to detect amblyopia and refractive errors in children: a systematic review.

OBJECTIVE: To synthesize and appraise economic evaluations of vision screening to detect vision impairment in children.
METHODS: Literature searches were conducted on seven electronic databases, grey literature, and websites of agencies conducting health technology assessments. Studies were included if they (1) were full, comparative economic evaluations that used cost-utility, cost-benefit, cost-effectiveness, cost-consequence, or cost-analysis methods; (2) described screening services designed to detect amblyopia, strabismus, or uncorrected refractive errors in children under 6 years of age; and (3) published after 1994. High-quality studies were synthesized descriptively. Currencies were reported in 2019 Canadian dollars. Quality was assessed with the Pediatric Quality Appraisal Questionnaire (PQAQ).
RESULTS: Vision screening services were conducted by paid staff, volunteers, or health care professionals in schools or clinics. Thirteen studies were published from five countries: China (n = 1), United States (n = 4), United Kingdom (n = 1), Canada (n = 1), and Germany (n = 6). Analytical techniques included cost-utility/cost-effectiveness combination (n = 2), cost-effectiveness analysis (n = 7), cost-utility analysis (n = 1), cost-benefit analysis (n = 1), cost-consequence analysis (n = 1), and cost analysis (n = 1). Incremental cost-effectiveness ratios ranged from C$1,056 to C$151,274 per additional case detected/prevented and from C$9,429 to C$30,254,703 per additional QALY gained, depending on the type of screening service and comparator. Six studies were determined to be of high quality.
CONCLUSION: Vision screening to detect amblyopia for young children may be cost-effective compared with no screening if amblyopia reduced quality of life. Studies varied significantly in the type of screening services and comparators used. Methodological limitations were common. Future studies would be aided immensely by prospective studies on the impact of amblyopia on the health-related quality of life of young children and guidelines on the effective conduct of vision screening.

Introduction

Amblyopia and refractive errors are the most common vision impairments affecting children worldwide, with a prevalence of 3–5% for amblyopia and 10% for refractive errors (Drover et al., 2008; Pai et al., 2012; Pascolini & Mariotti, 2012). These conditions may affect quality of life and add financial burden (Kandel et al., 2017; Langelaan et al., 2007; Resnikoff et al., 2008; Saw et al., 2007).

To enable the early detection of vision impairment from amblyopia and refractive errors, recommendations for routine vision screening and comprehensive eye exams (CEEs) are common across industrialized nations. In Canada and the United States, routine CEEs and vision screening are recommended by professional pediatric and optometry associations at similar regular intervals throughout childhood (American Optometric Association, 2021; Amit, 2009; Canadian Paediatric Society, 2018; Committee on Practice & Ambulatory Medicine Section on Ophthalmology, 2003). In Australia, health departments in each state and territory have their guidelines regarding vision screening for children (Murdoch Childrens Research Institute, 2009). Vision screening may be required for children turning 4 years of age as part of a health check (Services Australia, n.d). In the United Kingdom, vision screening is recommended by the United Kingdom National Screening Committee in schools for children aged 4 to 5 years (Public Health England, 2017).

In Ontario, Canada, annual CEEs are performed primarily by optometrists and ophthalmologists, and are paid for through the Ontario Health Insurance Program (OHIP), for children and seniors who are legal residents of the province. The Eye See… Eye Learn program, funded by the Ontario Association of Optometrists and the Ontario Ministry of Health (MOH), provides free prescription glasses after a CEE by a participating optometrist to 4-year-old children needing them (Ontario Association of Optometrists, 2015, 2017). In the 2016/2017 academic year, the Eye See… Eye Learn program reported only a 20% participation among eligible children in junior kindergarten (~ aged 4 years). Of these, 47% were first-time eye exams (Ontario Association of Optometrists, 2017). These numbers suggest that children with visual problems may not be identified in a timely manner. To improve early detection, the Ontario Ministry of Health introduced a requirement for universal vision screening programs into the Ontario Public Health Standards in 2017. The requirement stipulates the provision of vision screening by public health units in kindergartens of public schools across Ontario (Ministry of Health and Long-Term Care, 2017), but its cost-effectiveness has not been evaluated.

Several health units in Ontario have faced challenges with the rollout of school-based screening because of rising health care costs and provincial budgeting constraints further exacerbated by the COVID-19 pandemic (Personal Communication). Evidence on the cost-effectiveness of vision screening in the local context will enable health units to make informed decisions on delivering effective vision screening programs without sacrificing other important health programs.

In the face of limited resources and the growing demand on health care spending, policy makers have been increasingly drawn to economic evaluations to support decisions about resource allocation (National Institute for Health & Care Excellence, 2014). To inform the structure and model inputs of economic evaluations, systematic reviews are recommended as best practice in guidelines for health technology assessment (Akers et al., 2009). The purpose of this study is to synthesize and appraise economic evaluations of vision screening to detect amblyopia and refractive errors in children under 6 years of age. This review will inform the design and conduct of a future economic evaluation of vision screening programs in Ontario, Canada.

Methods

Data sources and search strategy

A literature search was carried out using seven electronic databases: MEDLINE (Ovid, PubMed, and Medline in Process), EMBASE (Ovid), The Cochrane Library, the Cost-effectiveness analysis Registry (CEA), Global Health CEA Registry (GHCEA), Paediatric Economic Database Evaluation (PEDE), and EconLit (EBSCO). The following grey literature sources were also searched: Programs for Assessment of Technology in Health (PATH), International Network of Agencies for Health Technology Assessment (INAHTA), Ontario Health Technology Advisory Committee (OHTAC), Health Economics Research Centre (HERC), and ProQuest Dissertations and Theses Global. The websites of agencies that routinely conduct health technology assessments included in the search were as follows: Canadian Agency of Drugs and Technologies in Health (CADTH), National Institute for Health and Care Excellence (NICE) Evidence search and Guidelines, and the European Network for Health Technology Assessment (EUnetHTA). Reference lists of key articles were searched. Citation tracking using the Web of Science database and personal knowledge skills were also employed.

Appropriate search strategies were developed for each database using text words and subject headings for the target disorders (amblyopia, strabismus, and refractive error), service (vision screening), and study type (cost-utility analysis, cost-effectiveness analysis, cost-consequence analysis, cost analysis, cost minimization, and cost-benefit analysis methods). The search strategy was supplemented with validated search filters from the InterTASC Information Specialists’ Sub-Group for economic evaluations and validated using The Peer Review of Electronic Search Strategy (PRESS) checklist (ISSG Search Filter Resource n.d.; McGowan et al., 2016).

Study selection

A study was included if it: (1) was a full, comparative economic evaluation; (2) used any one of the analytic methods (cost-utility analysis, cost-benefit analysis, cost-effectiveness analysis, cost-minimization analysis, cost-consequence analysis, and cost-analysis); and (3) evaluated screening services for children under the age of 6 years to detect amblyopia, strabismus (as a risk factor for amblyopia), and/or uncorrected refractive errors. A study was excluded if: (1) the full text was unavailable; (2) it was a review, commentary, case series, case report, editorial, letter, or conference abstract; or (3) it was published before 1995. Studies before 1995 were excluded because in that year, the first instrument-based screening tools—the Medical Technology and Innovations (MTI) Photoscreener and Nikon Retinomax K-Plus Autorefractor (Nikon Corp, Melville, New York, USA)—became commercially available (Ottar et al., 1995; Silverstein & Donahue, 2018), which revolutionized paediatric vision screening. It was anticipated that studies published after that year would include the newer screening technology to facilitate comparisons. Studies using traditional screening tools were included as they may still be relevant in some jurisdictions. No country or language restrictions were applied. Two reviewers (AA and YK) assessed the identified studies independently. Disagreements were discussed to achieve consensus. A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart was developed (Moher et al., 2009).

Data extraction and synthesis

Key aspects of studies were extracted, and results presented in summary tables and text. Currencies reported were converted into Canadian dollars for the year of pricing (or year of study conduct or publication if year of pricing was not reported). Bank of Canada annual average exchange rates were used (Bank of Canada, 2017; Cochrane Handbook for Systematic Reviews of Interventions, 2011; Statistics Canada, 2018, 2020). Prices were then inflated to 2019 Canadian dollars using consumer price indices (CPI) for eye care services from Statistics Canada (Statistics Canada, 2018, 2020). For studies before 2008, health care services CPI were applied because CPI were not reported for eye care services until 2008.

Quality assessment and risk of bias

The quality of the included studies was assessed with the Pediatric Quality Appraisal Questionnaire (PQAQ), a comprehensive instrument demonstrating face and content validity, and strong interrater and test–retest reliability in the appraisal of pediatric economic evaluations (Ungar & Santos, 2003). It is made up of 57 items in 14 domains: (1) Economic evaluation, (2) Comparators, (3) Target population, (4) Time horizon, (5) Perspective, (6) Costs and resource use, (7) Outcomes, (8) Quality of life, (9) Analysis, (10) Discounting, (11) Incremental analysis, (12) Sensitivity analysis, (13) Conflict of interest, and (14) Conclusions. Forty-six of the 57 items can be scored to rate the quality of studies. Items were scored independently between 0 and 1. An unweighted mean was calculated for each domain using all scorable items within the domain. The domain scores ranged between 0 and 1, with higher scores indicating better quality. Two reviewers (AA and YK) conducted quality assessments independently, and discrepancies were resolved through discussion.

Results

Literature sample

The initial search was conducted on July 13, 2018. A total of 671 publications were identified, of which 13 met the inclusion criteria (Arnold et al., 2005; Carlton et al., 2008; Drover, 2006; Gandjour et al., 2003; Joish et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2000, 2002; Miller et al., 2003; Rein et al., 2012; Schlichtherle et al., 2000; Wang et al., 2019). None of the studies published before 1995 met the study inclusion criteria. Figure 1 is a PRISMA flowchart describing the process for selecting studies and the reasons for exclusion (Moher et al., 2009). The search strategy developed for Medline is provided in Appendix 1.

Fig. 1

PRISMA flow diagram

Quality appraisal

The mean domain quality scores for each included study are summarized in Table 1, with details in Appendices 2 and 3. Key characteristics of all 13 studies are summarized in Table 2. Tables 3 and 4 provide characteristics and outcomes of high-quality studies. Appendices 4 and 5 provide similar details of studies determined to be low quality per the PQAQ. With one exception, studies had an average domain score in the intermediate (0.34 to 0.66; n = 6) or highest quality (0.67 to 1.00; n = 6) range. Some items within a domain were not scored because they were not applicable to the study in question. The three highest scoring domains were Target population (subscore mean 0.92, SD 0.16), Discounting (subscore mean 0.75, SD 0.52), and Outcomes (subscore mean 0.73, SD 0.30). The lowest scoring domains were Analysis (subscore mean 0.54, SD 0.23), Incremental analysis (subscore mean 0.53, SD 0.44), and Costs and resource use (subscore mean 0.36, SD 0.31). Studies using cost-utility and cost-effectiveness techniques generally scored higher because of their inclusion of both costs and consequences/effects (e.g., QALYs). Also, cost-utility and cost-effectiveness analyses lend themselves better to formal modeling techniques, specifically decision analysis and Markov modeling.

Table 1

Summary of results of quality appraisal of economic evaluation studies of vision screening in young children using the Pediatric Quality Appraisal Questionnaire (n = 13)

PQAQ domain		PQAQ score (0–1)
	Mean	Standard deviation	Range
Economic evaluation	0.71	0.33	0.00–1.00
Comparators	0.68	0.31	0.00–1.00
Target population	0.92	0.16	0.50–1.00
Time horizon	0.63	0.35	0.00–1.00
Perspective	0.60	0.33	0.00–1.00
Costs and resource use	0.36	0.31	0.00–0.80
Outcomes	0.73	0.30	0.00–1.00
Analysis	0.54	0.23	0.00–0.83
Discounting	0.75	0.52	0.00–1.00
Incremental analysis	0.53	0.44	0.00–1.00
Sensitivity analysis	0.62	0.38	0.00–1.00
Conflict of interest	0.61	0.42	0.00–1.00
Conclusions	0.61	0.20	0.17–1.00

PQAQ Pediatric Quality Appraisal Questionnaire

Table 2

Frequency distribution of key characteristics of included studies (n = 13)

Key characteristic	Frequency	Percent (%)
Country of publication
Germany	6	46
China	1	8
UK	1	8
Canada	1	8
USA	4	31
Analytic technique
Cost-utility analysis	1	8
Cost-effectiveness analysis	7	54
Both cost-effective and cost-utility analysis	2	15
Cost-benefit analysis	1	8
Cost-consequence analysis	1	8
Cost analysis	1	8
Cost perspective
Societal	3	23
Third-party payer	7	54
Both societal and third-party payer	1	8
Not reported	2	15
Modelling techniques
Decision tree	4	31
Markov model	2	15
Both decision tree and Markov model	1	8
Not reported	6	46
Discounting
Costs	1	8
Effects	1	8
Both costs and effects	3	23
Not reported	8	62
Sensitivity analyses
Probabilistic sensitivity analysis (PSA)	3	23
One-way sensitivity analysis	5	38
Both PSA and one-way sensitivity analysis	1	8
Type not reported	2	15
Not conducted	2	15
Time horizon
Lifetime	3	23
Up to diagnostic exam	3	23
1 year	1	8
4 years	1	8
10 years	1	8
None	4	31

Table 3

Main characteristics of economic evaluation studies evaluating the cost-effectiveness of vision screening interventions in children to detect amblyopia or refractive errors

First author, year published, country	Evaluation, model	Service(s)	Population size, n	Key assumptions	Target condition(s) and definition(s)	Discounting (%)	Time horizon
Rein DB, 2012, USA	CUA/CEA, not reported	1. No Screening (NS) 2. Kindergarten acuity/stereopsis screening (KA/S) 3. Preschool and kindergarten A/S screening (PKA/S) 4. Preschool photoscreening followed by kindergarten A/S screening (PPKA/S)	10,000	• Arbitrary QALY decrement of 0.01/year of unresolved monocular impairment • No CEEs before age 3 years • Perfect sensitivity and specificity of tests by eye care professional • Sensitivity of preschool stereopsis testing reduced	Amblyopia: VA of 20/80 (0.25) in the worse-seeing eye	3 (costs and QALYs)	Lifetime
Carlton J, 2008, UK	CUA/CEA, Markov model	1. No screening (NS) VA testing by orthoptist at age: 2. 3 years + cover tests without autorefraction (3WOA) 3. 4 years + cover tests without autorefraction (4WOA) 4. 5 years + cover tests without autorefraction (5WOA) 5. 3 years + cover tests + autorefraction (3WA) 6. 4 years + cover tests + autorefraction (4WA) 7. 5 years + cover tests + autorefraction (5WA)	10,000	• Proportion of children diagnosed without screening • No utility decrement associated with amblyopia in reference case • No vision-specific health and social care costs for healthy individuals • Apparent strabismus detected outside screening program • Diagnostic visit: orthoptic testing, cycloplegia refraction, fundus, and media examination	Amblyopia/amblyopia risk factors/strabismus Refractive error: abnormality of at least 2D Strabismus: clinically significant strabismus Moderate amblyopia: 20/40 (0.50) to 20/80 (0.24) Normal vision: ≤ 20/32 (0.63) Amblyopia risk factor (causing moderate impairment): > 20/32 (0.63) and < 20/63 (0.32); Amblyopia risk factor (causing severe impairment): ≥ 20/63 (0.32)	3.5 (costs and QALYs)	Lifetime (death or 100 years)
Konig HH, 2004, Germany	CUA, decision tree/Markov model	1. Usual care (UC)* 2. Orthoptic screening in kindergarten (OS)	412,830**	Permanent visual impairment without treatment	Amblyopia Orthoptic screening – visual impairment: corrected VA of < 0.5 (20/40) Monocular visual impairment: VA < 0.5 (20/40) in the worse eye and ≥ 0.5 (20/40) in the better eye Bilateral visual impairment: < 0.5 (20/40) in OU Ophthalmologic criteria for amblyopia: any newly administered patching therapy or spectacle therapy, if corrected VA ≤ 0.4 (20/50) in either eye or difference of VA between OU ≥ 3 lines	5 (costs and effects)	Lifetime
Gandjour A, 2003, Germany	CEA, not reported	Screening of children: 1. High-risk aged 0 to 1 year (ophthalmologist) (HOPH) 2. Aged 0 to 1 year (ophthalmologist) (OPH) 3. Aged 3 to 4 years (pediatrician or GP) (PGP) 4. Aged 3 to 4 years visiting kindergarten (orthoptist) (ORT)	HOPH: not stated OPH: not stated PGP: not stated ORT: 340,340***	• False positives within 1 year of initial diagnosis and two visits to ophthalmologist • Proportion of patients identified with amblyopia prior to screening • Shorter treatment duration of amblyogenic factors for children aged 0 to 1	Amblyopia/amblyopia risk factors Age 0 to 1 year: amblyogenic factors -anisometropia (1 D), strabismus, and hyperopia ≥ 3.5 D Age 3 and 4 years: amblyopia—VA of 0.63 (20/32)	None	1 year
Konig HH, 2002a, Germany	CEA, decision tree	1. Usual care (UC)* 2. All children aged 3 years in kindergarten screened by orthoptist (ORTH) 3. Variation of ORTH — children with no current ophthalmologic treatment only (ORTH +)	1180	• Participants had no copay for medical services received	Amblyopia/ amblyopia risk factors: Visual deficits with a corrected monocular VA of < 0.5 (20/40) in either eye, or with a corrected monocular VA < 0.8 (20/25) in both eyes and > two logarithmic line difference between eyes on ophthalmologic examination	None	None
Konig HH, 2002b, Germany	CEA, decision tree	1. Monocular VA, pass threshold ≥ 0.8 (20/25) in both eyes, or ≥ 0.5 (20/40) in both eyes and VA difference of ≤ 1 line between both eyes (MVA1) 2. MVA1 with pass threshold ≥ 0.8 (20/25) in both eyes, or ≥ 0.6 (20/32) in both eyes and VA difference of ≤ 1 line between both eyes (MVA2) 3. MVA1 + cover tests + eye motility + head posture (MVA3) 4. MVA2 + cover tests + eye motility + head posture (MVA4) 5. Refractive screening without cycloplegia with Nikon Retinomax autorefractor in normal mode, pass threshold as follows: spherical equivalent ≥ -1D and ≤ 3D, cylindrical power ≤ 1.5D, and spherical equivalent anisometropia ≤ 1D (REFS) Two model options. Children with inconclusive screening results: 1. Referred to ophthalmologist 2. Rescreened after 1 year	1180	• MVA3 and MVA4 exams would take 25% more time than MVA1 and MVA2 • REFS exam time equal to that measured for device-based screening	Amblyopia: Positive test — when any screening item was abnormal Negative test — all screening items were within normal limits Inconclusive: insufficient cooperation	None	Up to diagnostic exam

* No screening is described as usual care and stated as the comparator, but no costs or effects were recorded

**Represents the number of children attending kindergarten who participated, i.e., 90% of 458,700 children attending kindergarten

***Represents the number of children who attended kindergarten who participated in screening, i.e., 91% of 374,000 children attending kindergarten

CEE comprehensive eye exams, VA visual acuity, CA cost analysis, CUA cost-utility analysis, CEA cost-effectiveness analysis, CBA cost-benefit analysis, CCA cost-consequence analysis, AAP American Academy of Pediatrics, AOA American Optometric Association, AAPOS American Association for Pediatric Ophthalmology and Strabismus, ABCD Alaska Blind Child Discovery, OU both eyes, pre-K pre-kindergarten, QALY quality-adjusted life year, GP general practitioner

Table 4

Study outcomes and results

First Author, year, country	Original currency reported: Cost items	Health outcomes/effects	Payer perspective	Costs (2019 C$) mean ± SD or mean (CI)	Health outcomes or effects mean ± SD or mean (CI)	ICER/net expected benefits
Rein DB, 2012, USA	2005 US$: amblyopia treatment, screening, comprehensive eye examination, adult visual health	QALY, amblyopia avoided	Societal	Per child: NS: 1507.7 (1505.9, 1509.6) KA/S: 1544.5 (1542.7, 1546.4) PKA/S: 1579.5 (1577.7, 1581.4) PPKA/S: 1605.3 (1603.5, 1607.1)	QALYs/child: NS: 26.1261 (26.1259, 26.1262) KA/S: 26.1274 (26.1272, 26.1276) PKA/S: 26.1283 (26.1282, 26.1285) PPKA/S: 26.1285 (26.1283, 26.1287) Cases resolved/100 screened: NS: - KA/S: 0.4905 (0.4904, 0.4906) PKA/S: 0.7584 (0.7583, 0.7585) PPKA/S: 0.8247 (0.8246, 0.8248)	C$/QALY gained: NS: - KA/S: 28,322.0 PKA/S: 32,634.6 PPKA/S: 40,653.8 Cases resolved/100 screened: ICERs not calculated
Carlton J, 2008, UK*	2006 GBP: administration, orthoptist time, equipment, room rental, recording screen results, ophthalmologist, optometrist, data entry clerks, clerical staff	Case of amblyopia prevented, QALY gained	Third-party payer (National Health Services)	Per population: NS: 1,753,491.7 3WOA: 2,657,595.9 4WOA: 2,884,784.4 5WOA: 3,127,012.0 3WA: 3,188,545.1 4WA: 3,425,608.4 5WA: 3,728,155.5	QALYs lost per population: NS: 3.21 3WOA: 2.62 4WOA: 2.55 5WOA: 2.48 3WA: 2.46 4WA: 2.36 5WA: 2.35 Cases remaining: NS: 480 3WOA: 393 4WOA: 381 5WOA: 371 3WA: 368 4WA: 353 5WA: 351	C$/QALY gained: NS: - 3WOA: 1,532,380.1 4WOA: 3,245,548.7 5WOA: Dominated 3WA: Dominated 4WA: 2,846,442.6 5WA: 30,254,703.3 C$/case prevented: NS: - 3WOA: 10,392.0 4WOA: 18,932.4 5WOA: Dominated 3WA: Dominated 4WA: 19,315.2 5WA: 151,273.5
Konig HH, 2004, Germany	2000 DM: Organization, orthoptic screening, ophthalmologic examination, treatment	QALY	Third-party payer	Not stated	Not stated	C$/QALY: UC: - OS: 9428.7
Gandjour A, 2003, Germany	1999 DM: direct medical, transportation, productivity losses of caregivers	True positive cases of amblyopia requiring treatment	Societal Health insurance	Per population: HOPH: 6,903,361 ± 1,254,386.7 OPH: 38,469,797.8 ± 5,095,002.2 PGP: 13,979,949.5 ± 1,839,726.6 ORT: 5,924,290.1 ± 1,043,761.4 HOPH: 3,111,647.3 ± 702,161.9 OPH: 17,358,469.4 ± 2,650,316.9 PGP: 9,085,933.6 ± 1,112,509.8 ORT: 4,322,337.5 ± 761,794.0	Per population: HOPH: 3682 ± 1161 OPH: 10,694 ± 994 PGP: 4406 ± 777 ORT: 2809 ± 420 HOPH: 3258 ± 1027 OPH: 9464 ± 879 PGP: 3900 ± 688 ORT: 2486 ± 372	C$/case detected: HOPH: - OPH: 4501.8 PGP: 9774.3 ORT: Dominated HOPH: - OPH: 2295.7 PGP: Dominated ORT: Dominated
Konig HH, 2002, Germany	2000 DM: labour, material costs, travel, ophthalmologic examinations	Number of newly diagnosed cases of amblyopia and amblyogenic factors	Third-party payer (German Social Health Insurance Funds)	Per population: UC: - ORTH: 27,084.0 ORTH + : 24,971.1	Per population: UC: - ORTH: 23 ORTH + : 21	C$/case detected: UC: - ORTH: 1177.6 ORTH + : 1056.4
Konig HH, 2002, Germany	2000 DM: labour, materials, diagnostic ophthalmologic exams, investment, and maintenance costs of the Nikon Retinomax autorefractor	Proportion of newly detected cases of amblyopia	Third-party payer (German Social Health Insurance Funds)	Not stated**	Per populationa: MVA1-1: 22.0 MVA1-2: 21.7 MVA2-1: 23.0 MVA2-2: 22.8 MVA3-1: 23.0 MVA3-2: 22.8 MVA4-1: 24.0 MVA4-2: 23.9 REFS-1: 20.8 REFS-2: 19.5	C$/proportion of cases detected MVA1-1: Dominated MVA1-2: - MVA2-1: Dominated MVA2-2: 1348.3 MVA3-1: Dominated MVA3-2: Dominated MVA4-1: 17,137.6 MVA4-2: 1731.9 REFS-1: Dominated REFS-2: Dominated

VA visual acuity, QALY quality-adjusted life year, CI credible interval for the simulated mean, SD standard deviation

* Carlton et al. reported results for three different calibration methods: absolute difference, mean square difference, and MLE. The results provided here are based on the absolute difference method

** The cost of a single screening exam was provided, but not the total average cost to include ophthalmologic exams

Outcomes were reported as the proportion of newly detected cases of untreated amblyopia in all cases of untreated amblyopia among participating children. MVA1-1 = 84.7%, MVA1-2 = 83.6%, MVA2-1 = 88.6%, MVA2-2 = 87.9%, MVA3-1 = 88.6%, MVA3-1 = 87.8%, MVA4-1 = 92.5%, MVA4-2 = 92.1%, REFS-1 = 80.1%, REFS-2 = 75.1%. Prevalence of untreated amblyopia was 2.2%, n = 1180 ~ 26 children had untreated amblyopia

Low scores were received for the Analysis, Incremental analysis, and Costs and resource use domains for several reasons. Low Analysis scores were because of the lack of an explicit description of the valuation of costs (n = 2), the omission of a health outcome (n = 1), or assumptions instead of measurements of utility scores (n = 3). Analysis was low because studies did not include appropriate units for the indicated analytic technique, the valuation and aggregation of costs and outcomes were not described, and the sources and quantities of resources and their unit costs were not reported with details of statistical tests and confidence intervals where relevant. Incremental analysis scores were low because of missing incremental estimates (costs and consequences) or ratios with confidence intervals or limits (n = 5). Last, the Costs and resource use score was low because the studies lacked transparency which could be achieved by describing the identification, measurement, and valuation of all costs (Canadian Agency for Drugs & Technologies in Health, 2017). Also, included studies had missing costs such as future salary and productivity losses of the child (n = 13) or the parents (n = 2) and missing sources for either volume or unit costs (n = 3).

Sample characteristics

The 13 included studies were published from 2000 to 2019 from five countries: China (n = 1), USA (n = 4), UK (n = 1), Canada (n = 1), and Germany (n = 6). Seven studies used automated or instrument-based screening tools, i.e., autorefractors and photoscreeners such as the Nikon Retinomax. Appendix 6 provides a list of screening instruments used in the included studies. Although cost-utility analysis is considered the gold standard because of its use of a universal generic measure of effectiveness (QALY) (Canadian Agency for Drugs & Technologies in Health, 2017), only three included studies employed this technique. Cost-effectiveness analysis was used more frequently (n = 7). Cost benefit, cost consequence, and cost analysis were used in three studies. In the three studies using cost-utility analyses, assumptions had to be made about health utilities and QALYs based on expert opinion and/or reported correlations between health utility and visual acuity (Konig & Barry, 2004) because the literature is missing values for child populations. Most studies (n = 7) used a third-party payer perspective instead of the broader societal perspective that incorporates all costs and health benefits regardless of the payer. Two studies did not specify a perspective. Seven studies used modeling techniques—a decision tree (n = 4) or Markov model (n = 2), or both (n = 1). Where discounting was applied (n = 5), it ranged from 3% to 5% and was applied to either costs (n = 1) or effects (n = 1), or both (n = 3). The original currency, costs, and incremental cost-effectiveness ratios (ICERs) of included studies are reported in Appendix 7. The ICER is the incremental cost associated with one additional unit of effect (e.g., additional QALY gained, case detected or prevented) and calculated as follows: ICER = (difference in average costs in an alternative strategy relative to another strategy)/ (difference in average effects in an alternative strategy relative to another strategy).

Descriptive synthesis of high-quality studies

Six studies published in the USA (n = 1), the UK (n = 1), and Germany (n = 4) were determined to be high quality with an average PQAQ domain score between 0.67 and 1.00 inclusive (Carlton et al., 2008; Gandjour et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002; Rein et al., 2012). These studies described different vision testing services conducted by paid staff (Rein et al., 2012), volunteers (Rein et al., 2012), teacher’s assistants (Rein et al., 2012) and/or health care professionals (Carlton et al., 2008; Gandjour et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002; Rein et al., 2012) including general practitioners, pediatricians (Gandjour et al., 2003; Konig et al., 2002), orthoptists (Carlton et al., 2008; Gandjour et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002), and ophthalmologists (Gandjour et al., 2003). Screening was conducted in kindergartens, preschool, or clinics (Carlton et al., 2008; Gandjour et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002; Rein et al., 2012). Children who failed screening were referred to ophthalmologists (Gandjour et al., 2003; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002; Rein et al., 2012) or optometrists (Carlton et al., 2008). One study included services targeting high-risk children separately from services targeting all children regardless of risk (Gandjour et al., 2003). Another study included services with different combinations of tools and visual acuity thresholds that informed the decision to refer (König & Barry, 2002).

In terms of outcomes, three studies reported effectiveness as “case detected” (Gandjour et al., 2003; König & Barry, 2002; Konig et al., 2002), and one as QALYs (Konig & Barry, 2004). Two studies reported effects as both amblyopia cases prevented and QALYs (Carlton et al., 2008; Rein et al., 2012). One study considered costs from a societal payer perspective (Rein et al., 2012), four from a third-party payer perspective (Carlton et al., 2008; Konig & Barry, 2004; König & Barry, 2002; Konig et al., 2002), and one from both societal and third-party payer perspectives (Gandjour et al., 2003).

Cost-effectiveness in studies considering a societal payer perspective

Cost-effectiveness reported in incremental cost per additional case detected

Gandjour et al. (2003), compared three alternative strategies for children up to 4 years of age to the screening of only high-risk children under the age of 1 year by an ophthalmologist in Germany (HOPH). The three alternative strategies included (1) universal screening by an ophthalmologist (OPH) which yielded an incremental cost of DM2,571 (C$4,502) per additional case detected; (2) screening all children aged 3 to 4 years by a general practitioner or paediatrician (PGP) which yielded an incremental cost of DM6,445 (C$9,774) per additional case detected; and (3) screening all children aged 3 to 4 years by an orthoptist (ORT). In this comparison of ORT vs. HOPH, ORT was dominated by HOPH, meaning that it was more costly and less effective compared to HOPH. While the ORT program screened all children, the HOPH program screened only high-risk children, which explains the significantly higher costs in the ORT program. In these cases, ICERs are not calculated as per health technology assessment reporting guidelines (Canadian Agency for Drugs & Technologies in Health, 2017). A willingness-to-pay (WTP) threshold, which represents the maximum amount for which society would be willing to pay for particular health outcomes (i.e. cost per additional case of amblyopia detected in this study), was not defined in this study.

Cost-effectiveness reported in incremental cost per additional QALY gained

Rein et al. (2012) compared three strategies to no screening (NS) in a US population using trained staff, volunteers, and teacher’s assistants at a WTP of US$25,000 per additional QALY gained and reported the incremental cost per additional QALY gained as follows: (1) acuity/stereopsis screening in kindergarten (KA/S)— US$15,385 (C$28,322); (2) acuity/stereopsis screening in kindergarten and preschool (PKA/S)—US$17,727 (C$32,635); and (3) preschool photoscreening followed by kindergarten acuity/stereopsis screening (PPKA/S)—US$22,083 (C$40,654). All three strategies were cost-effective, with the first—acuity/stereopsis screening in kindergarten (KA/S)—being most cost-effective. Comparing each strategy to the next most costly, acuity/stereopsis screening in kindergarten and preschool (PKA/S) was cost-effective at an incremental cost of US$21,111 (C$38,864) per additional QALY gained compared to acuity/stereopsis screening in kindergarten (KA/S). Preschool photoscreening followed by kindergarten acuity/stereopsis screening (strategy 3) was not cost-effective compared to acuity/stereopsis in kindergarten and preschool (strategy 2) at an incremental cost of US$70,000 (C$128,865) per additional QALY gained.

Cost-effectiveness in studies considering third-party payer perspectives

Cost-effectiveness reported in costs per additional case prevented

Carlton et al. evaluated seven strategies (no screening, and screening at 3, 4, and 5 years with and without autorefraction) and compared each with the next most costly strategy based on a UK population with costs incurred by the National Health Service (Carlton et al., 2008). Screening at 3 years without autorefraction (3WOA) yielded an incremental cost of £3368 (C$10,392) per additional case prevented compared with no screening. Screening at 4 years (4WOA) compared with screening at 3 years yielded an incremental cost of £6,295 (C$18,932) per additional case prevented, while screening with autorefraction at age 4 years (4WA) compared with no autorefraction (4WOA) yielded an incremental cost of £6,348 (C$19,315) per additional case prevented. Screening at age 5 years with autorefraction (5WA) compared with screening at age 4 years (4WA) yielded an incremental cost of £57,673 (C$151,274) per additional case prevented. Screening at age 5 years without autorefraction (5WOA) and at age 3 years with autorefraction (3WA) was more costly and less effective than the next most costly strategy. A WTP threshold was not defined.

In the study by Rein et al., amblyopia cases prevented were not expressed as an ICER. However, 49% of amblyopia cases were prevented in the KA/S strategy, 76% in PKA/S, and 82% in PPKA/S.

Cost-effectiveness reported in costs per additional case detected

Konig et al. published two studies in 2002 for the German context from the perspective of the German Social Health Insurance (König & Barry, 2002; Konig et al., 2002). In the first study (Konig et al., 2002), three strategies—no screening (UC), screening of all children aged 3 years in kindergarten by orthoptists (ORTH), and screening of children aged 3 years in kindergarten not under treatment by an orthoptist (ORTH +)—were compared with the next most costly strategy. Screening of all children aged 3 years (ORTH) compared with no screening (UC) yielded an incremental cost of DM924 (C$1,178) per additional case detected. Screening of children not under treatment (ORTH +) compared with all children by orthoptists (ORTH) yielded an incremental cost of DM829 (C$1,056) per additional case detected. In the second study (König & Barry, 2002), 10 alternative screening options were modelled comprising screening with different visual acuity thresholds for referrals, and alternative sets of screening tests and tools. For each screening option, uncooperative children were modelled in two ways: (1) with immediate referral to an ophthalmologist; and (2) re-screening after a year. Visual acuity testing with a pass threshold of at least 0.6 (20/32) visual acuity in both eyes (MVA2-2), compared with decreasing the pass threshold to a visual acuity of at least 0.5 (20/40) in both eyes and re-screening a year later (MVA1-2), produced an incremental cost of DM1,058 (C$1,348) per additional case detected. Compared with MVA2-2, including more tests in addition to visual acuity tests (MVA4-2) produced an incremental cost of DM1,359 (C$1,732) per additional case detected. Finally, compared with MVA4-2, immediate referrals to an ophthalmologist instead of rescreening a year later produced an incremental cost of DM13,448 (C$17,138) per additional case detected. A WTP threshold was not defined in both studies.

Cost-effectiveness reported in cost per additional QALY gained

Carlton et al. (2008) examined seven strategies (no screening, screening at 3, 4, and 5 years old using cover tests with and without autorefraction). Comparing each strategy with the next most costly, screening at 3 years without autorefraction (3WOA) compared with no screening yielded an incremental cost of £503,842 (C$1,532,380) per additional QALY gained. Screening at 4 years without autorefraction (4WOA) compared with screening at 3 years without autorefraction (3WOA) yielded an incremental cost of £941,872 (C$3,245,549) per additional QALY gained, while screening with autorefraction at age 4 years (4WA) compared with without autorefraction (4WOA) yielded an incremental cost of £949,750 (C$2,846,443) per additional QALY gained. Screening at age 5 years with autorefraction (5WA) compared with screening at 4 years (4WA) yielded an incremental cost of £8,628,530 (C$30,254,703) per additional QALY gained. Screening at age 5 years without autorefraction (5WOA) and age 3 years with autorefraction (3WA) were dominated, indicating that these two strategies were not cost-effective relative to the next most costly strategy, 4WOA. A WTP threshold was not defined. In Canada, the WTP is typically reported as a range between C$20,000 and C$100,000 per QALY gained (Laupacis et al., 1992). With a WTP between C$20,000 and C$100,000 per QALY gained, none of the strategies was considered cost-effective compared with its comparator.

Last, Konig and Barry (2004) in 2004 compared the costs incurred by the German Social Health Insurance Funds and benefits of screening the vision of all children aged 3 years by orthoptists in German kindergartens (OS) to usual care screening (in which amblyopia could be detected by an ophthalmologist in the absence of orthoptic screening) to detect amblyopia. The incremental cost was determined to be DM7,397 (C$9,429) per additional QALY gained.

Discussion

A wide range in ICERs was reported in the included high-quality studies. From a societal perspective, cost-effectiveness was reported as ranging from C$4,502 to C$9,774 per additional case detected (Gandjour et al., 2003) and from C$28,322 to C$40,654 per additional QALY gained (Rein et al., 2012). From a third-party payer perspective, cost-effectiveness ranged from C$10,392 to C$151,274 per additional case prevented (Carlton et al., 2008), C$1,056 to C$17,138 per additional case detected (König & Barry, 2002; Konig et al., 2002), and C$9,429 to C$30,254,703 per additional QALY gained (Carlton et al., 2008; Konig & Barry, 2004). From both perspectives compared with no screening, screening programs were found to be cost-effective at a WTP threshold greater than C$10,392 per additional case prevented (Carlton et al., 2008), or greater than C$1,178 per additional case detected (Carlton et al., 2008; König & Barry, 2002; Konig et al., 2002). Considering cost per additional QALY gained, screening programs compared with no screening were less likely to be cost-effective with incremental costs ranging from C$28,322 to C$1,532,380 per additional QALY gained (Carlton et al., 2008; Rein et al., 2012). Despite the high quality of included studies, this wide range in ICERs is an indication that uncertainty persisted due to the vastly different ways vision screening services were organized within similar target age groups, differences in the study designs (e.g. the choice of comparator and the diverse outcomes considered), and the various country contexts (e.g. variation in prevalence and in how health care is organized, delivered, and subsidized). The variation in the organization of screening stems partly from the lack of guidelines on best practices for conducting vision screening programs in most jurisdictions. The variation in ICERs may also be caused by the lack of evidence on the impact of amblyopia on quality of life, resulting in a wide range of utility estimates used by different studies. While it is difficult to take a definitive stance on cost-effectiveness overall, the literature suggests that vision screening to detect amblyopia for children under 6 years may be a cost-effective approach compared with no screening if amblyopia reduces quality of life.

The results of the included studies may not be generalizable to populations at high-risk of developing vision disorders because of differences in the prevalence of target conditions, availability, and costs of follow-up care in high-risk vs general populations. The included studies suggest that if amblyopia reduces quality of life, vision screening interventions in large populations using tools with high accuracy may be cost-effective. Therefore, investing in interventions that are affordable and accessible such as those offered in kindergartens and preschools in communities of low socioeconomic status by trained lay-persons (e.g. volunteers and teacher’s assistants) using screening tools with high accuracy may be cost-effective relative to no screening. Autorefractors and photoscreeners are expensive tools compared with traditional tools (e.g. visual acuity charts and stereopsis tests). However, these tools can screen large numbers of children in the shortest possible time with high accuracy, a system that reduces the costs per child. Underserved communities may benefit from optometry exams in schools or supported referrals to eye care professionals by primary care physicians. To prevent children falling between the gaps, greater subsidies for prescription glasses and surveillance systems that allow primary and eye care professionals to identify children that miss follow-up appointments may be beneficial. Additionally, educational campaigns on key vision health topics in native languages may be effective in underserved communities. Indigenous and immigrant communities may especially benefit from the provision of culturally sensitive services which could be achieved by employing local community members as screeners, local champions to support awareness campaigns, and translators. Future economic evaluations should be designed to address the needs of underserved communities by considering relevant variables that account for the higher prevalence, and limited access to and costs of follow-up care and treatment.

One methodological limitation with the included cost-utility analyses was the lack of accurate health state utilities for amblyopia and refractive errors measured in children because of the absence of an appropriate, validated tool. Therefore, studies resorted to the use of proxies (Carlton et al., 2008; Griebsch et al., 2005; Konig & Barry, 2004; Rein et al., 2012), assumptions of no impact of the conditions on health utilities (Carlton et al., 2008), or the same health utilities in children as adults with other types of vision impairment besides amblyopia and refractive errors (Konig & Barry, 2004). These different assumptions on the impact of amblyopia and refractive errors on quality of life in children have led to significant variability in the results of cost-utility analyses. Assuming no impact on health utility is unsubstantiated because it implies an indifference to vision impairment caused by amblyopia and refractive errors despite indications in the literature of the adverse psychosocial impact of amblyopia and refractive errors on individuals (Horwood et al., 2005; Sabri et al., 2006; Senra et al., 2015). One study has reported no association between amblyopia and educational, social activities, employment, health outcomes, and psychosocial domains (Rahi et al., 2006). Only four published studies have attempted to derive health utility weights in children for vision impairment (Boulton et al., 2006; Carroll & Downs, 2009; Petrou & Kupek, 2009; Saw et al., 2003). No studies have been conducted to derive health utility weights in children with vision impairment caused by amblyopia and refractive errors. Another methodological limitation was the lack of reporting or justification of a time horizon. This raises concerns whether all relevant costs and outcomes were captured. Adopting lifetime horizons for economic evaluations in pediatric populations may require several assumptions because of the uncertainty regarding costs and effects in the future.

To the best of our knowledge, no other review of the literature has been published on the cost-effectiveness of vision screening to detect amblyopia or refractive errors in young children. Yet, several reviews on the efficacy and/or effectiveness of vision screening in preschools have noted methodological limitations precluding definitive conclusions on the effectiveness of vision screening programs (Chou et al., 2011; Evans et al., 2018; Jonas et al., 2017; Lagrèze, 2010; Mathers et al., n.d.).

A few limitations in the conduct of this review are worthy of note. Some relevant studies may not have been identified by the search strategy because of poor or no indexing, or because they were included in electronic databases not covered in our search strategy. This is unlikely, however, because our search strategy included a comprehensive number of electronic databases with limited geographical bias. We also validated the search strategy against an initial test set of key articles. To move toward a more definitive conclusion regarding the cost-effectiveness of vision screening programs to detect amblyopia and refractive errors in this population, a meta-analysis would be helpful. A meta-analysis was not conducted because of the heterogeneity in the included studies and disagreement regarding methods for pooling incremental cost-effectiveness (including utility estimates) or cost benefit ratios extracted from multiple economic evaluations (Cochrane Handbook for Systematic Reviews of Interventions, 2011; van Mastrigt et al., 2016).

Conclusion

This systematic review and quality appraisal of the literature on economic evaluations of vision screening strategies in children demonstrated significant variability in types of screening services and the type and quality of methods used, yielding highly variable results. Strategies for enhancing the quality of economic evaluations of vision screening strategies and guidelines on conducting effective vision screening programs are required. Most importantly, prospective studies on the impact of amblyopia and/or refractive errors on the health-related quality of life of young children to generate reliable utilities for use in cost-utility analysis are needed.

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 66 KB)