Assessing the prevalence of refractive errors and accuracy of vision screening by schoolteachers in Liberia.

BACKGROUND: Evidence indicates that school-based vision screening by trained teachers is an effective way of identifying and addressing potential vision problems in schoolchildren. However, inconsistencies have been reported in both the testing methods and accuracy of the screeners. This study assessed the prevalence of refractive errors and accuracy of screening by teachers in Grand Kru County, Liberia.
METHODS: We conducted a retrospective analysis of data from four schools where, in February 2019, children were screened for refractive errors by trained teachers and then re-examined by ophthalmic technicians. One row of five optotypes of the Snellen 6/9 (0.2 logMar) scale (tumbling E chart) was used at a distance of 3 m. The prevalence of visual impairment and associations with sex, age and school were explored. Sensitivity, specificity and predictive values were calculated.
RESULTS: Data were available for 823 of 1095 eligible children with a mean age of 13.7 y (range 5-18) and male:female ratio of 1:0.8. Poor vision was identified in 24 (2.9%) children with no differences by either sex or age but small differences by school. Screening by teachers had a sensitivity of 0.25 (95% confidence interval [CI] 0.077 to 0.423) and a specificity of 0.996 (95% CI 0.992 to 1.000). Positive and negative predictive values were 0.667 (95% CI 0.359 to 0.975) and 0.978 (95% CI 0.968 to 0.988), respectively. The results were influenced by a high number of misclassifications in one of the four schools.
CONCLUSIONS: Teachers can be trained to conduct vision screening tests on schoolchildren to an acceptable level of accuracy, but strong monitoring and quality assurance systems should be built into screening programmes from the onset. In settings like Liberia, where many children do not attend school regularly, screening programmes should extend to community platforms to reach children out of school.

Introduction

Refractive errors include a number of common eye disorders that occur when the shape and/or length of the eye prevents light from focusing directly on the retina, causing blurred or distorted vision.[1] Conditions can be easily diagnosed and corrected with spectacles, contact lenses or refractive surgery.[2] Yet unaddressed refractive errors (UREs) continue to be the leading cause of moderate and severe vision impairment and the second leading cause of blindness globally, affecting >157 million people. In addition, 510 million live with UREs affecting near vision.[3] The burden disproportionately falls on low- and middle-income countries (LMICs), particularly in South Asia and sub-Saharan Africa, where the majority of adults and children with refractive errors do not have access to diagnostic services and treatment.[4]

Myopia (near sightedness) is the most common refractive error found in children; it makes distant objects look out of focus and causes difficulties in reading the blackboard and other classroom materials presented at a distance.[5] Other types of refractive errors include hyperopia (far sightedness), which makes near objects look blurry, and astigmatism, which distorts or blurs vision at any distance.[2,6] Children with vision impairment experience a range of developmental, emotional, social and economic challenges that can be severe and long-lasting.[7,8] For school-age children, the consequences often include lower levels of school participation and educational attainment.[9]

Refractive errors in children usually develop between the ages of 8 and 12 y and can be easily corrected, in many cases, with ready-made spectacles.[10] This is an underlying principle of school-based vision screening programmes, which have been successfully introduced in many countries and have been shown to be an effective way of providing timely, good-quality eye health service to children.[11,12] Many LMICs, however, experience significant shortages of eye care personnel and vision screening in schools in these settings is often delivered by trained teachers. Vision screening is not a diagnostic test; its purpose is to identify potential vision problems and refer suspected cases for further examination by an eye care practitioner.[13] A systematic review of school-based vision screening in LMICs published by Opare et al.[14] in 2020 showed that vision screening in schools is cost effective and has positive effects on children's academic performance and learning outcomes, although the authors noted that the number of studies presenting such evidence continues to be small and they come primarily from Asia settings.

Furthermore, from initial pilots with a singular focus on vision screening of children in schools, programmes are increasingly focused on more integrated approaches that extend interventions to children with other impairments and those who are out of school.[15] However, despite their growing popularity, school eye health programmes have been reported to be inconsistent in both the testing methods and the accuracy of the vision screeners.[16] The latter concern was highlighted in the recent multicountry School Health Integrated Programming initiative funded by the Global Partnership for Education through a collaboration with the World Bank and Sightsavers in Cambodia, Ethiopia, Ghana and Senegal, where >57 000 children were screened over a period of 1 y.[17] This pilot study showed that the accuracy of screening is critical for the success and efficiency of school-based eye health programmes and emphasized the need for rigorous quality assurance built within such programmes.

However, published literature on the accuracy of vision screening by teachers remains limited, making it difficult to draw common lessons and recommendations for improvements. Many programmes do not assess and do not report the accuracy of screening. The majority of studies that are published come from Asia, where epidemiology of refractive errors in children is different from, for example, sub-Saharan Africa.[18-21] Available evidence suggests a great variability in the accuracy of screening between the programmes and between the screeners within the same programmes. But while this evidence has been documented, the reasons for the observed variability have not been extensively discussed.[21,22] A few studies that did so reported on issues with the quality of the teacher training, adherence to the screening protocols and variations in teacher attitudes and motivations.[19-21,23]

In this article we present data from a 3-y programme that supported school-based vision screening in Liberia with funding from Dubai Cares. The programme was launched in November 2018 as a partnership between the ministries of education and health and with the aim to train 2400 teachers and screen >76 000 children in 4 of the country's 15 counties: Bong, Grand Kru, Maryland and Sinoe. Thirty-three district education officers have also been trained as trainers to roll out teacher training across the counties and ophthalmic technicians (OTs) have been trained to support the limited number of eye care specialists to provide eye examinations and treatment.

In the screening model used by this programme, trained teachers carried out screening and referred children with suspected vision problems to a mobile team of eye care practitioners, in this case OTs who travelled to schools, examined the identified children and provided ready-made spectacles on the spot or ordered and delivered custom-made (usually high-powered) spectacles at no cost to the children and their families.

In four schools, where the screening methodology was first piloted, we assessed the accuracy of teachers as vision screeners by comparing their screening results with the screening by ophthalmic technicians for the purpose of quality control. In this article we present the results of this assessment and report data in response to two specific questions: What is the prevalence and distribution of vision impairment in school children in the selected programme schools? and How accurate are the teachers as vision screeners compared with the ophthalmic technicians? To our knowledge, this is the first study to address these two questions in Liberia.

Study setting

Despite no recent census, Liberia is estimated to have a population of around 5 million people with >40% being <15 y of age.[24] It is one of the poorest and least developed countries in the world, ranking 175 out of 189 countries on the Human Development Index (HDI) in 2019.[25] The rapid economic growth and development gains achieved in the first decade of the 21st century were wiped out by the aftermath of the 2014 Ebola outbreak. The education sector faces multiple challenges, with an estimated 15–20% of 6- to 14-year-olds being out of school and only slightly more than half (54%) completing primary education. Overage enrolment is a major problem at all levels of the education system, with approximately 4 in 10 primary school students >3 y older than the appropriate age for their grade. The average number of years of schooling is estimated at 4.4 (3 years for girls) and there are significant variations by region and household wealth.[26]

The all-age crude prevalence of vision impairment in Liberia is estimated at 9.5%, or 480 000 people, including 21 000 people who are blind.[27] The eye care infrastructure in the country is very limited, with only 5 ophthalmologists, 1 optometrist and 30 allied ophthalmic personnel.[27] Data on eye health-seeking behaviour are scarce. A retrospective analysis of patient records available in the Liberia Eye Health Centre in the country’s capital, Monrovia, found that about 13% of patients presenting to the centre in 1 y were children <16 y of age. Refractive errors were the most common diagnosis among adults (34% of all diagnoses) and the second most common diagnosis among children (10.7% of all diagnoses). However, the data need to be treated with caution, as they come from a large tertiary facility located in Monrovia and may not be representative of ocular morbidities presented in other hospitals.[28]

Methods

Study design, participants and sampling

We conducted a retrospective secondary analysis of data collected in February 2019 in four pilot schools in Grand Kru County in the southeast of Liberia. The study population consisted of all elementary (primary, ages 6–11 y) and junior high (middle, ages 12–15 y) school children (grades K1–12) enrolled in the four schools. Based on an estimated prevalence of refractive errors in children of 1.09% an α level of 0.05 and confidence interval (CI) of ±5% around the expected sensitivity and specificity of 90%,[19] we required a sample size of 384 children to ensure sufficient statistical power. However, as the programme data were available for all children screened, we assessed the accuracy of screening using all school population data.

Intervention

Two teachers in each school were trained in visual acuity measurement and referrals. The teachers were trained by district education officers (DEOs) who had been trained earlier by the national master trainer qualified in ophthalmology. The teacher training was conducted over 1 d and covered the process of screening and reporting, with half a day spent on practical demonstration of the visual assessment and how to complete the reporting tools (registers, summary and referral forms). The schools were organised in clusters based on their geographical location, both for the purpose of the training and for assessment by the OTs. In Grand Kru, 149 schools organised in 10 clusters participated in the programme. The OTs were divided between the schools with two OTs per cluster.

Prior to the school screening, a team of teachers, DEOs and country health teams (CHTs) conducted community mobilisation and sensitisation events to create awareness and to obtain community consent. Parents were sensitised through Parent–Teacher Associations and community meetings. On the day of the screening, one of the two trained teachers conducted the vision test for each child. The guidelines recommended using one row of five optotypes at a Snellen 6/9 (0.2 logMar) scale (tumbling E chart) at a distance of 3 m, using high-contrast black on white with a dark surround.[29] We chose the 6/9 visual acuity level based on published evidence by Chung et al.[30] suggesting that correcting myopia at a 6/9 instead of a 6/12 level reduces the undercorrection of myopia, hence reducing myopia progression. The 3-m testing distance facilitated observation of the child to ensure they did the screening accurately. A child was recorded as ‘failed the test’ and referred to the OT team if they could not see three of five optotypes. In addition, children who complained about eye pain or other vision-related symptoms or whose eyes appeared abnormal were referred to the OTs.

In the standard screening guidelines, an eye care team travels to schools after the screening, examines all children referred by the teachers, refracts those confirmed with vision impairment and dispenses spectacles or refers children for further examination to a nearby eye clinic. In the four schools described in this article, for the purpose of quality assurance, the OTs rescreened all children present on the day of their visit. After that, all children identified by OTs as having vision impairment were refracted and dispensed spectacles or referred for other treatment. The re-examination took place in the week following the teacher screening. All OTs participating in this programme were trained in India and all had completed the requisite post-training internship.

Data collection

All screening and examination data were extracted from the programme registers, one completed by the teachers and one completed by the visiting OTs. In addition to the results of the vision test, eye symptoms and appearance of the eyes, the registers included children's names, sex, age, school and district. Although the registers used by the teachers and OTs were identical, the information was not recorded consistently, e.g. names were spelled differently. Consequently, a combination of register serial number, school, sex and age of the student was used as a unique identifier to match the data in the registers. The data were entered into Excel (Microsoft, Redmond, WA, USA) and checked for errors.

Data analysis

Summary statistics were used to describe programme enrolment and participants’ sociodemographic characteristics. The prevalence of vision impairment was calculated using the OT-conducted examination as the diagnostic standard. Associations between vision impairment and child characteristics were examined using the χ2 test of independence (for sex and school) and one-way fixed-effects analysis of variance (for age). Sensitivity, specificity and predictive values with corresponding 95% CIs were calculated to measure the accuracy of the teachers’ screening. Statistical Package for Social Sciences (SPSS) version 25 (IBM, Armonk, NY, USA) was used for data analysis.[31] The possibility of clustering was deemed a priori to be insignificant and was not adjusted for.

Results

Overall, 1095 students were enrolled in the four schools at the time of the study. Vision data for 272 students (24.8%) were not available either because they were absent from school on the day of the screening or because the data in the two registers could not be reconciled. The mean age of the 823 students included in the analysis was 13.7 y (median 14, range 5–18) and 55% (n=455) were boys. Of the students screened by the teachers, nine were recorded as ‘failed the test’, ranging from none in FF Doe school to seven children in St Patrick's school. Based on the OT re-examination, poor vision was identified in 24 (2.9%) of the students, ranging from none in FF Doe school to 13 in Sasstown school (Table 1).

Table 1.

School enrolment, student profile and prevalence of poor vision

	School
Characteristics	St. Patricks	Sasstown	Barclayville	FF Doe	Total, n (%)
District	Grand Cess	Sasstown	Barclayville	Dorbor
Students enrolled, n	350	277	375	93	1095 (100)
Absent from school/invalid screens, n	35	32	172	33	272 (24.8)
Included in the analysis, n	315	245	203	60	823 (75.2)
Male, n (%)	169 (53.7)	133 (54.3)	122 (60.1)	31 (51.7)	455 (55.3)
Female, n (%)	146 (46.3)	112 (45.7)	81 (39.9)	29 (48.3)	368 (44.7)
Mean (median) age in years	12.0 (12.0)	13.9 (14.0)	16.2 (17.0)	13.7 (13.5)	13.7 (14.0)
All screens, n	315	245	203	60	823 (100.0)
Passed (good vision in both eyes), n
Teacher screened	308	244	202	60	814 (98.9)
OT screened	306	232	201	60	799 (97.1)
Failed (poor vision one/both eyes), n
Teacher screened	7	1	1	0	9 (1.1)
OT screened	9	13	2	0	24 (2.9)

Sixteen of the 24 children had refractive error, of whom 3 required spectacles, including one case of ready-made spectacles and two cases of custom-made spectacles. Spectacles were not provided to children requiring <0.5D of correction (n=13). This resulted in a prevalence of refractive errors (among children with the data) of 1.9% (16/823) and a prevalence of UREs of 0.4% (3/823). Other identified conditions were functional low vision (3/24), corneal opacity (1/24), optic nerve–related conditions (2/24), pterygium (1/24) and cataract (2/24). Vision impairment was significantly associated with the student’s school (χ2 (3)=9.42, p=0.024), although the effect size was small (Cramer's V=0.107). There were no associations with sex (χ2 (1)=3.16, p=0.075) or age (F(1,820)=4.53, p=0.529). The flowchart of the recruitment and assessment is shown in Figure 1.

Figure 1.

Flowchart of screening recruitment and assessment.

Among the 24 students with vision impairment identified by the OTs, 6 were also screened as ‘positive’ (failed the test) by the teachers, while 18 were missed, resulting in a sensitivity (true positives) of 0.25 (95% CI 0.077 to 0.423). Among 799 students identified as having no vision problems by the OTs, 796 were also passed as ‘negative’ by the teachers, resulting in a specificity (true negatives) of 0.996 (95% CI 0.992 to 1.000). Based on these, the positive predictive value (PPV) of the screening test was 0.667 (95% CI 0.359 to 0.975) and the negative predictive value (NPV) was 0.978 (95% CI 0.968 to 0.988) (Table 2).

Table 2.

Accuracy of vision screening by school teachers

Variable	Failed by OT	Passed by OT	All
Failed by teacher	6	3	9
Passed by teacher	18	796	814
All	24	799	823

Analysis of sensitivity and specificity by school showed little variation in specificity (0.996–1.000). However, the low sensitivity was primarily driven by the results in Sasstown school, where the teacher missed 12 of 13 children with vision impairment (Table 3). Excluding this outlier school from the analysis, the sensitivity of vision screening by teachers in the three remaining schools was 0.465 and the PPV was 0.720.

Table 3.

Sensitivity, specificity and predictive values by school

School	Sensitivity	Specificity	PPV	NPV
St. Patricks	0.444	0.990	0.571	0.984
Sasstown	0.077	1.000	1.000	0.951
Barclayville	0.500	1.000	1.000	0.995
FF Doe	–	1.000	–	1.000
All	0.250	0.996	0.667	0.978

Discussion

The objective of our study was to measure the prevalence of refractive errors among children attending schools and to assess the accuracy of vision screening by teachers compared with OTs in Liberia. We found that 2.9% (24/823) of children ages 5–18 y had vision impairment, with variations by location ranging from 0 to 5.3%. We did not find any differences in the prevalence of refractive errors by either sex or age. The majority of vision impairment was due to UREs, although only three children (0.4% of all screened) had UREs that required correction with spectacles. Where UREs were not considered severe enough to require immediate correction, the schools were recommended to monitor children annually and consider appropriate adjustments in classroom arrangements and teaching approaches. Importantly, the programme also identified children with other eye conditions that required referrals and treatments, and although some conditions were challenging to treat in the Liberia context (e.g. paediatric cataract), the findings highlight the importance of ensuring that school-based vision screening is not conducted as a stand-alone programme but is linked to the provision of eye care more broadly.

To place these findings in context, the all-age prevalence of vision impairment in Liberia is estimated at 9.5%, but similar to other countries, vision impairment in Liberia increases rapidly with age and we are not aware of any comparable national data on vision impairment among school-aged children. Data from hospital-based studies[28] are difficult to interpret, as they are not representative of community prevalence, particularly in settings like Liberia, where the majority of the population lives far away from secondary health facilities and does not have access to eye care services.

Studies from other settings may provide some comparable data but reported prevalence varies greatly due to differences in the study populations, methods of testing and reporting. For example, in a cross-sectional study among schoolchildren ages 10–14 y in Puducherry in southern India, 6.4% of students had vision impairment (potential visual acuity [PVA] <6/12), with significant differences between urban (9.1%) and rural (3.7%) areas.[32] Similar results were reported in studies in Mysore District and Andhra Pradesh.[33,34] In Africa, a number of published studies were conducted in Ethiopia. The most recent study among schoolchildren ages 5–15 y in Gondar town, northwest Ethiopia, reported unilateral prevalence of vision impairment at a PVA <6/18 was 3.7%, while an earlier study from Addis Ababa reported unilateral prevalence of 1.1% at a PVA <6/18 and 5.8% at a PVA <6/12.[35] In southern Nigeria, the prevalence of vision impairment among schoolchildren ages 6–17 y at a PVA <6/12 was 7.3%.[36] The majority of vision impairment in all studies was due to UREs.

Overall, consistent with other studies, our findings show that UREs are the most common cause of vision impairment in children of school age and, while prevalence varies between settings, the condition is common enough to recommend periodic vision screening of school-aged children, providing it is organised in a context-appropriate and cost-effective way. In a context like Liberia, with a relatively low prevalence of refractive errors in children, the integration of vision screening with other school health interventions (e.g. hearing impairment screening, deworming) may be an effective strategy to maximize the benefits of the economy of scale and subsequently the efficiency of screening.

In this study we also tested how accurate teachers were as vision screeners. This finding is critical for the scale up and sustainability of vision screening in low-resource settings like Liberia, where there is a dearth of ophthalmic personnel and conducting regular screening of all school-age children by this specialist cadre is not a viable option. Our results show that in this setting, teachers were very accurate in identifying children with normal vision, with an overall specificity of 99.6%. However, they were less accurate in identifying children with abnormal vision, with an overall sensitivity of 25%. The results were particularly low in one of the four schools with the highest prevalence of vision impairment, where the teacher misclassified 12 of 13 children with impaired vision. When this specific school was taken out of the analysis, the sensitivity increased to 45%. Low sensitivity (or high false negative rates) means that many children with poor vision are missed and continue to go unrecognized and untreated, with significant adverse consequences for their health, education and well-being.

Our earlier vision screening pilots did not integrate systematic measurements of sensitivity and specificity and the programmes picked up only false positives—children incorrectly referred to OTs. Low specificity in these pilots was largely related to insufficient teacher training and supervision, poor motivation and incorrect screening procedures.[20] In other programmes that reported the sensitivity and specificity of teacher screening, results varied greatly. For instance, in Port Harcourt in Nigeria, a study of vision screening in primary schools showed a sensitivity of 53.3% and a specificity of 94.6% (PVA <6/18).[37] In Tamil Nadu, India, the sensitivity was 24.8% and the specificity was 98.7%,[38] while in another India study in Delhi, both sensitivity and specificity were relatively high, 79.2% and 93.3%, respectively (PVA 6/9).[18] A recent study of teacher-led vision screening in Vietnam by Tuan Anh et al.[39] reported a sensitivity of 60.9% and specificity of 93.8%. In other studies, sensitivity varied from 37.5% in Iran[40] to 98% in India[41] and specificity varied from 27.8% in India[41] to 99% in Nepal.[42]

A number of factors are thought to lead to the variations in the accuracy of vision screening. At the epidemiological level, the accuracy may be influenced by the age of the screened children and the prevalence of refractive error in the population. For example, Rewri et al.[43] found that the school teachers in India were more accurate in screening older children (ages ≥5 y) and attributed it to the lack of cooperation and difficulties in understanding teacher instructions by younger children. In Thailand, teachers also reported that screening pre-primary schoolchildren was more complicated and required at least two screeners to conduct the assessment.

At the programmatic level, the accuracy of screening is dependent on the quality of the teacher training, the screening protocol and the adherence to it. For example, while Saxena et al.[18] measured a high degree of accuracy irrespective of the cut-off threshold for referral in India, Teerawattananon et al.[23] reported that the sensitivity of the screening increased from 59% to 74% in Thailand but the specificity decreased from 98% to 46% when the referral thresholds changed from 6/9 to 6/12 optotypes. Some authors also argue that the accuracy of screening varies based on unaided and presenting vision. This was particularly common in countries where there was a high prevalence of refractive errors. For example, in Vietnam,[21] the sensitivity of vision screening based on unaided vision was 10% higher than with presenting vision (86.7% compared with 75.3%). In Thailand, teachers also reported behaving differently when the child was wearing spectacles or complained about vision.[23]

Finally, a number of authors argue that the accuracy of vision screening by the teachers varied even when the screening protocol and the training were standardised. The reasons behind these variations have not yet been fully explored but most attribute these to teachers’ interests and motivations to conduct vision screening. For example, Bechange et al.[20] found that in Pakistan, while some teachers were highly motivated and diligent in following the screening protocol, others could not clearly recall the procedure, reported incorrect use of the E chart and, in a few cases, delegated their screening task to older students.

Various strategies have been recommended to improve teacher screening performance, including refresher training,[20] having standardised protocols,[44] involving all (rather than selected) teachers in the vision screening[45] and motivating teachers through financial incentives.[23,46] However, similar to other studies,[16] our analysis by school suggests that inaccurate screening is a localized problem determined by commitment, skills and practices of individual teachers rather than the quality of the programme as a whole. This highlights the importance of comprehensive and systematic quality assurance audits integrated in the implementation of vision screening programmes and repeated at regular intervals and random locations.

Another important insight from this study that needs to be taken into account when planning school-based vision screening programmes is that a quarter of the enrolled students were not included in the analysis. While inconsistent screening records (i.e. by teachers and OTs) was partly responsible, non-attendance in school on the day of screening was also a factor. School attendance and completion rates in Liberia are recognised to be poor[26] and this finding highlights the importance of both vision screening ‘mop-ups’ and utilising community opportunities for reaching children who are not in school.

Study limitations

The relatively small number of schools from only one county undermines our ability to generalise our findings to the whole programme. The impact of misclassifications from just one school highlights the risks of drawing conclusions from small samples such as this. Also, although it was possible to estimate an overall prevalence of vision impairment, the small number of students with vision impairment means that prevalence estimates for subgroups (e.g. sex and age) are subject to large intervals of uncertainty. Lack of community-based prevalence data among children who do not go to school—estimated to be 160 000 primary-age children—means that even with good coverage in schools, there are likely to be many children who never undergo vision screening.

Conclusions

In conclusion, the results of this study suggest that teachers can be trained to an acceptable level of accuracy to conduct vision screening tests on schoolchildren. However, strong monitoring systems and quality assurance spot checks followed by refresher training should be built into the screening programmes from the beginning. It is also important to consider opportunities for integration of vision screening with other school health interventions and for utilising community-based platforms for reaching out to schoolchildren. Future research should focus on the analysis of costs and cost-effectiveness as well as on the strategies for maximising the efficiency of school-based vision screening.