Teleophthalmology: Evaluation of Phone-based Visual Acuity in a Pediatric Population.

PURPOSE: With the recent rise of teleophthalmology due to coronavirus disease, health care needs accurate and reliable methods of checking visual acuity remotely. The visual acuity as measured by the GoCheck Kids application was compared with that of the Amblyopia Treatment Study (ATS) and the authors' clinic protocol.
DESIGN: This was a prospective, comparison of visual acuity assessment methods.
METHODS: Established patients (3-18 years of age) in the practice of a single pediatric ophthalmologist were eligible. Visual acuity was measured 1) by GoCheck Kids mobile application, by the patient's family member; 2) by HOTV-ATS, by study personnel; and 3) by regular clinic protocol, by an ophthalmic technician. To assess agreement between measurement of acuity, intraclass correlations with 95% confidence intervals (CI) were computed.
RESULTS: A total of 53 children participated. The mean differences between GoCheck Kids and HOTV-ATS acuities (0.094) were significantly different (P < .001). The intraclass correlation coefficient (ICC) was 0.55 (95% CI: 0.40-0.68). The mean differences between GoCheck Kids and chart acuities (0.010) were not significantly different (P = .319; ICC: 0.59; 95% CI: 0.45-0.71). The mean differences between HOTV-ATS and chart acuities (0.084) were significantly different (P < .001; ICC: 0.66; 95% CI: 0.53-0.76). The percentages of eyes with visual acuity measured by GoCheck Kids within 1 line of the HOTV-ATS and chart acuity were 65.3% and 86.7%, respectively.
CONCLUSIONS: GoCheck Kids as checked by a family member provided a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, although most were within 1 line.

Telemedicine has been thrust into ophthalmology practices by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and resulting coronavirus disease-2019 (COVID-19). In March, 2020, the American Academy of Ophthalmology had recommended significant reduction of the number of clinic visits and recommended only seeing patients for urgent and emergent reasons. These guidelines were reinforced by social distancing instructions and stay-at-home orders in many states like Virginia. With this new landscape of medicine, accurate ways of measuring vision at our patient's homes are needed. There are multiple methods of checking visual acuity in children in clinic: Lea symbols, Kay pictures, HOTV, Snellen letters, Sloan letters, Early Treatment of Diabetic Retinopathy Study (ETDRS), and tumbling “E”; all with and without crowding bars.3, 4, 5, 6, 7, 8, 9, 10 Lea symbols, HOTV, and letter acuity have been recommended for use in children. , Visual Acuity is also mass-tested in vision screening programs with the goal of detecting children with visual impairment as recommended by the American Academy of Pediatrics and the American Association for Pediatric Ophthalmology and Strabismus. , In those screening programs, many different eye charts are used, and visual acuity is measured by different people: teachers, nurses, certified medical assistants, public health care workers, students, and other lay persons.14, 15, 16, 17, 18, 19, 20, 21 Mobile and computer applications have been developed to improve and increase the consistency of checking visual acuity in children's screening programs. Some of those applications have been investigated through peer-reviewed publications. , 22, 23, 24, 25, 26, 27 The GoCheck Kids application is used for vision screening through photo screening and checking visual acuity. That application has been studied for photo screening and detecting amblyopia risk factors but not for checking visual acuity.28, 29, 30 The present study was approved by the Institutional Review Board in December 2018 with the goal of exploring a family member checking a child's visual acuity.

Methods

This prospective study was approved by the Virginia Commonwealth University Institutional Review Board. Informed consent was obtained from parents of the subjects, and assent to study them was obtained from subjects >7 years of age. From April 2019 to February 2020, established patients in the practice of a single-pediatric ophthalmologist (E.S.) between the ages of 3 and 18 were eligible for the study. This population was selected because the goal of this study was to evaluate the visual acuity of children with known and treated refractive errors (if required).

Visual acuity was measured in three ways: 1) by the GoCheck Kids mobile application (Scottsdale, Arizona); 2) by HOTV-Amblyopia Treatment Study protocol (HOTV-ATS) on M&S Technologies (Niles, Illinois); and 3) by regular clinic protocol. To mimic visual acuity tested at home, visual acuity was checked by the parent, guardian, or sibling (designated the “tester”) of the patient with GoCheck Kids. A medical student or the pediatric ophthalmologist checked the vision for HOTV-ATS, and an ophthalmic technician checked vision for the regular clinic protocol. The iPhone 7 plus (Apple, Cupertino, California) used for the study was provided to the study by GoCheck Kids. Visual acuity from the GoCheck Kids application was measured with the following protocol. A trained medical student or an ophthalmologist taught the parent or sibling of the patient how to use the application. GoCheck Kids is used to check threshold visual acuity by displaying a test letter in crowding bars (HOTV for children 6 years and younger and ETDRS letters for children 7 years and older) below 4 randomized letters in crowding bars (Figure 1 ). The child subject indicates which direction the tester should tilt the device until the test letter matches the letter in the randomized row. The child then notifies the tester of a correct placement of the letter, and the tester tilts the phone downward to lock in the answer (Video 1 [Supplemental Material available at ]). The protocol is based on software used in the video game EyeSpy. The test is performed at 5 feet, using a previously measured piece of rope held between the child and the tester to ensure proper distance. The right eye was tested before the left eye, and the nontested eye was occluded. Testers were instructed not to look at the screen to blind them to the test. No feedback was given to the parents during the test to mimic testing environment at home. The GoCheck Kids application can report vision from 20/20 (0.0 logMAR) to 20/63 (0.5 logMAR). HOTV-ATS visual acuity was measured at 16 feet (the length of the examination room from chair to visual acuity system) using the M&S system running the Amblyopia Treatment Study protocol. HOTV with crowding bars were used for all participants. The right eye was tested before the left eye, while the nontested eye was occluded. Visual acuity was also measured at 16 feet with the M&S system, using the clinic's protocol, testing with the most challenging acuity that the child could reliably perform: Sloan > HOTV line > HOTV crowding bar (“chart acuity”). Also recorded was the relationship of the tester to the child, the child's sex and race, the glasses prescription, and the child's ophthalmic diagnoses. The presence of vision disease was defined as a diagnosis that affected the child's visual acuity (eg, a child with juvenile idiopathic arthritis with normal examination results was classified as not having a vision disease, and a child with amblyopia was classified as having a vision disease). Data for this study were analyzed using SAS/STAT version 9.4 software (SAS, Cary, North Carolina) for Windows (Microsoft, Redmond, Washington). Descriptive statistics were computed for demographic variables and for GoCheck Kids acuity and HOTV-ATS acuity and the paired difference between them in LogMAR units. To assess the significance between acuities, generalized estimating equations were used to test whether the mean differences were equal to zero. To assess agreement between measures of acuity, intraclass correlations with 95% confidence intervals (CI) were computed. In addition, the percentage of children with visual acuity as measured by GoCheck Kids that were within 1 and 2 lines of HOTV-ATS and of regular clinic protocol were calculated. Study data were managed using Research Electronic Data Capture (REDCap) hosted by Virginia Commonwealth University. , REDCap is a secure, web-based software platform designed to support data capture for research studies, providing 1) an intuitive interface for validated data capture; 2) audit trails for tracking data manipulation and export procedures; 3) automated export procedures for seamless data downloads to common statistical packages; and 4) procedures for data integration and interoperability with external sources.

Figure 1

User interface for the GoCheck Kids mobile application for checking visual acuity. The application displays a test letter in crowding bars (HOTV for children 6 years and younger, and ETDRS letters for children 7 years and older) below four randomized letters in crowding bars. ETDRS = Early Treatment of Diabetic Retinopathy Study; HOTV = the four letters tested.

Results

Fifty-three children were enrolled in the study. The application was prematurely stopped when the vision measurements in 2 children were recorded as much worse than their true vision (96% testability). Observation from the study personnel suggested that the tester was tilting the phone downward while tilting the phone to the side, thus prematurely locking in an answer. These children were not used in the statistical evaluation. Demographics presented in Table 1 . There were 15 children <7 years of age. Four eyes could not be measured by the GoCheck Kids application because they could not identify the largest (20/63; −0.5 logMAR) optotype on the device. At least 2 testers physically dropped the phone during testing. The mean logMAR (imperial) acuity assessed by GoCheck Kids was 0.106 (20/25.5); 0.012 (20/20.5) by HOTV-ATS; and 0.096 (20/22) by chart. The mean differences and CI intervals between GoCheck Kids and HOTV-ATS acuities (acuity differences: 0.094; 95% CI: 0.074-0.114) were significantly different (P < .001). The mean differences and 95% CI between GoCheck Kids and chart acuities (acuity difference: 0.010; 95% CI: −0.010 to 0.030) were not significantly different (P = .319). The mean difference and CIs between HOTV-ATS and chart acuities (0.084: 95% CI: 0.014-0.063) were significantly different (P < .001). The intraclass correlation between GoCheck Kids and HOTV-ATS acuities was 0.55 (95% CI: 0.40-0.68) and 0.59 (95% CI: 0.45-0.71) between HOTV-ATS and chart acuities, indicating fair agreement between each of the 2 sets of measurements of acuity. The intraclass correlation between GoCheck Kids and chart acuities was 0.66 (95% CI: 0.53-0.76), indicating a modest agreement between the measures. Bland-Altman plots are presented in Figure 2 . The percentage of eyes with visual acuity as measured by GoCheck Kids that was within 1 line of the HOTV-ATS, and chart acuity was 65.3% and 86.7%, respectively. The percentage of eyes that was within visual acuity as measured by GoCheck Kids was within 2 lines of the HOTV-ATS, and both chart acuities were 96.9%. There were 14 children with a history of unilateral amblyopia. The differences between the acuity between the amblyopic eye and the nonamblyopic eye as measured by each method are in Table 2 .

Table 1

Patient Demographics

Variable	Statistic
Age, mo	n (%)	51
	Mean ± SD	120.6 (49)
	Min, Median, Maximum	47, 118, 215
Gender
Males		21 (41)
Females		30 (59)
Race
White		26 (52)
African American		15 (30)
Hispanic		5 (10)
Asian		4 (8)
Vision disease		19 (37)
Tester
Parent		49 (96)
Sibling		2 (4)

Data values are n (%) or mean ± SD.

Figure 2

Bland-Altman Plots for the 3 methods of checking visual acuity.

Table 2

Visual Acuity Measured by Each Method for Children with Amblyopia

Age, y	Differences in Acuity Between Eyes
	Phone Acuity	HOTV-ATS	Chart
10.8	0	0	0.2
5.0	0	0.1	0.1
5.8	0.1	0.2	0.2
3.9	0.1	0.1	0.1
5.5	0.1	0	0.1
10.3	0.2	0.4	0.2
7.0	0.2	0	0
7.2	0.2	0.1	0.1
10.7	0.3	0.4	0.1
7.4	0.3	0.1	0
5.4	0.3	0.1	0
13.5	Unablea	0.5	0.6
16.8	Unablea	0.5	0.4
17.3	Unablea	1	1

These children had visual acuities that were too low to be tested by the phone application.

Discussion

Telemedicine has arrived. COVID-19 has pushed us to adapt so we can continue to safely serve our patients. Before COVID-19, telemedicine was rapidly developing in the field of ophthalmology, mostly for diseases like diabetic retinopathy and glaucoma, by obtaining images and sending them to an ophthalmologist for review33, 34, 35, 36, 37 or for interpretation by artificial intelligence and in retinopathy of prematurity using deep neural networks. In the COVID-19 era of teleophthalmology, we do not have the benefit of these sophisticated methods of image acquisition and interpretation. We also need a way of accurately measuring visual acuity, especially in children, in order help diagnose and treat amblyopia. The ideal method of checking visual acuity at home would be easy to use, accurate, precise, inexpensive (or free), ensure monocular testing and be available to test every single patient and family. ABCD-Vision (Anchorage, Alaska) developed a paper-based HOTV crowding bar visual acuity test to be used for remote use in Alaska. This 8.5- × 11-inch paper acuity chart can be emailed for at-home printing or mailed to patients who do not have access to a printer. Another method of checking vision at home is using a smartphone or computer. Some of these programs are based on peer-reviewed studies (Peek Acuity; Peek Vision, London, United Kingdom); EyeHandbook (Cloud Nine Development, Overland Park, Kansas); and Jaeb Visual Acuity Screener (Jaeb Center for Health Research, Tampa, Florida) , 22, 23, 24, 25, 26, 27 , ; and some are not (Farsight; Farsight.care; London, United Kingdom); Eye Chart Pro (Dok LLC; Apple, Cupertino, California); Visual Acuity Charts (Fonlo); and Snellen Acuity (João Meneses [available as an application]) (Table 3 ). There are several issues with using electronic applications to check visual acuity. The first issue is access. Some applications are only available on Android (Google, Mountain View, California), and others are exclusive to iOS (Apple). Other applications are available only on Windows-based computers (Microsoft, Redmond, Washington). Although smartphones are considered ubiquitous, consideration should also be taken into account for those without access to this technology due to their financial situations. This limits the ability to provide uniform directions to patients and have consistent measurements across all patients. However, if a patient uses the same method for checking visual acuity over 2 visits, the acuities are theoretically comparable. Second, we have to trust that the developer has modified the size of the optotypes for different screen sizes. Some applications provide configuration screens to ensure the correct size of the optotypes. Third, the choice of optotypes can be important. Tumbling “E” optotypes (used by Peek Acuity) can be difficult for very young children who have not developed the ability to express the orientation of the optotypes, although more recent studies report excellent testability. Fourth, some orientation may be required for a family to check vision at home, for example, via in app directions, a video guide, or testing while on a teleconference with an ophthalmic technician. This unique study explores the use of GoCheck Kids to check visual acuity by the patient's relative with minimal training. There have been many studies evaluating the assessment of visual acuity by lay screeners in vision screening programs.14, 15, 16, 17, 18, 19, 20, 21 The goal of those studies is to refer children with decreased visual acuity below an age-based critical line for further examination by an eye care provider. Now, the narrative is shifting to explore testing visual acuity as measured by those methods as a visual acuity that an ophthalmologist will use for their evaluation and decision making process. This study shows that visual acuity using GoCheck Kids at 5 feet has fair agreement with HOTV-ATS. On average, the visual acuity as measured by GoCheck Kids is 1 line worse than HOTV-ATS and very close to the chart acuity. The visual acuity measured by HOTV-ATS averages one line better than the chart acuity. Therefore, it is unclear if both GoCheck Kids and chart acuity underestimate vision, or the HOTV-ATS overestimates vision. The average age of children in our study was 10 years old; these children may have understood that there were only 4 choices in the HOTV-ATS and guessed on the letters based on similar recognition rather than true identification, thus measuring vision as better than the other methods. In addition, prior studies have shown that HOTV-ATS overestimates vision compared to E-ETDRS by 0.68 lines for amblyopic eyes and 0.25 lines for fellow eyes. This is the largest weakness of this study. A similar study was performed for the Peek Acuity application. This study tested 111 children, 3-17 years of age and compared visual acuity from Peek Acuity with that from clinic protocol visual acuity and showed a good ICC for the first (0.88) and second (0.84) eyes tested; although the ICC dropped for second eyes (0.45) in children 3-5 years old. The authors did not test for that variable. When assessing and treating amblyopia, one might argue that the exact visual acuity does not matter as much as the difference of visual acuity between the 2 eyes. In children with amblyopia, there was no consistency in the difference between the two eyes among all the methods of checking visual acuity. A total of 1 in 14 children had the same difference of acuity among all methods. A strength of this study is the method that visual acuity was checked using the GoCheck Kids application. We purposefully had the family members of the child perform the measurement to simulate checking vision at home. The study personnel also limited communication and instruction to the tester once the test had started to further mimic home testing environment. Children were able to learn the visual acuity game quickly. Parents had mild difficulty learning the motions needed to manipulate the device, as indicated by the dropped phones and the premature stopping of the test. Further studies could be performed by having family members check vision at home with the application prior to presenting to the clinic for their examination. We recommend further studies use HOTV-ATS for children <7 years of age and E-ETDRS for children >7 years of age.

Table 3

Selected Options for Remotely Checking VA Using Mobile Applications or Computers

	VA Method	Testing Distance	Device	Studied	Available
GoCheck Kids	HOTVETDRSCrowded	5 feet	iOS Android	Method ✓27,aFDA class I device	Yes-free
Peek Acuity	Tumbling EBoxed	2 meters	Android	✓22, 23, 24,bFDA class I device	Yes-free
Jaeb Visual Acuity Screener	HOTVETDRSCrowded	5 feet	Windows	✓25	Yes-free
Eye HandBook	Modified Jaeger numberX-OTumbling ELandolt C	14 inches	iOSAndroid	✓c	Yes-free
“Visual Acuity Charts”- iOSSnellen Chart, ETDRS Chart, Landolt C Chart, Tumbling E chart- Android	Single optotypeSnellenTumbling ELandolt CETDRSHOTV numbers	2-6 meters	iOSAndroid	X	Yes, iOS: free until 5/13/2020, then $4.99Android: free: Snellen, Tumbling E$1.49: ETDRS, Landolt C
EyeChart Prod	SnellenSloanTumbling ELandolt C	4 feet	iOS – iPad only	✓26	Free basic; >$18.00 for increased features
Farsight.care	Numbers	14 inches	Web-based	X	Free
Kay iSight Test Pro	Kay LetterKay Picture lineboxed lineboxed	10 feet	iOS	X- App✓-Kay letters5Underestimate vision by 1 line	Free for 6 months during COVID-19

X = no; ✓ = yes; COVID-19 = coronavirus 2019; ETDRS = Early Treatment Diabetic Retinopathy Study; VA = visual acuity.

Underestimates HOTV-ATS by 0.09 LogMAR, comparable to Snellen by 0.01 LogMAR.

Underestimates vision by 0.07-0.08 logMAR.

Overestimates near vision by 0.11 logMAR (iPhone5).

EyeChart Pro has 2 applications for iPhone: the EyeChart-Vision Screening and the EyeChart HD-Vision Screening. Both apps work at 4 feet but do not recognize the size of iPhone 11 pro; the unit states it is using an iPhone 6. There are other mobile applications for checking visual acuity that are not included on this list.

A weakness of this study was that the children were not patched, using occlusion with patches over glasses, occlusion glasses (that have occluders that flip down over the nontested eye), or a handheld occluder. Children are more consistent with visual acuity when an occlusive patch was applied directly to the skin. In addition, as most children who would be treated for amblyopia may be <7 years of age, this study was limited the our small sample size in this age group. In its current form, GoCheck Kids is limited by the size of an iPhone Plus screen size. The lowest acuity that can be measured at 5 feet is 20/63 (0.5 logMAR). Some testing methods allow children to move closer to the chart and changing the numerator of the visual acuity, but this is not possible with the already short testing distance of 5 feet for GoCheck Kids. This significantly limits the use of this application when testing children with amblyopia, as many of them will have an acuity of <0.5 logMAR. When testing vision at home, it may be important to have a backup test if clinical suspicion arises that the acuity is not consistent with prior tests of vision or if the visual acuity is too low to be tested by the application. It is unknown how GoCheck Kids will perform for measuring visual acuity in children with uncorrected refractive errors. It may over-estimate distance acuity in children with myopia due to the 5-foot testing distance. A program that still allows for matching when needed and the ability to test at 10-feet would be beneficial for a pediatric testing device; a second device that is paired may be helpful for the child to hold when matching the letter at distance (though this would require the family to have two devices).

In June 2020, GoCheck released an application with an updated user interface dedicated to checking visual acuity. It is available outside of the GoCheck Kids vision screening ecosystem. In conclusion, GoCheck Kids has a high rate of testability and provides a modest correlation of visual acuity compared to the chart screen and a fair correlation of visual acuity compared to HOTV-Amblyopia Treatment Study protocol, although most are within 1 line. More studies are needed to evaluate this method of checking visual acuity at home for teleophthalmology.

CRediT authorship contribution statement

Evan Silverstein: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Writing - original draft. Jonathan S. Williams: Methodology, Data curation, Writing - review & editing. Jeffrey R. Brown: Investigation, Writing - review & editing. Enjana Bylykbashi: Investigation, Writing - review & editing. Sandra S. Stinnett: Formal analysis, Writing - original draft.