Patient and Provider Experience in Real-Time Telemedicine Consultations for Pediatric Ophthalmology.

Purpose: Telemedicine adoption hinges on positive experiences for patients and providers. We report participants' experience from our prospective study. Patients and
Methods: Ophthalmic examinations for children 0-17 years of age were conducted by an optometrist using digital exam instruments and streamed to an ophthalmologist. The ophthalmologist, optometrist, parent, and patient (≥10 years) completed surveys capturing patient and provider experience outcomes.
Results: Three hundred forty-eight examinations were conducted with 210 patients in a hospital-based pediatric ophthalmology clinic. About 99% of parents were comfortable with exam quality, and 97% indicated they would have another telemedicine examination. Fifty-four of 55 consented for surgery during the initial telemedicine examination. Thirty-seven percent of families traveled ≥2 hours round-trip to their appointment; 1/3 of parents and patients missed a full day of work/school. Video glasses were by far the most useful instrument, while technical proficiency was most challenging with the digital indirect ophthalmoscope. Problem-focused examinations took 33 minutes of the ophthalmologist's time on average. Equipment challenges caused delays in 40/348 (11.5%) of visits, with the majority lasting 5-10 minutes. In a few cases, a backup device was used. Despite seeing significantly fewer patients on telemedicine days, the ophthalmologist's surgical volume increased 25%.
Conclusion: All participants were satisfied with telemedicine visits despite longer durations and learning curve. Results indicate an opportunity for telemedicine in community settings to improve access to specialized care. Telemedicine enabled the optometrist to manage or co-manage more complex patients with a pipeline to the ophthalmologist for surgical cases. In the right setting, collaborative telemedicine consultations may be beneficial to one's practice.

Introduction

Telemedicine, in its broadest sense, refers to the delivery of health services remotely, via information and communications technology, for the purpose of diagnosing, treating, or preventing disease. Telemedicine has gained traction in ophthalmology over the past 1–2 decades with asynchronous services for disease screening and specialty referrals, mainly in diabetic retinopathy, glaucoma, macular degeneration, and retinopathy of prematurity.1–4

Despite the opportunity to increase access to care and mitigate workforce shortages, long-term adoption of telemedicine into clinical practice hinges on positive experiences for patients and providers. While many papers describe patient satisfaction in tele-ophthalmology, most focus on adults in rural or remote areas where geography is the main hurdle to access.5–10 By contrast, few studies report on patient experience in urban pediatric populations, which face more socio-economic barriers to care, and even fewer report on provider satisfaction in tele-ophthalmology.10,11 A 2020 survey found 67% of ophthalmic clinicians felt at least somewhat confident in their ability to deliver eye care through telemedicine, more than double the number in a 2013 survey of the same group.12,13 The same study reported an association between providers’ confidence and anticipated use of telemedicine, further illustrating the need to assess provider satisfaction.

The Vision Center at Children’s Hospital Los Angeles (CHLA) turned to telemedicine to address a critical access problem, with the demand for pediatric ophthalmology outstripping supply and patients waiting 7–10 months for new appointments. We previously reported the reliability of telemedicine for diagnosing and managing pediatric eye conditions, with a pediatric optometrist on the patient’s end performing real-time telemedicine consultations with a remote pediatric ophthalmologist.14 Secondary outcomes, reported here, explore patient and provider experience.

Materials and Methods

Design

Between February 2016 and April 2018, we conducted a prospective study assessing agreement in diagnosis and management plan between telemedicine and in-person examinations at The Vision Center. A pediatric optometrist within our practice (JC-S) conducted the telemedicine exam on the patient’s side, while a pediatric ophthalmologist (SN) viewed the exam in real time. A research coordinator on the patient’s side assisted with study procedures and data collection.

Eligible subjects were given a choice between telemedicine and in-person examination and were aware that telemedicine offered a shorter wait time for an appointment. Subjects were masked to the fact that if they opted for telemedicine, they would also see the ophthalmologist in-person that day. This was done to gauge their willingness to participate in telemedicine, satisfaction, and trust in management plans (including surgical recommendations) attained via telemedicine.

This study was approved by the CHLA Institutional Review Board (CHLA-15-00526) and conducted in accordance with the Helsinki Declaration. Informed consent was obtained from a parent/legal guardian of all patient subjects. Assent was obtained from children aged ≥7 years, if cognitively able.

At the visit, the optometrist obtained history, performed preliminary testing (vision, stereoacuity, intraocular pressures, retinoscopy when indicated), and conducted the telemedicine examination using wireless streaming video glasses with a high-definition camera at the nasal bridge for first-person point of view broadcasting, a digital slit lamp, and a digital indirect ophthalmoscope. Each device was connected to a local computer and the desktop was shared through a hard-wired video conferencing system. A full description of the platform and examination methods has previously been reported,11 and video of the equipment in use is available here:

Study Population

Eligible patients included children <18 years who could participate in an age-appropriate manner. Patients were recruited from two sources. The first source included patients who had already been evaluated in-person by the pediatric optometrist and required a virtual consult with a pediatric ophthalmologist for a specific referral-type question. These subjects underwent a focused “consult examination” limited to the anatomical area in question and were not dilated unless indicated. The second source included new patients referred from an outside optometrist, pediatrician, or other specialist, and therefore required a broader “comprehensive examination” including assessment of sensorimotor function, slit lamp examination (if age-appropriate), and dilated indirect ophthalmoscopy. The telemedicine examinations for both groups of patients were conducted by the same pediatric optometrist (JC-S) within our group and transmitted to the pediatric ophthalmologist (SN). Although this telemedicine model is best suited for addressing specific referral-type questions in established patients, such as those in the first source, this second source of patients was included in the study as well because wait times for new patients were very long, sometimes many months.

Surveys

The ophthalmologist, optometrist, parent/guardian, patient (if ≥10 years), and research coordinator completed surveys capturing data on clinical and experience factors.

The ophthalmologist’s survey was completed at two time points: first, following the telemedicine examination and, second, at the conclusion of the in-person examination. In addition to clinical findings, the survey ascertained: whether the patient needed to be seen by an ophthalmologist; whether future examinations could be performed by the pediatric optometrist alone or co-managed via telemedicine; which equipment was most useful in attaining the diagnosis; and questions assessing satisfaction with the ability to hear, communicate, and visualize areas of interest.

The optometrist completed her survey at two time points: first, at the conclusion of the examination but prior to the ophthalmologist’s discussion of her findings and plan, and second, following the ophthalmologist’s consultation. The first time point captured whether the optometrist would refer this patient to ophthalmology (45% were referred by outside providers), and, if so, the acuity, timeframe, and reason for referral. The second time point assessed whether the optometrist gained knowledge from the interaction; whether she would change future management of a similar patient and how; and whether she would feel comfortable co-managing that type of patient via telemedicine. Satisfaction questions gauged whether she could adequately hear and understand the ophthalmologist.

The parent/guardian survey was completed before subjects were unmasked to seeing the ophthalmologist in-person. Data included social demographics; prior use of video-call technology; duration of commute and time missed from work and school; and questions assessing comfort, confidence, and willingness to use telemedicine again. The patients’ survey contained a simplified version of the parents’ satisfaction questions.

Lastly, the research coordinator collected data on the duration of examination activities, equipment challenges, and whether the patient consented to surgery.

Satisfaction questions across all surveys were measured on a Likert scale, ranging from 1 = strongly disagree, 3 = neutral, to 5 = strongly agree. Data are presented as simple descriptive statistics.

Results

Two hundred and ten patients (ages 0–17 years, median = 6 years, 3 sets of 2 siblings) and 207 parents/guardians (one per family) enrolled. Participant demographics are displayed in Table 1. Of note, one parent/guardian did not complete the survey. Patients represented a medically underserved population, with 96% insured by Medicaid.

Table 1

Participant Demographics

	Total Number (%)
Patients
Boys	93 (44%)
Girls	117 (56%)
Age range
0–1	34 (16%)
2–3	28 (13%)
4–5	31 (15%)
6–7	31 (15%)
8–9	27 (13%)
≥10	59 (28%)
Race
American Indian or Alaska Native	4 (2%)
Asian	13 (6%)
Black or African American	20 (10%)
Native Hawaiian or Other Pacific Islander	0 (0%)
White	166 (79%)
Mixed or other	7 (3%)
Ethnicity
Hispanic or Latino	163 (78%)
Not Hispanic or Latino	47 (22%)
Primary language
English	94 (45%)
Spanish	106 (50%)
English and Spanish equally	3 (1%)
Other	7 (3%)
Insurance type
Public (Medicaid)	201 (96%)
Private	9 (4%)
Parents/Guardians
Relation to patient
Mother	161 (78%)
Father	37 (18%)
Other	8 (4%)
Age range
< 18 years	1 (<1%)
18–24	10 (5%)
25–34	77 (37%)
35–44	85 (41%)
45–54	27 (13%)
≥ 55	6 (3%)

Sixty percent of parents had used video-call technology before, such as FaceTime. Among them, 65.8% used it at least monthly; 34.2% used it a few times a year or less.

Parent and patient reasons for choosing a telemedicine examination are shown in Figure 1A and B, respectively. While parents rated all choices as important, the leading reason highlights the scheduler’s role in instilling parent confidence in telemedicine. Ninety-four percent of patients referred by the optometrist chose telemedicine in part because they liked their optometrist and would feel more comfortable having her at the exam.

Figure 1

(A) Parent and (B) patient reasons for choosing a telemedicine examination.

All patients were adequately assessed via telemedicine. Of the initial encounters, 94 were comprehensive and 116 were consult examinations. In total, 348 examinations were conducted. Results are reported for the initial visit only unless stated otherwise.

One hundred and thirty-one (62.4%) patients had strabismus as the primary diagnosis. Other common primary diagnoses included eyelid abnormalities (n = 12, 5.7%), glaucoma suspect (n = 10, 4.8%), and conjunctival disorders (n = 9, 4.3%). Sixty-two subjects (29.5%, ages 0.5–17 years, median = 6 years) had surgery, three for nasolacrimal duct obstruction and the rest for strabismus.

Patient Experience and Access Metrics

Nearly all parents (98.5%) felt satisfied with the quality of the telemedicine examination, with 97.0% agreeing to another telemedicine encounter (Table 2). All parents reported this was their first experience using telemedicine.

Table 2

Participant Satisfaction Responses

Parents/Guardians		Patients (≥10 Years)
n = 206		n = 51
I felt comfortable communicating my questions and concerns to the doctor over the telemedicine system.	98.5% agree1.0% neutral0.5% disagree	96.1% agree3.9% neutral	My parents and I felt comfortable talking to the doctor over the telemedicine system.
I felt the doctor could understand my child’s problem.	99.0% agree1.0% neutral	96.1% agree3.9% neutral	I felt the doctor could understand my eye problem.
I was able to adequately hear and understand the doctor over the telemedicine system.	99.5% agree0.5% neutral	N/A
I was able to adequately see the doctor over the telemedicine system.	99.0% agree1.0% neutral	N/A
I feel comfortable with the quality of the doctor’s exam with the telemedicine equipment.	98.5% agree1.5% neutral	96.1% agree3.9% neutral	I feel the doctor was able to do a good exam with the telemedicine equipment.
Being able to see the videos that the doctor was seeing during the exam made me feel confident in the doctor’s ability to conduct the exam remotely.	96.5% agree3% neutral0.5% disagree	N/A
I would have another telemedicine session.	97.0% agree1.5% neutral1.5% disagree	70.6% agree23.5% neutral5.9% disagree	I would have another telemedicine exam.
Overall, I was satisfied with today’s visit.	98.5% agree1.5% neutral	98.0% agree2% neutral	Overall, I was satisfied with today’s visit.
I would feel comfortable having my child’s exam recorded and sent to the ophthalmologist to review at a later time when I am not present, even if it meant I would not get to talk directly to her.	84.0% agree6.8% neutral9.2% disagree	N/A

An indicator of parents’ comfort and trust was the number who consented for surgery during the initial telemedicine examination before knowing there would be an in-person examination as well prior to surgery. Of those who ultimately consented for surgery, 54/55 (98.1%) parents did so during the initial telemedicine examination, while still masked to ever receiving an in-person examination prior to surgery. One parent declined surgery after the telemedicine examination but changed their mind after the in-person examination. Four parents did not consent at all. For two of those, surgery was reconstructive and not for visual function. In one case, the patient had already undergone surgery and the parent wanted to try a different treatment approach first. In the fourth case, the parents declined surgery recommended to improve binocularity even after seeing the ophthalmologist in-person. Of parents who consented for surgery during the initial telemedicine examination, 53.8% felt their child needed surgery urgently and chose telemedicine to see the ophthalmologist sooner. All parents found it helpful to see the magnified videos the doctor was seeing to better understand their child’s condition.

Because this study was conducted at the main hospital to allow an in-person, same-day examination by the ophthalmologist, we cannot say how much transportation time would have been saved had patients participated from a facility closer to home. However, we can report on the status quo (Figure 2A–C). Thirty-seven percent of families traveled two hours or more round-trip; almost one-third missed a full day of school or work.

Figure 2

(A) Duration of patient’s round-trip commute to the clinic. (B) Time parent missed from work to attend the examination. (C) Time patient missed from school to attend the examination.

Throughout the study, the average wait for an in-person new patient appointment in pediatric ophthalmology was eight months and one month for a telemedicine appointment. While this was in part due to some standard clinic days being converted to research telemedicine days, the in-person appointment availability remained stable throughout the study because outside referrals were also eligible for telemedicine (“comprehensive examinations”). Additionally, telemedicine allowed the optometrist at a community site to bypass the cumbersome referral system (which typically includes phone calls, paperwork, and scheduling appointments months in advance), thereby streamlining patients to be seen by the ophthalmologist.

Ophthalmologist Experience

In all 348 encounters, the ophthalmologist could adequately hear and see the patient and visualize areas of interest with telemedicine equipment. In three exams (<1%), she reported difficulty communicating with the optometrist due to a choppy connection or a crying child.

In her estimation, 68.1% of patients did need to be seen by an ophthalmologist (55.3% of comprehensive, 78.4% consultation examinations). Regarding follow-up, 58.1% of patients without active disease could be managed exclusively by the optometrist; 14.3% with active disease requiring close follow-up could be co-managed by the optometrist and ophthalmologist through telemedicine; and 27.6% required pediatric ophthalmology (55 for surgery, 3 for referral to a glaucoma or retina specialist).

Unexpectedly, the ophthalmologist’s surgical volume increased 25% from the same period the year prior, despite replacing regular (4x higher volume) clinics to conduct the study.

Anecdotally, the ophthalmologist felt it was easier to detect subtle eye movements on a big screen than in-person and telemedicine eliminated the multi-tasking of keeping a young child’s attention. Of course, the quality of telemedicine consultations may be influenced by many factors, such as in-person provider expertise and connection quality.

Optometrist Experience

In 6/348 examinations (1.7%), the optometrist reported difficulty hearing the ophthalmologist, but only one was due to a choppy connection (other causes were noise in the room, e.g. crying, and ophthalmologist’s laryngitis). Of the 116 patients she referred, the reasons for referral included surgical consultation (59.5%); uncertain diagnosis, etiology, or acuity (13.8%); need for additional/ancillary testing (16.4%); questions about management (5.2%); and general need for further examination by a specialist (5.2%). Of the 94 patients referred from the outside, she reported that she would have referred 41 (43.6%) for surgical consultation, and only 54 (57.4%) of them overall to the ophthalmologist, notably similar to the 55.3% the ophthalmologist felt actually required ophthalmology.

The optometrist gained knowledge in 49.5% (104/210) of all encounters; this was true throughout the study. In 48.1% (50/104) of those cases, the optometrist reported this knowledge gain would change her future management of a similar patient, 80.0% (40/50) of the time feeling comfortable in managing that type of patient on her own moving forward, mainly recommending observation or trying alternate forms of treatment before referring for surgery. In 26.0% (13/50), she would have recommended a less urgent referral.

For all patients she referred, the optometrist reported she would be comfortable co-managing the patient via telemedicine in the future if the ophthalmologist felt telemedicine was adequate.

Utilization

The average total duration of comprehensive examinations was 80 minutes (including history-taking and preliminary testing by the optometrist), with 44 minutes spent in telemedicine examination and 10 minutes spent counseling, taking 54 minutes of the ophthalmologist’s time. For consult examinations, average total duration was 43 minutes, of which 23 were spent in telemedicine examination, totaling 33 minutes for the ophthalmologist, including counseling.

Technical Issues

Forty of 348 total examinations (11.5%) experienced delay related to equipment challenges (24 new, 16 follow-up). Five (1.4%) required the assistance of Information Services; generally, these were network issues affecting call speed. The most common devices causing delays were the video glasses and the video conferencing system (Table 3). Most delays lasted 5–10 minutes or less. In five cases, backup video glasses were used due to difficulty charging, running out of battery, choppy streaming, or malfunction. There was one delay of 20 minutes, when both video glasses malfunctioned, and the video conferencing camera was used for the strabismus exam. Early in the study, there was one delay of 30 minutes in setting up the digital slit lamp. Averaged across all encounters, delays added approximately one minute extra per encounter.

Table 3

Sources, Duration, and Frequency of Technical Delays

Equipment That Caused Delay	Number of Examinations Affected(N = 348)
Video conferencing system	13
Including network connectivity and freezing
Video glasses	18
Including issues starting the device and
video being dark, pixelated, choppy,
“glitchy”, or freezing
Digital slit lamp	8
Includes issues starting the device,
static, and 1 instance of difficulty
seeing fluorescein staining with blue
light
Digital indirect ophthalmoscope	2
Local desktop computer	4
Length of Delay (minutes)	Number of Examinations(N = 40)
≤5 minutes	24
6–10 minutes	12
11–15 minutes	2
16–20 minutes	1
21–30 minutes	1

There was a learning curve to using the digital indirect ophthalmoscope, which took longer via telemedicine relative to in-person use, in part due to the optometrist needing to adjust her working distance to accommodate a discrepancy between what she saw and what the camera captured. There were six instances of difficulty focusing the video. LED light brightness on the digital indirect hindered patient cooperation in eight young children.

The video glasses were by far the most useful for attaining diagnosis since the majority of patients had sensorimotor and external eye findings.

Discussion

Provider shortages and geographic or socioeconomic barriers have long contributed to disparities in access to care. Telemedicine may enable physicians to focus on more complex patients while ensuring all children receive timely consultations. While telemedicine holds less value in communities where patients face few barriers, in settings where patients live far from the practice or face long waits, it may be a transformative bridge to care.

Telemedicine visits between providers and patients at home surged during the COVID-19 pandemic, including for ophthalmology.12,15–17 While this was a lifesaver to patients, providers, and practices, it also created barriers for patients with limited Internet access or experience with technology. Our model, however, uses provider-to-provider–or generalist-to-specialist–consultations in a clinic setting, offering several advantages. First, the onus of technology requirements is lifted from the patient. Second, having a skilled examiner on the patient’s side allows remote strabismus measurements, even eliminating the need for in-person pre-operative examinations. Third, having an examiner on the patient’s end may increase confidence in management plans.

In a 2020 survey of public attitudes towards telemedicine, most respondents felt “establishing trust and comfort” was best accomplished in-person and preferred meeting surgeons in-person prior to surgery.18 In our study, confidence-building started during scheduling, with the staff member answering questions and helping parents feel comfortable with telemedicine as a modality of care. After the telemedicine examination, satisfaction was overwhelmingly positive with nearly all parents willing to undergo another telemedicine visit. All but one family that consented to surgery did so before knowing they would see the ophthalmologist in-person, underscoring patient confidence in this model.

Real-time telemedicine examinations are time-intensive, making this modality optimal for consultations addressing a specific concern of the referring provider rather than for comprehensive exams. Nevertheless, telemedicine has allowed ophthalmologists in our practice to work more efficiently, as pediatric optometrists see most new patients first–both in our ambulatory clinic and remote community settings–using telemedicine for targeted consultations as needed. Close to half (44.7%) of comprehensive exams in our study did not require ophthalmology, which highlights the utility of this telemedicine model, though further research should assess this occurrence in diverse practice settings. Telemedicine enables us to shift stable, low-acuity, and/or post-operative patients to optometrists for management at community settings, thereby freeing ophthalmologists to focus on acute or surgical patients. Our ophthalmologists block time for telemedicine consultations, which are usually surgical cases pre-screened by optometrists providing care in community settings. Ideally, a single ophthalmologist would be available for telemedicine consultations by multiple optometrists in one block of time and could provide care for 7–8 patients in a half-day (“consult examinations” took 33 minutes of the ophthalmologist’s time). In this way, telemedicine may lead to increased surgical volume for ophthalmologists despite longer visit times, as in this study, and serve as a conduit to care for high acuity or surgical patients who otherwise would have faced long wait times for an appointment. This outcome is particularly beneficial in high-volume settings where coordination between general and specialty providers exists.

The cost of equipment could present a challenge to scaling telemedicine programs. However, the video glasses–by far the most useful piece of equipment for external and anterior segment eye disease–are relatively inexpensive (<$1000). Recent advances in handheld fundus cameras and other smartphone-based retinal imaging devices show great promise as tools for posterior segment evaluation and are also relatively inexpensive.19,20 As the use of telemedicine increases, it is our hope that these types of equipment will become even more user-friendly and affordable. Moreover, with each technological advancement, more research will be necessary to better understand how each type of device affects patient and provider experience.

There are potential limitations of this study. First, examinations were performed by one ophthalmologist and one optometrist who became adept at working together and operating the equipment. While this may impact the generalizability of their experience, both providers were objectively able to visualize areas of interest and communicate during every encounter.

Certain parts of the ophthalmic examination depend heavily on provider skill, and our model employs highly skilled pediatric optometrists on the patient’s end. This may be a barrier for practices without such qualified clinicians, or for resource-constrained practices that simply do not have the capacity to utilize their clinicians in this manner. However, with sufficient training and some modifications to the model, there is little reason to believe this technology cannot be deployed with an orthoptist, technician, nurse, or even a pediatrician on the patient’s end.

A second limitation is potential selection bias among subjects, with those more comfortable with technology being more likely to participate. However, the fact that 40% of parents had not previously used video-call technology suggests this was not uniformly the case with our subjects. Another potential source of bias is shorter wait times affecting patient and parent satisfaction. However, most survey questions in this study focused on satisfaction related to specific aspects of the telemedicine visit (for example, communicating questions and concerns to the ophthalmologist). Lastly, this study was conducted entirely on our internal network. Connectivity challenges at external sites with poorer broadband connection might impede adoption.

Patient and provider satisfaction are key to the long-term adoption of telemedicine. Positive experience outcomes reported in this study support the use of real-time telemedicine in managing pediatric eye conditions in underserved urban populations.

Conclusion

The results indicate an opportunity for telemedicine in community settings to improve access to specialized care. Telemedicine enabled the optometrist to manage or co-manage more complex patients with a pipeline to the ophthalmologist for surgical cases. In the right setting, collaborative telemedicine consultations may be beneficial to one’s practice.