Pediatric otolaryngology telemedicine amid a pandemic - And beyond.

INTRODUCTION: The coronavirus disease of 2019 (COVID-19) pandemic catalyzed an unprecedented redesign and innovative overhaul of health care delivery thrusting from fringe to mainstream virtual care. With a return to conventional practice, we now must create a research and policy agenda using the changes wrought by COVID-19 to help create a better health care system in its aftermath. The purpose of this study was to assess satisfaction of otolaryngology outpatient visits during the pandemic.
METHODS: A prospective survey study was performed on caregivers of all patients ages 0-26 years old seen in the Division of Pediatric Otolaryngology at our large tertiary care children's hospital from February-April 2020. The three study groups were those seen in-person 6 weeks before telemedicine was implemented (IBTM), those seen in telemedicine during the first 6 weeks (TM) it was implemented at our hospital, and those seen in-person during the telemedicine period (IDTM) in the same timeframe. The survey consisted of satisfaction questions related to their visit, if their child was recommended surgery at the time of the visit, and if the caregiver agreed with the recommendation. A medical record review was also performed.
RESULTS: A total of 176 caregivers completed the survey with 113(64.2%) completing the survey for an IBTM appointment, 59(33.5%) for a TM appointment, and 4(2.3%) for an IDTM appointment. There were 100(56.8%) male patients and 167(94.9%) were white. Families gave a higher response for the statement "The ability to communicate with the physician" (p = .012) and "The overall outpatient experience" (p = .004) in the IBTM cohort compared to the TM group. There were no significant differences for the other statements regarding the ability to understand recommendations, courtesy, and knowledge of the physician. Regardless of group, 98.6% of caregivers agreed with surgical recommendation when surgery was recommended. However, when surgery was not recommended at the appointment, caregivers were 11x more likely to disagree with the surgical recommendations, OR:11.49,95%CI:1.44-91.38,p = .005.
CONCLUSION: We conclude that telemedicine was equally well received by patients as compared to traditional live assessments suggesting that virtual care is a viable post-pandemic paradigm change. Satisfaction was rated as "Good" or "Excellent", however, messaging when surgery is not recommended was less acceptable and must be improved to obtain increased caregivers' agreement in an era of shared decision making.

Introduction

The coronavirus disease of 2019 (COVID-19) pandemic catalyzed an unprecedented redesign and innovative overhaul of health care delivery thrusting from fringe to mainstream virtual care. With a return to conventional practice, we now must create a research and policy agenda using the changes wrought by COVID-19 to help create a better health care system in its aftermath. The pandemic shall pass but whether we like it or not, telemedicine is here to stay. It is incumbent on us to establish a value-based ‘New Normal’ for telemedicine. Telemedicine will help reduce disparities of access to patient care [1]. Telemedicine in otolaryngology was first cited in the literature in 1998 and was coined a “threat or a fantasy” by some physicians [2]. In the early 2000s, telemedicine was perceived as a way to radically change how surgeons practice but remained in infancy [[2], [3], [4], [5]] [[2], [3], [4], [5]] [[2], [3], [4], [5]]. Twenty years later otolaryngologists were forced to abide by national laws for everyone's safety and almost 100% of patients were seen through telemedicine during the national shutdown. Recent literature has established that telemedicine will play a vital role in our future [6,7]; however, our study is the first to understand how this sudden transition has affected our number one stakeholder: the patients.

The purpose of this study was to assess satisfaction of otolaryngology outpatient visits during the pandemic between three study groups: those seen in telemedicine, those seen in-person during the telemedicine period, and those seen in-person before telemedicine was implemented. We hypothesized that there would be no difference in satisfaction 1) with surgical recommendations 2) between the three groups 3) between attending physicians and advanced practice providers (APPs) 4) over time in the telemedicine group.

Methods

An IRB protocol (STUDY20040033) was quickly approved by the University of Pittsburgh to perform a prospective survey and medical record review study. All patients ages 0–26 years old seen in the Division of Pediatric Otolaryngology at our large tertiary care children's hospital from February 2020 to April 2020 were included. A study timeline is shown in Fig. 1 . IRB approval was effective April 15, 2020, and surveys were sent thereafter. The three study groups were those seen in-person 6 weeks before telemedicine (IBTM) was implemented, those seen in telemedicine during the first 6 weeks (TM) it was implemented at our hospital starting on March 23rd, 2020, and those seen in-person during the telemedicine period (IDTM) in the same timeframe.

Fig. 1

Survey study methodology.

Patient name and age as well as their caregivers' email address were collected for every patient seen during our study period. Both APP and attending provider clinics were included for analysis. Caregivers were contacted by email with a recruitment letter with a link to the HIPAA compliant electronic consent form and survey through the web application Research Electronic Data Capture (REDCap) [8]. Participants were reminded twice about the study every two weeks for a total of three opportunities to complete the survey and participate in the study. The caregiver had the option to electronically decline inclusion and they were not contacted further. An introductory letter stated participation would not affect their child's care and was voluntary.

All three groups were sent the same satisfaction questions related to their visit and if their child was recommended surgery at the time of the visit and if the caregiver agreed with the recommendation at the time (Fig. 2 ). Additional questions were asked depending on which cohort the patient was in. For example, the TM group caregivers were asked on a 5-point Likert scale if they would have preferred their child to have been seen in-person, if they still wanted their child to be seen in-person for the same issue at a later date, and if they would recommend telemedicine to their family and friends.

Fig. 2

Six satisfaction statements related to pediatric otolaryngology clinic visit.

Medical records were reviewed for the patients whose caregivers completed the survey. Information regarding provider seen, demographics, proxy socioeconomic status, craniofacial and syndromic history, otolaryngology appointment and surgical data, follow-up nurse phone calls, virtual messaging to providers’ notes, and appointment notes. Appointment reason, diagnosis, and surgical recommendations were collected from the appointment. Appointment for a new problem and a follow-up appointment were not mutually exclusive.

Most, if not all, patients were seen and referred from primary care providers before their initial visit with the otolaryngology specialists. Patient diagnoses were given after careful review of caregiver reported symptom history and a review of relevant medical records pertaining to the scheduled visit. No commercially available otoscopy apps were used in this study. Pediatrician pneumatic otoscopy and otolaryngologic microscopic otoscopy if they were taken to surgery were used to diagnose pathology of the tympanic membrane. We did not solicit photos from other providers, however, we asked caregivers to provide a light source (i.e., flashlight, cell phone light) in order to facilitate the physical exam of the child via the computer monitor. We initiated our telemedicine program through Doxy.me® and later integrated a telemedicine platform through our Cerner® EMR system. All of the department's providers were involved in care during this time period with ten physicians and seven APPs.

SPSS version 24 was used for analysis with p < .05 denoting statistical significance [9]. Along with descriptive statistics, Mann-Whitney U test and Fisher's Exact test were used for analysis.

Results

A total of 176 caregivers completed the survey with 113 (64.2%) completing the survey for an IBTM appointment, 59 (33.5%) for a TM appointment, and 4 (2.3%) for an IDTM appointment. There were 100 (56.8%) male patients and 167 (94.9%) were white. A majority were seen by a physician, 150 (85.2%), as opposed to an APP. The average age at the appointment was 5.47 years (SD: 4.99). Demographics and patient characteristics were broken down into the three patient groups in Table 1 .

Table 1

Demographics and characteristics of the patient sample who completed the post-appointment satisfaction survey.

	In-person Before Telemedicine n = 113	Telemedicine n = 59	In-person During Telemedicine n = 4
Sex, n (%)
Male	70 (61.9%)	29 (49.2%)	1 (25.0%)
Female	43 (38.1%)	30 (50.8%)	3 (75.0%)
Race, n (%)
White	107 (94.7%)	56 (94.9%)	4 (100%)
Black	5 (4.4%)	0 (0%)	0 (0%)
Not specified	1 (0.9%)	3 (5.1%)	0 (0%)
Craniofacial abnormality, n (%)	14 (12.4%)	9 (15.3%)	2 (50.0%)
Syndrome, n (%)	5 (4.4%)	10 (16.9%)	1 (25.0%)
Age at appointment (years), M±SD; mdn	4.99 ± 4.86; 3.45	6.15 ± 5.07; 4.54	8.78 ± 6.40; 9.19
Distance from home zip code to hospital (miles), M±SD; mdn	24.69 ± 23.33; 18.90	44.03 ± 32.58; 35.40	22.35 ± 13.95; 18.50
Time from home zip code to hospital (minutes), M±SD; mdn	40.27 ± 24.82; 31.00	59.25 ± 33.41; 56.00	39.75 ± 14.86; 33.00

Abbreviations: M, mean; SD, standard deviation; Mdn, median.

Those seen in TM were more likely to have a syndrome compared to those seen IBTM, p = .009. There were no differences in age at appointment between IBTM and TM groups (p = .058), but TM families had a longer distance from their home zip code to the hospital with an increased time to the hospital, p < .001 and p < .001, respectively. New evaluation of chronic otitis media with effusion (COME) and/or recurrent acute otitis media (RAOM) was the reason for the appointment for 54 (30.7%) patients and a tonsil evaluation was the reason for 37 (21.0%) appointments with a total of 87 (49.4%) patients having either of these reasons. About half of the patients for both IBTM and TM appointments had either a new ear or tonsil issue (IBTM 53 (46.9%) and TM 33 (55.9%)). The four patients who were seen IDTM period were seen for RAOM, functional hearing loss, postop tympanoplasty, and nasal fracture.

Surgical recommendations x satisfaction

For a total of 72 (40.9%) patients, surgery was recommended at the appointment during the study period. Table 2 shows the three groups of patients with frequencies of appointment type and surgical recommendations. Overall, 47 (49.5%) patients had surgery when presenting to the appointment for a new problem and 25 (27.8%) patients with follow-up appointments had surgery. Including the whole sample, of patients for whom surgery was recommended at the appointment, 71/72 (98.6%) caregivers reported that they agreed with the surgical recommendations. However, when surgery was not recommended at the appointment, caregivers were 11 times more likely to disagree with the non-surgical recommendations, 68/79 (86.1%), OR: 11.49, 95%CI: 1.44–91.38, p = .005. Excluding follow-up appointments there was a 16-percentage difference in agreement between surgical recommendations and caregiver agreement (81.8% no surgery and agree versus 97.8% surgery and agree), p = .019. Although not significant, there was a 5.8% decrease in caregiver agreement with surgical recommendations from the shift of IBTM to TM (93.8% vs 88.0%, p = .340).

Table 2

Appointment type and surgical recommendations, n (%).

	Whole sample n = 176	In-person Before Telemedicine n = 113	Telemedicine n = 59	In-person During Telemedicine n = 4
Follow-up appt	90 (51.1%)	61 (54.0%)	28 (47.5%)	1 (25.0%)
New problem	95 (54.0%)	55 (48.7%)	37 (62.7%)	3 (75.0%)
Surgery recommended	72 (40.9%)	46 (40.7%)	24 (40.7%)	2 (50.0%)
Surgery performed	67 (38.1%)	41 (36.3%)	24 (40.7%)	2 (50.0%)
Caregiver agreement with surgical recommendations	139/151 (79.0%)	91/97 (93.8%)	44/50 (88.0%)	4/4 (100%)

Satisfaction

When caregivers were asked if their child was seen in the ENT clinic before the visit, 60% were seen previously by our department, with both IBTM and TM groups equally seen previously, 61.1% and 59.3%, p = .870. There were also no differences between the two latter groups when asked “Was it easy for your child to access a pediatric ENT specialist for a telemedicine/in-person visit”, 90.2% versus 91.4%, p = 1.000. However, when the TM cohort was asked “Did telemedicine make it easier for your child to access a pediatric ENT specialist, 48 (81.4%) caregivers responded ‘Yes’. The families were then asked to explain their answer to the previous yes/no question. Open-ended responses were dichotomized with 39 positive experiences noted: 10 (20%) responses relating to a positive experience due to the pandemic, 15 (30%) relating to distance to the hospital and convenience, 12 (24%) citing they were able to receive an earlier appointment, 2 (4.0%) were positive in general, and 11 (22%) families responded negatively, citing their child needed to be seen in-person for an exam.

Caregivers were asked to rate 6 statements on a scale of 1 (Poor) to 5 (Excellent) for the question “How would you rate the following aspects of your child's experience”? seen in Table 3 . Families gave a higher response for the statement “The ability to communicate with the physician” (p = .012) and “The overall outpatient experience” (p = .004) in the IBTM cohort compared to the TM group. There were no significant differences for the other statements regarding the ability to understand recommendations, courtesy, and knowledge of the physician, p > .05. A box and whisker plot is seen in Fig. 3 with asterisks to display significance.

Table 3

Statement responses to the satisfaction survey question “How would you rate the following aspects of your child's experience?”

	In-person Before Telemedicine n = 113		Telemedicine n = 59
	M (SD)	Mdn (Range)	M (SD)	Mdn (Range)	p value
The ability to communicate with the physician	4.6 (0.8)	5.0 (1–5)	4.4 (0.7)	5.0 (2–5)	.012
The ability to understand clinical recommendations	4.8 (0.6)	5.0 (1–5)	4.6 (0.6)	5.0 (2–5)	.120
The courtesy of the physician	4.9 (0.4)	5.0 (3–5)	4.9 (0.3)	5.0 (4–5)	.545
The courtesy of the staff	4.8 (0.5)	5.0 (2–5)	4.8 (0.4)	5.0 (4–5)	.825
The knowledge of the physician	4.8 (0.5)	5.0 (2–5)	4.9 (0.4)	5.0 (3–5)	.499
The overall (telemedicine) outpatient experience	4.7 (0.7)	5.0 (2–5)	4.3 (0.5)	5.0 (1–5)	.004

Response scale: 1 – Poor | 2 – Fair | 3 – Average | 4 – Good | 5 – Excellent.

Fig. 3

In-person Before Telemedicine versus Telemedicine responses on a rating scale from 1 (Poor) to 5 (Excellent) for the ability to communicate with the physician, the ability to understand clinical recommendations, The courtesy of the physician, The courtesy of the staff, The knowledge of the physician, The overall (telemedicine) outpatient experience; *p < .05, **p < .01. Min, max, median, and first and third quartiles are shown. All medians were a ‘5’ on the scale.

Lastly, 41 (69.5%) caregivers responded that they would seek a telemedicine outpatient visit for their child's care in the future and 40 (69.0%) caregivers agreed or strongly agreed that they would recommend telemedicine to their friends and family.

Satisfaction between providers

Most of the completed survey responses came from families who were seen by a physician as opposed to APP (84.9%). Due to low APP representation, satisfaction scores were only analyzed across provider groups for the whole study period. There were no differences in scores for the six satisfaction statements between the physicians and APPs, p > .05. There were also no differences in satisfaction scores when looking at the IBTM cohort, p > .05.

Satisfaction over time for telemedicine

The telemedicine time period was split into two halves between the first and last three weeks to assess the potential learning curve of telemedicine for staff and providers. There were no differences in scores between halves for the ability to communicate with the physician, the ability to understand clinical recommendations, the courtesy of the physician, the knowledge of the physician, nor the overall outpatient experience, p > .05. All response rating means corresponded to “Good” or “Excellent”. However, the first half of the telemedicine period scored higher for the statement “The courtesy of the staff” than the last half of the period (M±SD = 4.9 ± 0.4 versus 4.7 ± 0.5), p = .016. Both responses correspond to the rating of “Excellent” on the five-point scale.

Sub-analysis

When only COME/RAOM and tonsil patients were included in the analysis (n = 87), 38 (71.7%) patients from the IBTM group were recommended surgery at the appointment compared to 17 (51.5%) patients in the TM group, p = .048. There was no difference in surgical recommendation agreement between the two groups (IBTM 95.9% agreement versus TM 87.1% agreement), p = .153. The IBTM group scored higher in satisfaction in the ability to communicate with the physician (p = .028), the ability to understand clinical recommendations (p = .049), and the overall outpatient experience, p = .029.

Discussion

This is the first otolaryngology-related study regarding the COVID-19 pandemic that combined a medical record review and survey responses from 6 weeks before telemedicine was adopted and 6 weeks during the beginning of the pandemic. Our three study groups were those who were seen virtually through telemedicine, a historical control group of those seen in our clinic in-person prior to COVID-19, and to those seen in our clinic during the first 6 weeks of the national shutdown. In addition to the surveys, we have compared physician surgical recommendations, such as if the surgery was recommended at a preoperative appointment compared to a more conservative watch and wait approach. Our findings assessed if telemedicine led to a relevant change in paradigm and platform for pediatric otolaryngologists for after the pandemic has passed.

Although the prevalence of COVID-19 was and is less in Allegheny County than other regions in the United States and abroad, the degree of infection penetration increased rapidly. At the start of telemedicine at our hospital on March 23rd, there were 57 cumulative cases according to the Pennsylvania Department of Health with a total population of 1,216,045. At the end of the 6-week study period on April 30th, there were 1,318 cases. As of late October 2021, 10.9% of the county's population (132,673) had become infected [10]. Due to the rapid increase in infection numbers and future uncertainty, telemedicine is here to stay.

A plethora of articles, mostly reviews, have been published since the start of the pandemic on the implementation and great success and satisfaction of telemedicine in many different medical specialties globally [[11], [12], [13], [14], [15]] [[11], [12], [13], [14], [15]] [[11], [12], [13], [14], [15]]. More specifically, the literature has expanded on the implementation and satisfaction in otolaryngology telemedicine [6,16] [[6], [16], [17], [18]] [[6], [16], [17], [18]]. A systematic review by Moentmann et al. found that out of 35 articles that met inclusion, 32 (91.4%) of them had an effective telemedicine platform, compared to in-person appointments [19]. Darr et al. performed a retrospective review of a satisfaction survey given from mid-March to mid-June 2020 called the modified pediatric otolaryngology telemedicine satisfaction survey (POTSS) [20]. High satisfaction was found when assessing the consultation, the doctor-patient relationship, and privacy and trust during a virtual outpatient clinic [20]. In our study, most of our families reported a “Good” or “Excellent” experience at their visit, including the newly adopted telemedicine platform.

When caregivers were given the six satisfaction statements to rate on a 5-point scale from poor to excellent, those in the IBTM group rated the ability to communicate with the physician and the overall outpatient experience as slightly higher than those in the TM group. Despite longer visits for telemedicine, caregivers were more satisfied with face-to-face physician communication, suggesting the need to adopt different communication skill sets virtually. Ironically, despite the purported goal to utilize telemedicine to minimize health care disparities, our data revealed an overwhelming preponderance of white patients (94.9%), possibly a result of willingness to participate. The distance from the hospital was longer, as expected for those opting virtual care, and the specific diagnoses seen live were by design and patients pre-selected per our divisional protocol for live assessment. Although we did not survey the reason for being seen in-person versus telemedicine, we suspect syndromic patients were seen at increased frequency via telemedicine due to the accessibility of the platform as well as care available in a less stressful environment without the need for traveling with a complex patient.

The most unexpected yet significant finding in our study was the disparate satisfaction with surgical vs. non-surgical messaging. Interestingly, surgery was recommended an equal percentage (40.7%) for patients seen IBTM and seen during TM. When surgery was recommended, almost 100% of caregivers self-reported that they agreed with the surgical recommendations. When surgery was not recommended, caregivers were 11x more likely to disagree with the recommendations, and, although not significant, there was a 5.8% decrease in agreement from in-person before telemedicine to telemedicine (93.8% vs 88.0%). A prospective study powered to show a difference is warranted to confirm this trend. In a sub-analysis of COME/RAOM and tonsil patients, there was a significant 20.2% decrease in surgical recommendations from being seen before the pandemic to during (TM group). Many studies have been published describing aerosol-generating otolaryngology procedures and the higher risk of exposure to COVID-19 for otolaryngologists [[21], [22], [23], [24]] [[21], [22], [23], [24]] [[21], [22], [23], [24]]. At our hospital, there was a detailed algorithm for those that were eligible to receive these surgeries at start of the pandemic. However, messaging when surgery is not recommended must be improved to obtain increased caregivers' agreement in an era of shared decision making. Caregivers perhaps were frustrated with the conservative treatment of otolaryngology-related disorders during the telemedicine period, which was under the constraints of uncertainty. We also hypothesize that a variety of tools available during a live visit that serve as support for a non-surgical recommendation were not available in our virtual visits. For example, in the case of tympanostomy tube insertion, in a live visit, technology enables the parents to actually see their child's tympanic membranes and middle ears. Audiometry and tympanometry are also available to further reassure parents that watchful waiting is reasonable. Though simple improvements in communication and messaging sensitive to the differences in virtual vs. live encounters would likely improve patient satisfaction, employing more sophisticated diagnostics which are commercially available such as otoscopic apps would likely markedly improve the credibility and parental buy-in of non-surgical recommendations when the pre-consultation expectation was surgical.

As a future direction, effective communication strategies must be deployed. Providers in all disciplines were forced to learn how to provide telemedicine to patients quickly and as a result of this, many different platforms, guidelines, checklists, and pathways were implemented [[25], [26], [27], [28]]. Patel et al. implemented a Surgical Telemedicine Encounter Checklist to decrease the risk for errors and privacy concerns pre-visit, when starting the visit, and closing the visit [29]. Of importance from the checklist, when starting the visit, it is crucial to understand the patients’ privacy level in their location, to have a plan if there were to be a virtual disconnection, and to comment that gaze may be focused on note taking or chart review [29]. Providers may also direct their attention to literature outside of medicine for an understanding of effective communication skills, such as in business and teaching. Common themes include building rapport and trust and providing clear information [[30], [31], [32]].

The strengths of the study include the prospective design and capturing the critical time period at the beginning of the pandemic where telemedicine was in infancy. Limitations of the study exist. Not all participants filled out the survey in the same time frame from otolaryngology appointment to date the survey was sent, with those having appointments in April having a shorter recall time. Survey sampling and response bias could be a limitation of the study with unknowns of if the responses can be generalized to the whole population. Due to the lack of responses from families in the in-person during telemedicine group, this group had to be excluded from sub-analyses. In addition, due to the mass volume of patients during the study period, a recruitment letter was sent out electronically to caregiver emails on file. Because of this, families may have been hesitant to fill out an electronic consent form. Although there was a low response rate, this was expected during the beginning of the pandemic with uncertainty and priorities taking place during this time period. Approximately 1,000 families had to be excluded from the IBTM group because of no email available with the patients’ charts. Lastly, a majority of the caregivers had a child that identified in the medical charts as white race. Because survey responses were not random, the racial distribution of our responders could bias results and reduce generalizability as well.

The strengths and limitations of our study notwithstanding, this study merits publication in the literature to serve as an evidence-based foundation upon which to improve, strengthen, and to establish telemedicine in pediatric otolaryngology. We anticipate that publications specific to our specialty will prompt professional societies such as ASPO, ESPO, SENTAC and the AAO-HNS to leverage the recent surge in telemedicine experience to develop clinical appropriateness criteria using approaches such as those embraced by the National Commission for Quality Assurance and Choosing Wisely® initiative. This study must be repeated in a prospective manner and appropriately powered to drill down to specific outcome measures of importance.

Determination of the post-pandemic role of telemedicine will be complex and consequential and should be grounded in a value-based approach. It is our hope that this study prompts collaboration from our peers domestically and globally to work together towards that end. We also hope that papers like this unify health care providers in identifying and surmounting obstacles to telemedicine including the lack of a consistent payment strategy, unclear relative advantage and implementation scope, education, and infrastructure investment requirements (such as wireless broadband), lack of experiential foundation, and concerns surrounding fraud and abuse. A silver lining of the COVID-19 pandemic has been the unprecedented adoption of telemedicine paradigms as the “new normal” for clinical care across the United States and beyond.

Conclusion

Our findings assessed if telemedicine led to a relevant change in paradigm and platform for pediatric otolaryngologists for after the pandemic has passed. Satisfaction was rated as “Good” or “Excellent.” However, messaging when surgery is not recommended must be improved to obtain increased caregivers’ agreement in an era of shared decision making.

Funding

The project described was supported by the through Grant Number UL1 TR001857.

Declaration of competing interest

There are no conflicts of interest to report for any author.