Brief Report: Telehealth Satisfaction Among Caregivers of Pediatric and Adult Psychology and Psychiatry Patients with Intellectual and Developmental Disability in the Wake of Covid-19.

Telehealth has been shown to be both acceptable and effective in many areas of healthcare, yet it was not widely adopted prior to the SARS-CoV-2 (COVID-19) pandemic. Additionally, previous evaluations of telehealth for autism spectrum condition (ASC) and intellectual and developmental disability (IDD) populations are limited in both number and scope. Here, we investigated satisfaction amongst Psychology and Psychiatry patient caregivers at Cincinnati Children's Hospital Medical Center (CCHMC) after the onset of the COVID-19 pandemic. Results (640 responses) showed high rates of satisfaction across departments, appointment types, and diagnoses, with 92% indicating overall satisfaction with their appointment. There were, however, notable decreases in satisfaction among Group Therapy respondents, and those whose diagnosis was classified as Other.

The SARS-CoV-2 (COVID-19) pandemic prompted the rapid expansion of telehealth across all areas of healthcare (Chen et al., 2020). Pre-pandemic, telehealth—defined in this paper as the provision of healthcare services using interactive technology—was underutilized for a number of reasons that can be categorized into two groups: barriers to access and barriers to adoption. Barriers to telehealth access include strict regulations, low/no reimbursement, and location and/or technology requirements (Keesara et al., 2020). Barriers to adoption include questions of whether diagnosis or treatment will suffer, as well as whether patients, caregivers, and providers will accept and be satisfied with telehealth. Temporary regulatory and reimbursement changes in support of telehealth practice during the pandemic have effectively diminished the biggest barriers to access (Chen et al., 2020) and subsequently telehealth, a previously infrequently used practice, has seen widespread adoption in hospitals across the U.S (Fischer et al., 2021). While enhanced telehealth is often viewed as a positive (if not overdue) change by many, more research is needed with respect to the barriers to telehealth adoption, particularly among diagnostic and demographic groups that have been historically underserved in healthcare.

Multiple systematic reviews on psychiatric telehealth have agreed that both assessment and treatment outcomes are comparable to in-person care across diagnoses and in diverse populations(Bellanti et al., 2021; Drago et al., 2016), with high satisfaction rates found among both telehealth patients and providers for a variety of services (Chen et al., 2020; Kruse et al., 2017). Another recent meta-analysis found that video-based cognitive behavioral therapy (CBT) delivered to adults was at least as effective as CBT provided face-to-face (Norwood et al., 2018), and the same was found for teletherapy combined with medication management (Chakrabarti, 2015). Another review concluded that telehealth psychiatric consultations—typically for medication management—are a feasible and effective mode of treatment in child and adolescent cohorts (American Academy of et al., 2017). Several other types of clinical encounters, including cognitive processing therapy, group therapy, family therapy, exposure and response prevention, and rehabilitation of those with chronic conditions have shown to be as effective as face-to-face interventions when delivered via telehealth (Chakrabarti, 2015; Gentry et al., 2018).

While it’s been established that telehealth is useful for a wide range of mental health conditions, many specific conditions have yet to be adequately investigated, particularly in youth (Cain & Sharp, 2016). In addition, it’s been well-documented even before the COVID-19 pandemic that individuals with autism spectrum condition (ASC) and/or intellectual and developmental disability (IDD) face steep barriers to adequate healthcare (Lindly et al., 2019; Malik-Soni et al., 2021; Whittle et al., 2018). The financial cost carried by families raising children with ASC is considerable, which may further contribute to inequity in care and service access with an estimated one third of families spending more than 3% of their annual income on services for their affected child (Parish et al., 2012). Moreover, caregivers of children with both ASC and concurrent IDD were more likely than those with only one of the conditions to report having to cut work hours and to stop working altogether (Saunders et al., 2015). Additional constraints include traveling long distances to specialty clinics, which can exacerbate severe behaviors and safety risks for both child and family during transport and while in clinics, further limiting access to psychiatric and psychological services for those who need these services the most. The coupling of these added difficulties is thought to cause greater overall negative impact on these families (Saunders et al., 2015). Access to specialized care in ASC is critical yet still very limited in many areas. Improving this access through telehealth options could not only reduce caregiver stress but also provide significant returns on individual and family quality of life.

While limited in scope, existing reviews focused on telehealth for persons with ASC have concluded that it is a promising delivery approach for both diagnosis and treatment of the condition (La Valle et al., 2022; Sutherland et al., 2018). Several newly published studies back these findings, with moderate to high satisfaction reported following diagnostic evaluations (Adamou et al., 2021; Jang et al., 2021) and treatment (Lynch et al., 2021; Pollard et al., 2021; Srinivasan et al., 2021) for those with ASCs. Unfortunately, relatively few studies or reviews have been conducted to determine the satisfaction of telehealth among individuals who have IDDs. A recent review on telepsychiatry use for children and adults with IDDs revealed high rates of satisfaction among parents and caregivers as well as a significant decrease in emergency room visits in the 12 months following tele-treatment, which translated to significant cost savings for both patients and healthcare providers (Madhavan, 2019). Despite these promising results, more studies are needed to examine if telehealth is an acceptable approach for those with ASC and IDDs.

The goal of the current study is to further investigate telehealth satisfaction and acceptability within ASC/IDD populations during the COVID-19 pandemic. For the purpose of this study, telehealth will refer to appointments completed remotely using video interfacing technology. We examined caregiver responses to a 10-question satisfaction survey completed after a variety of telehealth visits in the neurobehavioral psychiatry and psychology departments at Cincinnati Children’s Hospital Medical Center (CCHMC) during the COVID-19 pandemic. In addition to exploring general acceptability, we examine rates of acceptability for different types of visits including medication management, behavioral therapy, and psychological evaluations.

Methods

Potential participants were identified using CCHMC’s electronic medical record system. All patients who had psychology or psychiatry telehealth appointments using video conferencing software with neurobehavioral psychologists, clinical counselors, clinical social workers, psychiatrists, or advanced practice nurses starting May 13, 2020, were considered eligible for participation. All patients treated in these specialty clinics have diagnoses of ASC, ID, genetic conditions, or dual psychiatric diagnoses in addition to ID/DD. There were no typically developing individuals included in survey distribution. It should also be noted that, though CCHMC offers primarily pediatric care, care is sometimes extended into adulthood when appropriate based on factors including rare diagnoses, specialized patient needs, and specialized care provided by clinicians. For example, those surveyed included patient caregivers from a specialized Fragile X Syndrome (FXS) clinic that offers lifetime care.

Each business day, survey invitations were sent out via email by designated staff to all patients who were seen in an eligible telehealth appointment the previous business day. Types of telehealth appointments included behavior therapy (including parent training, child treatment with caregivers present, cognitive behavioral therapy, social skills, applied behavioral analysis), group therapy (with a concurrent caregiver group), medication management, and psychology evaluations (diagnostic interviews or feedback sessions only). The standardized emails contained a code and a link to an anonymized survey using REDCap (Research Electronic Data Capture), a secure, web-based application designed to support data capture for research studies. No names or identifying information are linked to the codes. CCHMC human subjects IRB approval was obtained to retrospectively examine the survey data. As of January 22, 2021, a total of 7,306 survey invitations were sent out and 646 survey responses were submitted. Of the completed surveys, six were excluded due to responses reflecting an appointment outside of the departments being examined based on respondent comments, leaving 640 surveys included in the analysis. Additionally, of the remaining 69 surveys with Other indicated for appointment type, 54 were re-categorized by staff prior to analysis based on the caregiver comments which matched a more specific appointment type option. The remaining 15 surveys that marked Other for appointment type did not provide any additional explanation and therefore could not be placed in a more accurate category.

Survey

The survey questions were designed through collaboration by researchers at Cincinnati Children's Hospital Medical Center, Cincinnati, OH (CCHMC) and CHOC Children’s Hospital, Orange County, CA. The survey was created by utilizing past telehealth survey examples to generate questions and adding questions that were most helpful to our specific site. To reduce response bias, the questions were created to be objective, with a range of response options, and not be leading toward specific responses. Additionally, clinicians were purposefully not included in the survey completion in order to reduce any possibility of participants feeling pressured to respond in a specific way. This also reduced clinician burden in the midst of a very difficult period of the pandemic. Efforts to increase response rates were utilized including anonymous completion and easy accessibility through REDCap. The present study focuses only on data collected at CCHMC. The initial REDCap survey contained 2 demographic questions regarding the reason for appointment and frequency of telehealth use and 7 questions regarding satisfaction and privacy, as well as a field for comments. A later version of the survey added 3 demographic questions including patient’s age in years, diagnosis, and gender as well as 1 additional satisfaction question regarding future preference for telehealth vs. in person visits (Table 1).

Table 1

Survey questions, response options, and indication of when each question was added

Part of original survey or version 2	Survey question	Response options
Version 2	Child’s age	Free response
Version 2	Child’s gender	Male Female Other/prefer not to say
Version 2	Child’s diagnosis	Free response
Original	(1) Check the reason for your telehealth appointment	Psychology evaluation Behavior therapy appointment Group therapy Medication management other
Original	(2) How many times have you seen a provider via telehealth?	This is my first time 2–5 times More than 5 times
Original	(3) Overall, I am satisfied with my telehealth appointment	Strongly agree Agree Neutral Disagree Strongly disagree
Original	(4) I felt that things were kept in private	Strongly agree Agree Neutral Disagree Strongly disagree
Original	(5) How did today’s telehealth appointment compare to previous in person appointments?	Much better Better Same Worse Much worse
Original	(6) It was easy for me to state my concerns and ask questions through telehealth	Strongly agree Agree Neutral Disagree Strongly disagree
Original	(7) I had to wait less time for the telehealth appointment than I do for one that is in person	Strongly agree Agree Neutral Disagree Strongly disagree
Original	(8) I would use telehealth to have my child’s provider see us again	Strongly agree Agree Neutral Disagree Strongly disagree
Original	(9) I will tell the people I know about telehealth	Strongly agree Agree Neutral Disagree Strongly disagree
Version 2	(10) If you had to choose between telehealth and in person visits, what would you choose?	Telehealth In person Mix of the two

Analysis

To control for partially completed surveys and questions not presented due to changes in survey version, analysis for each question included all surveys that provided an answer to that question regardless of overall survey completion. Microsoft Excel 2013 was used for all descriptive statistics. Survey data was analyzed in total, as well as by department (psychology or psychiatry), appointment type, times used, and diagnoses. Further comparative analysis of questions 3–9 was completed for appointment type, diagnoses, and times used with the use of Kruskal–Wallis Test followed by pairwise comparison using Dunn's Test with Bonferroni adjustments for multiple comparisons using R version 4.1.0 (Core Team, 2014). Satisfaction questions 3–9 were further analyzed by department using Wilcoxon-Mann–Whitney pairwise comparisons using R. Question 10 contained only categorical data and was analyzed by department and times used with Chi-Square Test of Independence in R. Question 10 was not evaluated by diagnosis or appointment type due to some groups not reaching the minimum cell count for valid statistical analyses. When reported in demographics, surveys that included multiple diagnostic categories were counted in all categories noted. In analysis, surveys were counted only in the category determined to be the primary diagnostic category. Categories included ASC, FXS, Psychiatric Conditions, Developmental Conditions, Genetic Conditions, and Other. Because the patients who received survey invitations included a specialized FXS clinic, this diagnosis was examined in its own right rather than being grouped into the Genetic Conditions category. Comments were grouped into categories of “positive”, “negative”, and “both/neutral”. Comments were reviewed by two independent raters and in the case of discrepancies, ratings were reviewed by a third more senior rater to arrive at a final rating.

Results

Demographics

Of the 640 surveys analyzed, 367 indicated the subject’s gender with 286 of 367 (78%) male, 80 of 367 (22%) female and 1 of 367 (< 1%) other/prefer not to say (Table 2). 364 surveys indicated the patient’s age in years with a mean age of 13.37, a median age of 13, and a range from 2 to 50 (Table 2). Of the 364 surveys with age included 291 (80%) were pediatric (ages 2–17 years) and 73 (20%) were adult patients (ages 18–50 years). Diagnosis was reported on 302 surveys. The most common diagnostic category was ASC (n = 166; 55%), followed by Psychiatric Conditions (n = 116; 38%), Developmental Conditions (n = 37; 12%), FXS (n = 19; 6%), and Other Genetic Conditions (n = 13; 4%) (Table 2). The remaining surveys with a response provided (n = 41; 14%) were categorized as Other either because the diagnosis listed did not accurately fall into any of the categories above, or because the response contained information other than a diagnosis such as a specific behavioral concern or symptom (Table 2).

Table 2

Patient demographics

	Total	Psychology	Psychiatry
Age	N = 364	N = 158	N = 206
Mean (median)	13.37 (13)	9.19 (9)	16.57 (15)
Range	2–50	2–24	5–50
Sex	N = 367	N = 159	N = 208
Male: % (n)	78% (286)	78% (124)	78% (162)
Female: % (n)	22% (80)	22% (35)	22% (22)
Other/prefer not to say: % (n)	< 1% (1)	0% (0)	< 1% (0)
Diagnosis	N = 302	N = 122	N = 180
ASC: % (n)	55% (166)	48% (58)	60% (108)
FXS: % (n)	6% (19)	0% (0)	11% (19)
Other genetic: % (n)	4% (13)	5% (6)	4% (7)
Psychiatric: % (n)	38% (116)	43% (52)	36% (64)
Developmental: % (n)	12% (37)	18% (22)	8% (15)
Other: % (n)	14% (41)	16% (19)	12% (22)

Previous Experience with Telehealth and Reason for Appointment

187 of 638 (29%) responses, indicated that the interaction was their first time using telehealth. 310 of 638 (49%) respondents indicated they had used telehealth 2–5 times in the past, and 141 of 638 (22%) respondents reported having used telehealth more than 5 times in the past. 622 surveys included responses for appointment types, the most common being Medication Management (n = 253; 41%), followed by Behavior Therapy (n = 207; 33%), Psychology evaluation intakes or feedback sessions (n = 118; 19%), and Group Therapy (n = 29; 5%). Only 15 of 622 (2%) respondents reported an appointment type of Other (as indicated by respondent as explained above) that did not include an explanation allowing for re-categorization into a more accurate appointment type.

Reported Satisfaction

Satisfaction rates were generally very high (Table 3). In response to the statement “Overall, I am satisfied with my telehealth appointment”, 590 of 638 (92%) respondents agreed or strongly agreed. 618 of 636 (97%) respondents agreed or strongly agreed with the statement “I felt things were kept in private”. When comparing the appointment to previous in person appointments, 581 of 628 (93%) respondents indicated the telehealth interaction was the same (n = 380; 61%), better (n = 107; 17%), or much better (n = 94; 15%). 597 of 636 responders (94%) agreed or strongly agreed with the statement “It was easy for me to state my concerns and ask questions”. 473 of 635 (74%) respondents agreed or strongly agreed with the statement “I had to wait less time for the telehealth appointment than I do for one that is in person”. Of 636 respondents, 556 (87%) agreed or strongly agreed with the statement “I would use telehealth to have my child’s provider see us again”. In response to the statement “I will tell people I know about telehealth”, 523 of 634 (82%) respondents agreed or strongly agreed. Finally, in regard to future choice of telehealth or in person visits, 125 of 367 (34%) respondents indicated they would choose telehealth alone, 187 (51%) indicated they would opt for a mix of the two, and 55 (15%) indicated they would choose in person only.

Table 3

Satisfaction overall, by department, and by appointment type

Survey question	Satisfaction criteria	All% (n of N)	Departments		Appointment types
			Psychology% (n of N)	Psychology% (n of N)	Psych Evaluation% (n of N)	Behavior therapy% (n of N)	Group therapy% (n of N)	Medication management% (n of N)	Other% (n of N)
(3) Overall, I am satisfied with my telehealth appointment	Agree or strongly agree	92% (590 of 638)	93% (224 of 242)	92% (366 of 396)	91% (10 of 1177)	94% (194 of 206)	79% (23 of 29)	93% (236 of 253)	93% (14 of 15)
(4) I felt that things were kept in private	Agree or strongly agree	97% (618 of 636)	97% (235 of 242)	97% (383 of 394)	97% (113 of 117)	97% (200 206)	93% (27 of 29)	98% (247 of 251)	100% (15 of 15)
(5) How did today’s telehealth appointment compare to previous in person appointments?	Same, better, or much better	93% (581 of 628)	92% (218 of 237)	93% (363 of 391)	93% (109 of 117)	91% (183 201)	86% (24 of 28)	94% (235 of 250)	93% (13 of 14)
(6) It was easy for me to state my concerns and ask questions through telehealth	Agree or strongly agree	94% (597 of 636)	93% (224 of 242)	95% (373 of 394)	95% (112 of 118)	93% (191 of 206)	86% (25 of 29)	97% (243 of 251)	73% (11 of 15)
(7) I had to wait less time for the telehealth appointment than I do for one that is in person	Agree or strongly agree	74% (473 of 635)	77% (186 of 241)	73% (287 of 394)	74% (86 of 117)	74% (151 of 205)	66% (19 of 29)	77% (193 of 251)	93% (14 of 15)
(8) I would use telehealth to have my child's provider see us again	Agree or strongly agree	87% (556 of 636)	82% (198 of 242)	91% (358 of 394)	85% (100 of 117)	86% (177 of 206)	86% (25 of 29)	90% (227 of 251)	100% (15 of 15)
(9) I will tell the people I know about telehealth	Agree or strongly agree	82% (523 of 634)	79% (192 of 242)	84% (331 of 392)	80% (94 of 117)	81% (167 of 206)	76% (22 of 29)	84% (210 of 249)	93% (14 of 15)
(10) If you had to choose between telehealth and in person visits, what would you choose?	Telehealth or mix of the two	85% (312 of 367)	89% (142 of 160)	82% (170 of 207)	88% (72 of 82)	87% (97 of 112)	89% (24 of 27)	83% (111 of 83)	67% (2 of 3)

When evaluating responses by department (Psychology vs. Psychiatry) (Table 3), appointment types (Table 3), and diagnoses (Table 4), satisfaction rates remained similar. It is worth noting that there were slight decreases in satisfaction across most questions for Group Therapy appointments. Even so, the Group Therapy satisfaction rates remained high (Table 3). There was also a notable decrease in agree or strongly agree responses to the statement “It was easy for me to state my concerns and ask questions through telehealth” for respondents with an appointment type of Other, with just 11 out of 15 (73%) respondents agreeing or strongly agreeing compared to 597 of 636 (94%) respondents when looking at total survey responses. This was, however, a very small subset of respondents given that surveys classified as Other for appointment type made up only 15 out of the 636 (2%) surveys that provided a response to this question.

Table 4

Satisfaction overall and by diagnosis

Survey question	Satisfaction criteria	All% (n of N)	Diagnoses
			ASC% (n of N)	FXS% (n of N)	Other genetic% (n of N)	Psychiatric% (n of N)	Developmental% (n of N)	Other% (n of N)
(3) Overall, I am satisfied with my telehealth appointment	Agree or strongly agree	92% (590 of 638)	93% (150 of 161)	95% (18 of 19)	92% (12 of 13)	93% (63 of 68)	93% (14 of 15)	73% (19 of 26)
(4) I felt that things were kept in private	Agree or strongly agree	97% (618 of 636)	98% (156 of 160)	100% (19 of 19)	100% (13 of 13)	100% (67 of 67)	93% (14 of 15)	88% (23 of 26)
(5) How did today’s telehealth appointment compare to previous in person appointments?	Same, better, or much better	93% (581 of 628)	94% (148 of 157)	89% (17 of 19)	92% (11 of 12)	99% (66 of 67)	93% (14 of 15)	88% (23 of 26)
(6) It was easy for me to state my concerns and ask questions through telehealth	Agree or strongly agree	94% (597 of 636)	94% (152 of 161)	95% (18 of 19)	92% (12 of 13)	99% (66 of 67)	93% (14 of 15)	92% (24 of 26)
(7) I had to wait less time for the telehealth appointment than I do for one that is in person	Agree or strongly agree	74% (473 of 635)	73% (115 of 158)	79% (15 of 19)	62% (8 of 13)	78% (53 of 68)	73% (11 of 15)	65% (17 of 26)
(8) I would use telehealth to have my child’s provider see us again	Agree or strongly agree	87% (556 of 636)	91% (146 of 160)	89% (17 of 19)	92% (12 of 13)	90% (60 of 67)	87% (13 of 15)	58% (15 of 26)
(9) I will tell the people I know about telehealth	Agree or strongly agree	82% (523 of 634)	84% (133 of 159)	89% (17 of 19)	77% (10 of 13)	90% (60 of 67)	80% (12 of 15)	77% (20 of 26)
(10) If you had to choose between telehealth and in person visits, what would you choose?	Telehealth or mix of the two	85% (312 of 367)	87% (140 of 161)	63% (12 of 19)	92% (12 of 13)	90% (61 of 68)	87% (13 of 15)	88% (23 of 26)

Satisfaction rates remained high across all diagnostic categories (Table 4), though we did note a stronger preference for in person only appointments in the FXS group (7 of 19; 37%) compared to total survey respondents (55 of 367; 15%). There also appeared to be a decrease in respondents indicating that they would use telehealth to see their provider again among those whose diagnosis was categorized as Other. However, this is difficult to interpret as we did not see a similar decrease in response to preference for choosing telehealth or mix of telehealth and in person for question 10 for surveys with a diagnostic category of Other.

Comparative Analysis

When comparing diagnostic categories, a Kruskal–Wallis H test revealed significant differences across groups for survey questions 3 (overall satisfaction), question 4 (privacy), and question 8 (use again) (Table 5). In pairwise comparisons, using Dunn’s Test with Bonferroni Correction for multiple comparisons, we saw decreased satisfaction among the Other group compared to both the ASC and Psychiatric groups for questions 3, 4, and 8 and compared to the FXS group for questions 3 and 8 (Table 5).

Table 5

Examination of diagnostic group differences

Survey question	Kruskal–Wallis H test	Dunn’s test w/Bonferroni correction
		ASC/FXS	ASC/psychiatric	ASC/genetic	ASC/developmental	ASC/other
(3) Overall, I am satisfied with my telehealth appointment	χ2(5) = 15.16 p = 0.01	p = 0.52 p adj = 1.00	p = 0.72 p adj = 1.00	p = 0.56 p adj = 1.00	p = 0.41 p adj = 1.00	p < 0.01 p adj = 0.01
(4) I felt that things were kept in private	χ2(5) = 14.87 p = 0.01	p = 0.82 p adj = 1.00	p = 0.90 p adj = 1.00	p = 0.36 p adj = 1.00	p = 0.16 p adj = 1.00	p < 0.01 p adj = 0.01
(5) How did today's telehealth appointment compare to previous in person appointments?	χ2(5) = 8.03 p = 0.16	–	–	–	–	–
(6) It was easy for me to state my concerns and ask questions through telehealth	χ2(5) = 9.17 p = 0.10	–	–	–	–	–
(7) I had to wait less time for the telehealth appointment than I do for one that is in person	χ2(5) = 7.69 p = 0.17	–	–	–	–	–
(8) I would use telehealth to have my child’s provider see us again	χ2(5) = 17.75 p < 0.01	p = 0.67 p adj = 1.00	p = 0.78 p adj = 1.00	p = 0.27 p adj = 1.00	p = 0.52 p adj = 1.00	p < 0.01 p adj < 0.01
(9) I will tell the people I know about telehealth	χ2(5) = 3.96 p = 0.56	–	–	–	–	–

Bolded indicates significant p value for Kruskal-Wallis H test or significant p adj value on Dunn’s Test with Bonnferroni correction

χ(5) = 15.16

= 0.01

< 0.01

= 0.01

χ(5) = 14.87

= 0.01

< 0.01

= 0.01

χ(5) = 17.75

< 0.01

< 0.01

< 0.01

χ(5) = 15.16

= 0.01

p = 0.69

p = 0.37

< 0.01

= 0.04

p = 0.47

χ(5) = 14.87

= 0.01

p = 0.77

p = 0.38

p = 0.35

p = 0.03

p adj = 0.45

p = 0.42

χ(5) = 17.75

< 0.01

p = 0.81

p = 0.58

p = 0.24

< 0.01

= 0.03

p = 0.65

χ(5) = 15.16

= 0.01

p = 0.34

< 0.01

= 0.01

p = 0.89

p = 0.09

p = 0.11

χ(5) = 14.87

= 0.01

< 0.01

= 0.02

p = 0.09

p adj = 0.06

p = 0.31

χ(5) = 17.75

< 0.01

p = 0.23

< 0.01

< 0.01

p = 0.59

p = 0.32

p adj = 0.48

p = 0.11

Significant differences were detected across appointment types for question 6 (ease of stating concerns and asking questions) using a Kruskal–Wallis H Test (χ2(4) = 9.70, p = 0.046), but no significant differences were detected across appointment types in pairwise comparisons after adjustments were applied. Therefore, it appears satisfaction was consistent across appointment types. When comparing department types (psychology vs psychiatry), pairwise comparison using the Wilcoxon–Mann–Whitney revealed significant differences for question 8 (will use again) and question 9 (will tell people I know) with Psychiatry showing stronger preference towards future use and telling people they know about telehealth (question 8: U = 53,703, p < 0.01; question 9: U = 53,295, p < 0.01).

When using a Kruskal–Wallis H Test to compare satisfaction across number of times telehealth had been used, no significant differences in satisfaction were reported across those who had been seen using telehealth for the first time, 2–5 times, or more than 5 times. When looking at preference for future visits between telehealth, in-person, or mix of the two, Chi-Square Test of Independence was significant for both department (χ2 = 5.84, p = 0.016) and times used (χ2 = 8.35, p = 0.015), indicating that both department and previous telehealth use have an overall effect on choice to use telehealth again. Due to unequal and low sample sizes post hoc analyses were not completed for the results of the Chi-Square tests, but from visual inspection it appears those seen in Psychiatry have a stronger preference for telehealth in some capacity than those seen in Psychology and that preference for continued use of telehealth in some capacity increased with number of times telehealth was used (Table 6).

Table 6

Responses to Future Visit Preference by Department and Times Used

Choice	Department		Times used
	psychology% (n of N)	Psychiatry% (n of N)	first time% (n of N)	2–5 times% (n of N)	> 5 times% (n of N)
In Person	58% (93 of 160)	45% (94 of 207)	58% (54 of 93)	54% (94 of 174)	39% (38 of 98)
Telehealth	31% (49 of 160)	37% (76 of 207)	23% (21 of 93)	37% (64 of 174)	40% (39 of 98)
Mix of the two	11% (18 of 160)	18% (37 of 207)	19% (18 of 93)	9% (16 of 174)	21% (21 of 98)

Respondent Comments

In total 210 (33%) surveys contained comments. Comments were separated into categories of “positive”, “negative”, and “both/neutral”. Of 210 comments the two raters were in agreement for 207 of them (99%). Only 3 comments (1%) ratings had to be decided by the third more senior rater. 127 (60%) comments were rated as positive. The most common items listed in positive comments were decreased commutes and wait times, less anxiety in patients due to being able to avoid the hospital setting, and the ability for the provider to see the patient’s typical behavior in the home. Many positive comments thanked specific providers and/or generally commented that they liked telehealth. 39 (19%) comments were rated as negative. The most common items listed in negative comments were technology difficulties, long waiting times, or difficulty keeping the patient engaged. 44 (21%) of comments were rated as both/neutral. These comments either contained both a positive and negative statement about the appointment (Ex. “At times the connection would be a little off, but other than that things were great! Thank you so much!:)”), mentioned that the appointment went well, but they still would like in person visits or believe the quality of care is better at in person visits (Ex. “Worked well but in person is always optimal. It is nice to have telehealth as a substitute”), or the comment was either a clarification about a survey answer or unrelated to the quality of the appointment.

Discussion

This study sought to examine whether caregivers of children and adults with ASC and/or IDDs felt satisfied with the care they received via telehealth during the COVID-19 pandemic, and if they feel continued use of telehealth would be potentially acceptable moving forward. In summary, we saw high rates of satisfaction across departments, appointment types, diagnoses and previous use, in line with previous reviews’ findings (Chen et al., 2020; Hilty et al., 2013; Kruse et al., 2017); however, satisfaction was lower than the average for almost all questions for Group Therapy participants (Table 3), which was not seen in other studies that looked at group therapy in neurotypical populations (Gentry et al., 2018). Our group treatment focuses mainly on social skills or emotion dysregulation and responses may reflect obstacles to socialization that can come with video interfacing technology such as lack of eye contact, difficulty for group leaders to monitor individual engagement, muted video and/or audio among other participants, inability to see all group members on the screen at once, delays in audio interface, or other technical issues.

Comparing across departments, Psychiatry patient caregiver responders showed statistically significant stronger preferences in response to questions about if they would use telehealth again and if they would tell people they know about telehealth compared to Psychology responders—a difference in preference not reported in any studies we reviewed. Because of several differences between the two departments, these results are heavily confounded and difficult to interpret. Differences include provider type (only psychologists in Psychology, but psychiatrists, nurse practitioners, and clinical counselors or social workers in Psychiatry), typical visit duration, visit types, and severity of patient needs. Psychiatry tends to see patients with more severe needs, meaning it may be more feasible for caregivers to be seen virtually versus having to transport their child and wait for appointments in a clinic setting.

When responses were separated by specific diagnoses, satisfaction rates remained similar across all diagnostic categories indicating that most diagnoses were satisfied with the care they received. However, the Other group appeared less satisfied compared to all other diagnostic categories on most questions, but because Other is not clearly defined, this is difficult to interpret. Additionally, most respondents across diagnoses indicated they would be open to continued telehealth care. Though not statistically significant, it is worth noting that families affected by FXS indicated slightly lower rates of openness to continued care via telehealth with a higher preference for in person only care (7 of 19; 37%) compared to total survey respondents (55 of 367; 15%). This may reflect the fact that families affected by FXS typically attend appointments at CCHMC every 6–12 months and often participate in multidisciplinary care and/or research in a single visit through a specialized FXS Research and Treatment Center at CCHMC, which was not possible in telehealth given the nature of the research studies. Further evaluation is needed with this specific group to understand the need and preference.

Looking at satisfaction based on previous number of telehealth encounters, we did not see any significant differences between first time users, those who had used telehealth 2–5 times, or those who had used it more than 5 times. However, we did see a significant relationship between number of times telehealth was used and the choice to use telehealth in the future with those with previous telehealth experience appearing to have an increased preference for future telehealth use.

Our analysis shows that telehealth is viewed as a well-liked and acceptable approach among caregivers of pediatric and adult patients with ASC and IDDs, adding further support to existing research (Boisvert et al., 2010; Madhavan, 2019; Sutherland et al., 2018). Moreover, comments received in response to our survey directly state several previously known advantages of telehealth over traditional in-person visits, including decreased travel costs and time, improved access to providers, and greater overall convenience. While these advantages apply to several patient populations, they may be of even greater benefit to families of those with ASC and/or IDDs. For example, children with ASC typically have higher utilizations of primary care (pediatric visits), specialty care (e.g. psychiatric visits, neurology visits), and acute care (emergency department encounters, hospitalizations) than children without ASC (Cummings et al., 2016), and this increase in encounters can cause significant logistical and financial burdens for caregivers and families. By replacing at least some of these encounters with telehealth appointments, these burdens may be reduced. Several responders also commented on perceived benefits related to providers being able to see how their child behaved in the home, and reduced anxiety for children/adults who have difficulty in new places or medical settings, which can often be triggering and lead to additional behavior difficulties for some.

Although the majority of comments received were positive, there were also negative, mixed, or neutral responses. The most common negative comments pertained to technical difficulties. To have a successful telehealth visit, not only does a clinician and patient need to be present, but also many discrete pieces of technologies need to work correctly. Weaknesses in this chain include first-generation telemedicine clients, poor internet bandwidth, and lack of appropriate audio/visual equipment. This points to a need for better technical support, better access to technical resources, and/or clearer communications with patients and caregivers about technical needs for the visit. Since the COVID-19 pandemic, telehealth appointments in the psychology and psychiatry departments at CCHMC have been predominantly in home appointments with no technology access assistance or ancillary staff to aid with technology issues. Perhaps an introduction of these methods in the future may alleviate the technology frustrations that respondents commented on. Several comments also mentioned being happy with the care they received over telehealth during the pandemic, while also indicating that they still feel in person visits are better overall. This points to the possibility that survey results may have been inflated due to caregivers feeling telehealth was either the only option, or only “safe” option for their appointment, and therefore rating the service in the context of the pandemic and not how they would feel were they to use the service outside of the scope of the COVID-19 pandemic.

It is worth noting that 451 of 638 (71%) respondents indicated that this was not their first telehealth interaction. In the greater Cincinnati area, in person neurodevelopmental mental healthcare appointments began transitioning to telehealth in March 2020 with some limited telehealth care in the years prior to the pandemic, but survey collection did not start until mid-May 2020 and was collected through January 2021. While it is possible many survey respondents may have used telehealth in the past outside of the scope of the COVID-19 pandemic, we believe the majority of survey respondents likely had their first telehealth interaction after the onset of the COVID-19 pandemic.

Limitations

Our survey findings must be taken in the context of limitations of this work. First, it should be noted that only 646 of 7,306 (9%) invited to complete the survey did so, and we cannot speculate about the satisfaction rates among those who did not respond. Given the low response rate it is difficult to generalize conclusions made. Additionally, the anonymous nature of the questionnaire prevents us from identifying those who may have completed the survey more than once. For this reason, the reported statistics should be considered with caution as independence of data points cannot be assumed. Our results indicate that telehealth is well-liked among caregivers of pediatric and adult psychology and psychiatry patients, but the study also has limitations in that race, ethnicity, socioeconomic status (SES), and other important demographic information was not collected or assessed for patients or caregivers. Therefore, we were unable to evaluate the relationship between these important factors and caregiver satisfaction. Another limitation was the widely variable sample sizes across groups analyzed, especially in regard to diagnosis. Though there were multiple questions in which the ASC, FXS, and Psychiatric diagnostic groups were significantly more satisfied compared to the Other group, this may be an effect of sample size and outlier results rather than a true difference.

Additionally, diagnosis and appointment were reported by caregivers only and did not have input from clinicians, which may have led to inaccurate reporting of appointment types or diagnoses. For example, 29 (8%) respondents indicated that their appointment was with psychiatry but identified their appointment type as psychology evaluation even though there are no psychology evaluation appointments in the psychiatry department. Because surveys remained anonymous, responses were analyzed in the categories indicated by the caregiver, even when these categories conflicted with our knowledge of appointment types in each department. Additionally, behavior therapy encompassed many types of interventions and future studies would benefit from further exploring each of these types of treatment. Because clinicians were excluded from the survey, we also were not able to evaluate clinician satisfaction or if there were any discrepancies between patient and clinician acceptability of telehealth.

Future Directions

This study took place in the context of the COVID-19 pandemic which created a unique experience with increased stress for both families and clinicians. Once the pandemic is over, the need for access to telehealth will still exist and many providers are advocating for its continued use and expansion. For these reasons, examination of telehealth will need to occur again, outside the confines of the pandemic regarding satisfaction, effectiveness, acceptability, and feasibility to determine if continued use of telehealth is a viable option for specific patient populations and demographic groups long-term. In order to evaluate telehealth as a viable method for continued care, future studies should continue to probe satisfaction among caregivers outside of the scope of the pandemic and examine the effectiveness of telehealth more scrupulously in comparison to, and in tandem with, other treatment models. Future studies should also identify ways to increase response rate by possibly including incentives for completion or encouraging completion immediately post treatment. Additionally, future surveys should include a wider range of demographic information in order to gain a better understanding of participants and caregivers. Finally, a broader understanding of satisfaction could be gained by surveying the patients themselves, as well as the patient providers, in addition to caregivers.

It is also important to consider the impact caused by barriers to access to telehealth and to advocate for laws and regulations that aim to meet families where they are, including continued coverage by insurance companies, continued expansion of coverage regions, and more relaxed technology requirements. In addition, reciprocity between states and/or broader licensing across state lines by boards of medicine and psychology is needed for those with ASC/IDD to access specialized care. Income, educational attainment, age, race, ethnicity and geographic location have all been identified as factors that have limited broadband uptake (Silva et al., 2018). Yet, access to high-speed internet is crucial for successful use of telehealth. As the use of telehealth continues to expand, it is also of the utmost importance to evaluate the influence of factors such as race, ethnicity, and SES on efficacy and accessibility. The field must strive to implement practices that address barriers to equitable care such as access to high-speed internet, which may be unavailable to minority and low-SES group.