The impact of a didactic and experiential learning model on health profession students' knowledge, perceptions, and confidence in the use of telehealth.

BACKGROUND: Training of health profession students in telehealth is important to ensure proper implementation for healthcare delivery. This prospective study aimed to analyze the effects of didactic and experiential learning on knowledge, confidence, and attitudes of telehealth among health profession students (Survey 1). The perceptions of a mixed model telehealth platform were also considered among these students and community clients (Survey 2).
MATERIALS AND METHODS: A quasi-experimental repeated-measure study was conducted on 153 university health profession students in physician assistant, physical therapy, occupational therapy, and nursing (NR) across the 2020-2021 academic year. Survey 1 was administered to students pre/postdidactic telehealth training and at two sequential points within two semesters of telehealth experiential learning. Survey 2 was distributed among students and a pool of 19 community clients at 4 time points across the experience. Survey data were analyzed using R software.
RESULTS: There was a significant improvement in telehealth knowledge, confidence, and attitudes among all student disciplines after the didactic module with marginal means ranging 3.313/5-4.318/5 for pretest to posttest 1. Improvement continued through experiential learning with marginal means ranging 4.170/5-4.369/5 in posttest 3. There was also a significant student and client approval of the telehealth platform with a student mean high of 3.962/5 ± 0.527 and client mean high of 4.727/5 ± 0.238.
CONCLUSION: A didactic training module combined with experiential learning is effective for health profession students' improvement in perception, knowledge, and attitudes toward telehealth. Health profession students and community clients approve a mixed model telehealth platform. Copyright:

Introduction

The Mayo Clinic defines telehealth as “the use of digital information and communication technologies, such as computers and mobile devices, to access healthcare services remotely and manage health care.”[1] Through this delivery of health care and education from one location to another, telehealth can benefit patients, providers, and health systems with increased accessibility, convenience, distribution, and engagement.[2] Telehealth allows providers to collaborate across a variety of settings and across remote distances to improve utilization and patient monitoring.[2] The ability to extend contact with providers and specialty services also helps mitigate health-care access concerns related to provider shortage.[3] Telehealth can increase health-care services for early intervention, management of chronic conditions, prevention of acute episodes, and reduced incidence of hospital readmissions – all of which improve cost-effectiveness of health systems.[2]

Over the past several decades, as telecommunications and information technologies have advanced, telehealth has been evolving as a multidisciplinary, dynamic tool to promote increased access to high-quality, efficient, and effective health care.[3] Studies have shown positive clinical outcomes and patient satisfaction with the use of telehealth;[45] however, full immersion into the health-care landscape was affected by regulation and reimbursement issues.[3] Limitations in traditional face-to-face health service encounters brought on by the COVID-19 pandemic forced health systems to “radically and rapidly rethink the delivery of care.”[67] This led to a significant rise in the use of telehealth platforms for health care in both triage and routine primary care.[89]

According to Lieneck, Garvey, and Collins, et al., this increased use of telehealth has demonstrated its effectiveness for the delivery of both clinical and nonclinical health services while also increasing access to care.[10] Telehealth has allowed safe, cost-effective patient care through the COVID-19 pandemic, and it is likely that the utilization of telehealth technology will continue to expand. Prior to COVID-19, a common barrier to the use of telehealth in clinical practice among providers was a lack of training.[5811] Therefore, it is imperative that students across all health-care disciplines are taught the best practices for the use of telehealth and as Edirippulige and Armfield suggest, telehealth education should be a “standard component” in health-care education.[1213] There is a definite skill set for the use of telehealth and a level of competency that can be reasonably expected for entry-level health-care providers.[1214] Some of these skills include professionalism, communication, the understanding of technological capacity, and appropriate use of telehealth; equitable access and barriers to use; and impact on privacy, patient–provider relationship development, and online etiquette.[1214]

The COVID-19 pandemic necessitated a fundamental change to the format of an established interprofessional education (IPE) curriculum with PA, PT, OT, and Nursing program students. The IPE program typically includes didactic coursework and face-to-face experiential clinical practice training with a community partner to meet the needs of medically underserved clients. In response to limitations brought on by the pandemic, faculty developed and implemented a telehealth didactic training module and rapidly transitioned the experiential component to a mixed model telehealth platform. This was structured specifically to facilitate student growth in IPEC competencies[15] and the development of client relationships while addressing telehealth skills. This study applies a novel curricular and telehealth model developed for the education and training of a variety of health profession students in an IPE course. The purpose of this study was to assess multidisciplinary health profession students' knowledge, attitudes, and confidence in telehealth with the specific application of both didactic and experiential learning. The perceptions of telehealth among students and community clients were also considered based on a mixed model telehealth platform.

Materials and Methods

Setting and participants

As part of a Health Resources and Services Administration Primary Care Training and Enhancement grant (T0BHP29989), university faculty developed an IPE curriculum which collaborates with community partners to improve the health and wellness of seniors in medically underserved areas.[161718] 153 students in graduate-level physician assistant (PA), physical therapy (PT), and occupational therapy (OT) programs, as well as year 4 undergraduate nursing students participated in the study. For the experiential component, interprofessional student teams were paired with clients from the community clinic partner. These client volunteers were recruited by the clinic from previous or current patients. Clients were excluded if they were under the age of 55. Thirteen clients were primary English speaking, while seven were primary Spanish speaking and required an interpreter. Interpreters were either members of the team who were self-assessed as functionally competent in Spanish, university undergraduate Spanish students, or employees of the partnering clinic. Activities were reviewed and approved by the Western Institutional Review Board as standard educational practice under B1 exemption (work order: #1-963419-1).

Didactic curriculum

Students in PA, PT, and OT programs first participated in an interprofessional didactic course where they received education on professionalism, IPEC competencies, integrative health, social determinants of health, health literacy, and communication strategies. Nursing students were similarly prepared for interprofessional collaboration through IPE training modules.[161718] Twenty interprofessional teams were then created with these students. Before the first meeting with their client, the interprofessional teams completed a comprehensive training specifically focusing on telehealth practice. This was delivered by faculty through a synchronous learning experience on Microsoft Teams. Learning objectives included an understanding of the uses for telehealth with respect to the impact of the COVID-19 pandemic. Focus areas included an examination of the ways telehealth can be utilized to address health-care disparities, the ability to recognize factors to protect patient privacy and safety, and effective communication strategies within a telehealth platform. As part of this didactic training, students also participated in a simulated telehealth experience implementing motivational interviewing principles to prepare for the experiential component.

Experiential learning-telehealth design

The twenty interprofessional teams each chose one student to meet with the client face-to-face in the clinic. This person remained the same throughout the experience to limit potential COVID-19 exposures. Adhering to COVID-19 restrictions, this student met with the client at the clinic to assist the client with the technology and to facilitate client-team relationship development with all team members. The remainder of the students on each team joined the client meetings remotely, in a synchronous manner via Microsoft Teams. Within this telehealth format, students were able to work as interprofessional teams to develop their IPEC competencies, effect change for the client, and maintain COVID-19 restrictions.

Table 1 outlines the experiential component schedule. The teams were scheduled to meet with their clients once a week for a total of five weeks in the fall semester and an additional six weeks in the spring semester. Some of the teams were unable to meet with their client, each of the scheduled times due to varied client circumstances including illness, occupational change, travel, and transportation issues. The goal of these client meetings was to develop a client-team relationship via the telehealth platform and take a comprehensive history for the development of a wellness intervention. After the client meeting, each team held a debriefing with their faculty advisor. Topics at these debriefings were intentionally created to include health literacy, motivational interviewing, social determinants of health, SMART goals, interprofessional collaboration, mental health, and client outcomes. In the second half of the fall semester, teams performed three telephonic client check-ins. At the end of the fall semester, teams developed a wellness intervention in collaboration with their clients which were then implemented in the spring semester. Examples of wellness interventions included stress and pain management, strengthening, balance training, weight management through nutritional education, and lifestyle modification. The final product of the 2-semester experience was a professional Poster Presentation presented asynchronously. Faculty advisors for the experience included professors and practicing providers from the represented professions including a certified PA, occupational therapist, physical therapist, and nurse.

Table 1

Schedule details

Week	Fall 2020	Spring 2021
	Survey 1: Pretest
Week 1	Orientation session 1–Introduction to fall activities and team orientation	Info session 1-Assessment and evaluation of intervention outcomes
Week 2	Orientation session 2–Telehealth training and motivational interviewing	Info session 2 Working with an interpreter
	Survey 1: Posttest 1
Week 3	Orientation session 3–Cultural orientation and team building	Client session 1
		Theme: Health literacy
Week 4	Client session 1	Client session 2
	Themes: Health literacy + motivational interviewing	Theme: IP team background (poster)/interprofessional collaboration
		Survey 2: Application 3
Week 5	Client session 2	Info session 3-mental/behavioral health
	Themes: Social determinants of health + helping styles
Week 6	Client Session 3	Client session 3
	Themes: SMART goals + patient motivation	Theme: Mental/behavioral health
	Survey 2: Application 1
Week 7	Client session 4	Client session 4
	Themes: Client assessment/evaluation	Theme: Team interventions (poster)
Week 8	Client session 5	Client session 5
	Client intervention presentation	Theme: Team outcomes (poster)
	Survey 2: Application 2
Week 9	Virtual teamwork meeting 1	Make up client session or virtual teamwork meeting
Week 10	Virtual teamwork meeting 2	Virtual teamwork meeting
		**Telephonic check-in call 1**
Week 11	Virtual teamwork meeting 3	Final poster presentation
Week 12	Virtual teamwork meeting 4	Info session 4-focus group feedback
	**Telephonic check-in call 2**
Week 13	Virtual teamwork meeting 5	Client session 6
		Survey 2: Application 4
Week 14	Final day of classes	Final day of classes
	**Telephonic check-in call 3**
	Survey 1: Posttest 2	Survey 1: Posttest 3

SMART=(Specific, Measurable, Attainable, Relevant, Time-Based)

Survey tools

Survey 1 was adapted from an instrument used by Phillips et al.[19] to assess the impact of a telehealth simulation experience in a graduate-level nursing program. The authors granted permission for the use of the survey. This telehealth assessment was administered before (pretest) and after the didactic experience (posttest 1) as well as at the end of the fall semester (posttest 2) and spring semester (posttest 3) to all student disciplines (PA, OT, PT, and Nursing). The goal of this tool was to assess the impact of didactic and experiential learning on health profession students' attitudes, knowledge, and confidence when using a telehealth platform. Questions were answered anonymously (linked to a code number for paired t-test analysis) on a Likert-type scale with scores from 1 (Strongly Disagree) to 5 (Strongly Agree); a score of 3 was neutral.

Survey 2 was developed to assess the perceptions of clients and students with respect to the equipment used (video, sound), privacy, personal connection, comfort levels, and satisfaction of the mixed telehealth platform. Faculty were also interested in how students and clients were perceiving the comfort and satisfaction of the other as a source of feedback for professional communication skill development within a telehealth platform. A literature review found no single tool to assess each area of interest to the study. Core faculty of the experience-each expert in their health profession field-developed questions with valid existing tools and a telehealth study conducted by Donelan, et al. as models.[20212223] The result was a 12 question Likert-type survey with scores from 1 (Strongly Disagree) to 5 (Strongly Agree); a score of 3 was neutral. The survey questions were reviewed by other experienced health profession faculty to address validity. Reliability was not addressed prior to distribution as the survey was created for purposes specific to this experience; however, the consistency of response among the 4 applications of the survey to the same group of students and clients speaks to the reliability. The survey given to clients was translated to Spanish if the client preferred that as their primary language. The survey was completed anonymously during the experiential phase of the curriculum to both students (N = 153) and participating clients (N = 8–19) following the 3rd and 5th client sessions in the fall semester and 2nd and 6th client sessions in the spring semester.

Statistical analysis

Data from Survey 1 and 2 were analyzed using R software 2019 (R Foundation for Statistical Computing, Vienna, Austria).[2425] Significance was considered at P < 0.05. Survey 1 data were analyzed using a linear mixed-effects model. Due to significant interaction between program and test, post hoc pairwise t-tests were performed. Paired t-tests were performed for the pretest to posttest 1 (to assess the effects of the didactic curriculum), pre-test to posttest 2, and posttest 3 (to assess the effects of didactic curriculum and experiential learning components). In addition, posttest 1 through posttest 3 for all combinations were analyzed to assess the effects of experiential learning components only after the didactic curriculum had been conducted. The Tukey correction was employed.

Data from Survey 2 was analyzed for all students (PA, PT, OT, NR) and for all clients of the community clinic. Data were analyzed separately for each application of the survey (4 total applications as described above). A one-tailed, 95% one-sample t-test was conducted with comparison to neutral, i.e., μ =3 based on Likert-type scale from 1 to 5 with 5 being strongly agree with respect to a positive perception of telehealth. For student data, means pairwise comparisons between applications of Survey 2 were conducted with the Tukey Correction. For client data, an analysis of variance (ANOVA) analysis was conducted to identify any differences between Survey 2 applications.

Results

Survey 1

A total of 153 students participated in the survey. Figure 1 shows the marginal means for each student discipline for Survey 1 from the pretest through posttest 3. Table 2 shows the paired t-tests by student discipline for all combinations of pretest through posttest 3. All four student disciplines showed significant improvement in their knowledge, confidence, and attitudes regarding telehealth from the pretest to posttest 1 (effects of didactic curriculum) and from pretest to posttest 2 and posttest 3 (effects of didactic curriculum and experiential learning components). In contrast, there was no significant improvement in any student discipline when measuring the effects of experiential learning only (after didactic curriculum was conducted).

Figure 1

Marginal means plot for survey 1

Table 2

Paired t-test for survey 1 by student discipline

Comparison	Student discipline	P
Pretest to posttest 1	PA	<0.0005
Pretest to posttest 2	PA	<0.0005
Pretest to posttest 3	PA	<0.0005
Posttest 1 to posttest 2	PA	NS
Posttest 1 to posttest 3	PA	NS
Posttest 2 to posttest 3	PA	NS
Pretest to posttest 1	OT	<0.0005
Pretest to posttest 2	OT	<0.0005
Pretest to posttest 3	OT	<0.0005
Posttest 1 to posttest 2	OT	NS
Posttest 1 to posttest 3	OT	NS
Posttest 2 to posttest 3	OT	NS
Pretest to posttest 1	PT	<0.0005
Pretest to posttest 2	PT	<0.0005
Pretest to posttest 3	PT	<0.0005
Posttest 1 to posttest 2	PT	NS
Posttest 1 to posttest 3	PT	NS
Posttest 2 to posttest 3	PT	NS
Pretest to posttest 1	NR	<0.0005
Pretest to posttest 2	NR	<0.0005
Pretest to posttest 3	NR	<0.0005
Posttest 1 to posttest 2	NR	NS
Posttest 1 to posttest 3	NR	NS
Posttest 2 to posttest 3	NR	NS

NS=Not significant, PA=Physician assistant, PT=Physical therapy, OT=Occupational therapy, NR=Nursing

Survey 2

A total of 153 students participated in the Survey 2. Client survey completion ranged from 19 to 8 based on client availability for telehealth sessions. The means (all questions for all students or clients), standard deviations, number of students or clients, and P values for one-sample t-tests are shown in Table 3 for students and Table 4 for clients. Student results show the only significant difference between applications of Survey 2 was between the second and fourth (showing improvement in the perception of telehealth). For client data, an ANOVA analysis did not show any significant differences between applications of Survey 2. In summary of Survey 2 data, the one-sample t-test for students and clients in all applications showed significant approval of telehealth compared to neutral (μ =3) based on their perceptions of equipment, privacy, personal connection, comfort, and satisfaction.

Table 3

One-sample t-tests for applications of survey 2 to all students

Application	Mean±SD	n	P
1st	3.962±0.527	153	<0.0005
2nd	3.831±0.584	153	<0.0005
3rd	3.960±0.505	153	<0.0005
4th	3.942±0.584	153	<0.0005

SD=Standard deviation

Table 4

One-sample t-test for applications of survey 2 to free clinic clients

Application	Mean±SD	n	P
1st	4.537±0.388	19	<0.0005
2nd	4.537±0.377	12	<0.0005
3rd	4.727±0.238	8	<0.0005
4th	4.504±0.336	12	<0.0005

SD=Standard deviation

Discussion

Literature reviews confirm the importance of IPE and the important benefits of experiential learning curriculums;[2627282930313233] however, there is a paucity in the literature related to consistent telehealth curricular content and integration into multidisciplinary health profession programs.[3435] Many studies are simulation based rather than true experiential with clients, and others did not assess patient satisfaction.[35] The study described here is novel in its assessment of a curricular approach with both didactic and true experiential learning across multiple health-care disciplines through IPE. The telehealth training model used here was a mixed platform to facilitate rapport-building and client satisfaction-this model was assessed and found to be a positive experience for clients and students.

Similar to studies with a focus on medical students,[35] the results of this study demonstrate significant improvement in knowledge, confidence, and attitudes regarding telehealth among PA, PT, OT, and nursing students. Here, the greatest improvement was seen after the didactic training module, which the authors believe expanded students' knowledge in telehealth application and expectations to subsequently foster confidence in the use of telehealth with a positive attitude about the telehealth platform. The structure of this curriculum including both didactic and experiential learning within student interprofessional student aligns with positive education and learning seen in studies with team-based learning, problem-based learning, and multimodal educational strategies.[3637]

The experiential learning phase of the curriculum reinforced the didactic component as seen in the significant improvement in knowledge, confidence, and attitudes regarding telehealth between pretest and Posttest 2 and 3 (during the time of experiential learning) for Survey 1. Rather than seeing fatigue in knowledge development after the didactic component, the experiential component resulted in sustainability of the benefits originally gained.[38]

Several studies speak to telehealth difficulties and decreased satisfaction due to technical challenges, difficulty with patient-provider relationship building, difficulty with assessment, patient discomfort and difficulty “reading patient affect.”[35] In the experiential model developed here, the student teams were uniquely arranged so that the telehealth experience included one interprofessional team member live with the community client at the clinic while the other students joined remotely. Results of Survey 2 indicate that clients and students perceived this telehealth arrangement to be an acceptable and effective format of health-care delivery in their own experience and in their perception of the other's experience.

The authors believe this model allowed for the development of a strong client-team relationship fostered with face-to-face visits, while still having access to multiple interprofessional specialty services through the telehealth platform. The availability of one team member in the room with the client also served to assist with any technological challenges, thereby improving digital literacy and satisfaction with the telehealth experience. This arrangement facilitated a positive health-care experience and overall client satisfaction.

It was noted that in each application of Survey 2, the clients scored higher than the students (although both were statistically significant). There are several potential explanations for this finding. One reason is that clients were recruited for participation and volunteered with knowledge of the telehealth format, and anticipated benefit of a wellness program – this may have led to overall acceptance of the experience and the telehealth format. Another reason may be the structure of the telehealth platform which allowed for technological support, interpreter services, and the development of relationships with their student team members.

Apart from the use of telehealth during the pandemic, it is suggested that future studies continue to expand and assess the application of a standardized telehealth training module and experiential learning across a broad variety of health profession students. Further study suggestions also include the assessment of a standardized telehealth training module with practicing health-care providers to improve both provider and client satisfaction with telehealth services. In addition, assessment of access, efficiency, patient outcomes, provider-patient relationships, and satisfaction within a mixed telehealth platform involving both in-person and remote interprofessional team access should be explored.

Limitations and weaknesses

Student and client participants were not assessed on prior telehealth knowledge or experience before the applied study interventions or survey applications. The disproportionate participant numbers between students and clients as well as the impact of multiple missed client visits, lowered the power of the statistically significant perception surveys and decreased the overall experiential exposure time of the client with the student. This may have altered the Survey 1 results (especially posttests 2 and 3) and Survey 2 results. Another limitation in this study includes the method used for language interpretation. Interpreters included the student liaison who had a self-assessed functional level of knowledge of Spanish, university undergraduate students majoring in Spanish, and partner clinic staff proficient in Spanish. This inconsistency in language interpretation delivery may have impacted the client responses in Survey 2.

Conclusion

A curriculum which couples telehealth didactic training modules with experiential learning is an effective teaching strategy to improve knowledge, attitudes, and confidence in telehealth for multidisciplinary health profession students. This model should be considered to prepare health profession students for entry-level readiness in telehealth – a platform that will undoubtedly be used in their future careers.[35]

Financial support and sponsorship

This project is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number T0BHP29989 titled Primary Care Training and Enhancement. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS, or the U.S.

Conflicts of interest

There are no conflicts of interest.