Lessons Learned from the Development and Implementation of Virtual and Telehealth Interprofessional Educational Clinics.

PURPOSE: The Interprofessional Educational (IPE) Clinic at Duke is a clinical experience that has allowed an interprofessional team, including health professions students, to care for patients in the emergency department (ED) since 2015. COVID-19 presented fundamental challenges to the structure of this experience, such as student restrictions on attending clinical experiences and limitations on the number of providers in a patient room, which necessitated a transition from face-to-face encounters to virtual ones.
MATERIALS AND METHODS: As a result, two virtual experiences were implemented; one was based in the ED with in-person faculty and patients with virtual learners and one staffed by ambulatory providers engaging in telehealth clinics. These experiences sought to provide an interprofessional clinical experience for students while following appropriate safety guidelines. Surveys were distributed to students post-clinic to gather student demographics and their feedback regarding the experience. Additionally, faculty preceptors provided insight into the experience, especially regarding logistics and infrastructure.
RESULTS: The virtual experiences successfully allowed teams of students to participate remotely in aspects of care including history taking, physical assessments, and medical decision-making. Additionally, the virtual care team structure allowed for senior students to mentor junior learners and for faculty members to provide point of care feedback. Students gained practical experience in telehealth that included logistics and challenges of providing virtual care and appreciating how technological barriers such as lack of access to internet-connected devices can be a source of disparity.
CONCLUSION: The COVID-19 pandemic required the reconfiguration of an in-person clinical experience to a virtual experience and this pivot was well received by students and faculty. The lessons learned can be generalizable to other professional schools who may be seeking to develop an interprofessional clinical experience and are exploring telehealth options.

Introduction

The COVID-19 pandemic necessitated that health professional schools develop and provide virtual clinical experiences to continue educational opportunities in clinical care while maintaining social distance1,2 As students return to on-site clinical duties, the lessons learned from these transitional experiences are noteworthy, and may be helpful for the development of future virtual clinical experiences for health professions students. Additionally, based on provider and patient satisfaction with telehealth and anticipated ongoing coverage from insurance providers, it is likely that telehealth will remain as a platform for the provision of medical care and it will be important to incorporate telehealth into interprofessional health professions curricula so that the next generation of providers will excel in this mode of health care delivery and become competent members of interprofessional virtual care teams.3,4

Duke University School of Medicine, in collaboration with the Duke School of Nursing, with the support of the Chancellor of Health Affairs and Duke University Health System began an Interprofessional Education (IPE) Clinical Experience in 2015. Often referred to as the “IPE Clinic,” this experience developed to include a team that varied each night between 4 to 9 individuals representing professional students and faculty from multiple health professions programs including medical doctors (MD/DO), physician assistants (PA), doctors of physical therapy (DPT), accelerated bachelors of science in nursing (ABSN), and nurse practitioners (NP). This experience was developed in acknowledgement of and response to publication of the Core Competencies for Interprofessional Collaborative Practice in 2011 (updated in 2016), developed by six major health professions organizations, which highlighted the importance of preparing health professions students for deliberatively working together.5 The clinic provided team-based care to patients in the emergency department (ED) during weekday evenings. Care was provided by the interprofessional student team under the direct supervision of a faculty physician and other health professions faculty who provided teaching during the patient care experiences. COVID-19 presented many logistical challenges to the nature and structure of this clinical experience and the development of alternative virtual clinical experiences became a necessity. The rationale for the study was to determine if the pilot virtual clinical experiences were successful and what should be considered in the future as the academic landscape continues to shift and there is a gradual return to on-site clinical activities.

Materials and Methods

Clinical Design

Due to the COVID-19 pandemic, the IPE Clinic transitioned to a virtual interprofessional clinical experience in the fall of 2020. Prior to COVID-19 pandemic, the IPE Clinic was voluntary for physician assistant (PA) students, nurse practitioner (NP) students and fourth year medical students (MS4) but required for first year medical students (MS1) and a specific subset of the third year medical students (MS3). However, for the virtual transition, the experience became optional for all students.

To increase student opportunities, two virtual experiences were offered. One virtual experience pilot was based in the emergency department where the students virtually joined a faculty member who was present on location with the patient. Another pilot involved students joining outpatient physicians performing telehealth encounters and actively participating in the visit. Students were notified of the experiences by email. The goals of these activities were 1) to practice team-based care skills by sharing components of the patient encounter, 2) to gain medical knowledge and practice telemedicine skills, 3) to provide an opportunity for leadership and mentorship by senior students, and 4) to assess the ability to teach and perform a virtual physical examination. Similar to pre-COVID conditions, the patients were already seeking care at the health system, were agreeable to student involvement, and were always directly observed and facilitated by physician faculty. For the virtual experiences, an institutionally approved HIPAA compliant video conferencing platform was utilized. In the event that videoconferencing was unsuccessful, telephonic communication was implemented using either institutional landlines or mobile phones utilizing a HIPPA compliant phone application. Additionally, to protect patient privacy, a visit script approved by the institutional compliance leadership that confirms the patient’s identity, their comfort with discussing private information, and their consent to proceed with a telehealth visit was delivered prior to any discussion of clinical information.

To accomplish these goals, the junior learners, who were first year medical students (MS1s), completed sessions with the support of a senior student who were a mix of third year medical students (MS3s), fourth year medical students (MS4s), physician assistant (PA) students and nurse practitioner (NP) students. Dates and sessions were set by a combination of both faculty availability and the MS1 curricular calendar. Students signed up voluntarily for sessions with a goal of three students (two MS1s and one senior student) participating in each session. Prior to the sessions, the students were provided with a mix of video and written presentations as well as published articles on telemedicine, virtual physical examination, familiarization with the electronic record, and instructions on accessing the virtual visits. After the session there was point of care verbal debrief with feedback, which was facilitated by the faculty preceptor as well as a post session survey for reflection and written feedback on the experience.

Faculty facilitators were all physicians who were interested in medical education and had previously participated in IPE teaching. The ED sessions were led by emergency medicine faculty and the outpatient telehealth sessions were led by four primary care physicians (two internal medicine physicians, a medicine-pediatrics physician, and a medicine-psychiatry physician). All faculty completed this IPE clinic outside of regularly scheduled clinical visits and were compensated for their time.

In this pilot, outpatient sessions were set by faculty on their administrative time and, usually, in the early evening to accommodate the students who had other courses that extended into the late afternoon. Because these sessions were not part of the providers’ regular clinic, new scheduling templates had to be built which required administrative support.

ED sessions were held early morning, mid-week, in order not to impede the flow of the ED. This was different from the in-person IPE clinic where the clinical sessions took place in a room adjacent to the ED during the evenings, typically the busiest time of day in the ED. However, due to COVID restrictions and the need to use ED resources, the session had to be in the ED itself and thus had to change to a time when the ED was not very busy so that a room would be available for use.

The goal was to see two patients in the ED and three patients in the outpatient sessions with a general structure of introduction and orientation, patient care, and subsequent discussion and debrief. Sessions were 2.5 hours and faculty were compensated for their time. The three students worked as a team during the encounter with the objective that each student had ownership over a certain role for each encounter with the senior student providing role modeling and support for their junior colleagues. On average, most students attended one to two clinical sessions with a preponderance of MS1s attending two sessions and most other students attending one session. Additionally, we had two longitudinal students, one MS4 who participated 12 times (six outpatient and six ED sessions) as part of an elective and an NP student who participated 13 times (seven outpatient and six ED sessions) as part of her continuity clinical requirement.

Survey Measures

For both virtual experiences, the students were asked to complete a survey to assess demographics, namely professional program and/or year, as well as different domains of their experience. The surveys were designed and reviewed by a convenience panel of interprofessional faculty who lead the IPE Clinic (). The survey content was designed to assess how well the four main IPEC core competencies (values/ethics for interprofessional practice, roles and responsibilities, interprofessional communication, and teamwork) were met through the experience.6 The survey was distributed and collected via an electronic survey platform (Qualtrics, Provo, UT).

The survey included six questions using a 5-point Likert scale from strongly agree (1) to strongly disagree (5) to assess the five session objectives/core competencies: 1) The patient interaction allowed me to learn about history taking techniques (interprofessional communication); 2) Team discussed the role of physical examination in the context of the chief complaint (responsibilities and teams); 3) Medical decision-making discussion felt complete and aided in my understanding of the patient course of care (interprofessional communication); 4) I was able to actively participate in the care of the patient (roles and responsibilities); and 5) Team worked well together (values/ethics for interprofessional practice). Additionally, using an open-ended free-text question format, the survey asked the students to reflect on their experience and offered areas to consider including barriers to care, areas for improvement, and what they learned about the roles of students from other health professions if their experience included a student from another discipline.

Patients who participated in a virtual experience were asked to complete a post-experience web-based survey. Patients in the ED experience were provided the survey on a tablet following the encounter while patients participating in a telehealth experience were provided a link to the survey which they were asked to complete after the encounter was ended (). Participation in the survey was encouraged but voluntary.

Given the ordinal survey design, relatively low number of participants, and non-normalized data, a chi-square test was used to assess for differences between the outpatient and ED experiences in their strength of agreement in the different learning domains. P values were calculated comparing “Strongly agree” responses to all other responses. Similarly, we used this method to assess for differences between different types of students in their reported level of agreement with achieving the goals/objectives of the sessions.

This study was approved by the Institutional Review Board at Duke University School of Medicine (IRB pro00065059) and allowed for collection of anonymized results.

Results

Demographics

A total of 129 students participated in the virtual telehealth experience, 69 in the outpatient sessions and 60 in the ED sessions. All students were asked to complete a survey upon completion of the clinical experience. A total of 76 students completed the survey for an overall response rate of 59% with an individual response rate of 65% (45) for the outpatient sessions and 52% (31) for the ED sessions. Of the students that responded: the majority were MS1s 63% (48), followed by senior medical students (MS3/MS4) 24% (18), and PA/NP students 13% (10).

Assessment Domains

In the outpatient telehealth experience, 82% of the students reported that they “strongly agree” that they were able to actively participate in the telehealth experience compared to 81% of the students participating in the ED virtual experience (Table 1). Additionally, in the outpatient experience, 80% of the students reported that they “strongly agree” that they were able to practice history taking as compared to 87% of the students in the ED experience. For the ability to address and understand medical decision-making and their ability to work together as a team, 100% of the students in both the outpatient and ED experience reported “Strongly agree.” None of these differences reached statistical significance with all p-values being >0.05. With regard to the physical examination, 62% of the students in the outpatient experience reported that they “strongly agree” that they discussed the role of physical examination as compared to 97% of the students in the ED experience. This result was statistically significant with p=0.0005. Of note, 7% of the students reported that they “somewhat disagree” that they discussed the role of physical examination.

Table 1

Students’ Assessment of Different Educational Domains from Participation in the IPE Clinic in the Outpatient (OP) and Emergency Department (ED) Settings

OP Total=45 ED Total=31% (n)	Strongly Agree		Somewhat Agree		Neither Agree or Disagree		Somewhat Disagree		Strongly Disagree		P value (χ2)
	OP	ED	OP	ED	OP	ED	OP	ED	OP	ED
History Taking	80% (36)	87% (31)	18% (8)	10% (3)			2% (1)	3% (1)			0.30 (1.06)
Discuss the Role of Physical Examination	62% (28)	97% (30)	20% (9)	3% (1)	11% (5)		7% (3)				0.00050 (12.12)
Medical Decision-Making	82% (37)	90% (31)	18% (8)	10% (3)							0.26 (1.30)
Actively Participate	82% (37)	81% (25)	16% (7)	7% (4)		3% (1)	2% (1)	3% (1)			0.86 (0.030)
Teamwork	96% (45)	97% (30)	4% (2)	3% (1)							0.82 (0.053)

Notes: All values are % (n), Blank= No respondent selected the category. Chi-Square statistic calculated using 1 degree of freedom. P >0.05 for comparisons between student types.

Abbreviations: OP, outpatient; ED, emergency department.

When analyzing the data by type of student participating, 88% of first year medical students (MS1), 67% of third or fourth year medical students (MS4), and 90% of physician assistant (PA) or nurse practitioner (NP) students reported “Strongly agree” to having been able to learn about history taking techniques (Table 2). When asked to assess whether the medical decision-making discussion felt complete and aided their understanding of the patient’s course of care, 79% of MS1, 94% of MS3/4, and 100% of PA/NP students selected “Strongly agree.” These results did not reach statistical significance. There was a trend towards significance when comparing MS1 students and MS3/4 students when comparing their responses to the question, “This patient interaction allowed me to learn about history taking techniques” with 88% of MS1s responding “Strongly agree” compared to 67% of MS 3/4’s (P=0.051). While the results did not reach clinical significance, this data shows a trend towards a difference in the learning areas in which students at different stages reap the greatest benefit. In this intervention, MS 1 students reported benefitting more from learning history taking techniques whereas the more senior medical students reported benefitting more from the medical decision-making component of the session, which would be consistent with the focus of skill acquisition for these different levels.

Table 2

Students’ Response to Survey Questions by Student Type

Student Type	Strongly Agree	Somewhat Agree	Neither	Somewhat Disagree	Somewhat Agree
This patient interaction allowed me to learn about history taking techniques
MS1	88% (42)	12% (6)
MS3/4	67% (12)	22% (4)	5.5% (1)	5.5% (1)
PA/NP	90% (9)	10% (1)
The medical decision-making discussion felt complete and aided in my understanding of the patient course of care
MS1	79% (38)	21% (10)
MS3/4	94% (17)	6% (1)
PA/NP	100% (10)

Notes: All values are % (n). Blank= No respondent selected the category. Chi-Square test used 1 degree of freedom.

Abbreviations: MS1, first year medical student; MS 3, third year medical student; MS4, fourth year medical student; PA, physician assistant student; NP, nurse practitioner student.

Qualitative information on the experiences gathered from post-session debriefs led by the faculty and from the free-text reflection on the survey were not formally analyzed. However, responses were generally favorable to the experience and supported the findings of the ordinal data and included enjoyment of working with students from other health professions as well as the ability to practice supervising more junior learners.

Although Initial data are limited, 13 out of 14 patients who completed the survey from the ED model reported “very good” and one reported “good” on a Likert scale of their overall care (“very good,” “good,” “fair,” “poor,” “very poor”) and 13 out of 14 reported that they would participate again and one reporting “maybe” they would participate again.

Similarly, patient perceptions were obtained through patient comments at the conclusion of the encounters but were not recorded. Anecdotally, feedback was uniformly positive and supported by the data from the returned patient surveys. The survey response rate for patients in the telehealth experience was very low due to the voluntary nature and provision of the survey after the conclusion of the telehealth encounter.

Discussion

Telehealth

As the academic landscape again changes with the relaxation of social distancing and return to on-site learning activities, health professional schools are again tasked with a redesign of their curricular programs, especially their clinical experiences. Our transition of this IPE Clinic from in-person to telehealth can offer insights to professional schools, that may be siloed, providing distance learning, or not on the same clinical campus, a way to provide their students interprofessional clinical experiences beyond the pandemic.

Telehealth can be used successfully by a team of students and facilitators to provide care for patients seeking care at a large institution with learners reporting educational gains. Currently, the availability and inclusion of telehealth experiences for learners is a large barrier to the provision of telehealth education and has implications in graduate medical education as well. Kehanian et al reported in Gastroenterology that gastroenterology fellows noted a compromise in the outpatient education mission with the adoption of telehealth for several reasons but specifically, not being able to participate in the faculty’s’ appointments was paramount.6 In this opportunity, students were active participants in these telehealth visits and were directly patient-facing, not just in the virtual room as an observer.

Assessment Domains and Clinical Practice

In these experiences, the majority of students, regardless of stage or training or profession, were able to actively participate in the care of the patient in both models in terms of history taking, physical examination, medical decision-making, and overall teamwork. This was within a team of three students sharing in one patient encounter. A team size of three students was smaller than the pre-pandemic model. Striking the right balance of team size will require further investigation; although three students worked in the pilots, the impact on the learning and patient experience with additional student participants is unclear. Additionally, team size is an important consideration as each profession may have different requirements for supervising faculty and this will increase the number of people participating in a session.

A focus on physical assessment is possible in both virtual models, although the role of physical examination was discussed more in the ED model. This was likely due to the faculty physician in the ED experience being physically present with the patient, which guaranteed a physical examination as compared to the completely virtual outpatient sessions, which may have had technology issues limiting the performance of a physical assessment. Another consideration is that the physical examination may not have been pertinent in the outpatient session, for example, counseling regarding a radiology study, medication change, follow-up, or even the discussion of certain medical conditions.

It appears that a single clinical telehealth session with multiple learners can meet learning goals for students at different stages of training and with different professions. While the results did not reach clinical significance, it is suggested that MS 1 students may benefit more learning history taking techniques, whereas more senior medical students report more benefit from the medical decision-making component of the session, which would be consistent with the focus of skill acquisition for these different levels. Virtual care experiences appear to allow the majority of students to learn about history taking, virtual physical assessment, and medical decision-making as appropriate to their level of training.

In this type of experience, utilizing a team comprised of interprofessional senior and junior learners worked well, as the senior can model clinical skills and provide mentorship to the junior learners with the result of junior learners feeling supported and safe in these early clinical experiences. Students can learn from and about each other through the roles they took on during the experience. Of note, the study only included medical students, nurse practitioner students, and physician assistant students. Future studies should include other health professions students to better assess interprofessional teamwork and the effect that the additional health professions students might have on the reported attainment of learning goals.

Faculty Perceptions

This virtual IPE Clinic did highlight specific infrastructure and administrative needs. Finding space within existing health profession curricula will be important to allow universal participation in a standardized telehealth experience. This will require collaboration with educational leadership and existing curriculum stakeholders. Additionally, as calendars are set early, these conversations must begin well in advance of starting the experience.

Major barriers to implementation of an educational telehealth clinic include 1) procuring protected time or compensating faculty for participation, 2) coordination of student availability with telehealth session availability, and 3) patient identification and scheduling. Dedicated administrative support staff to coordinate student and patient scheduling would be necessary to scale this program to be able to accommodate all students.

The outpatient sessions revealed equity and access issues as the patients served had variable access to technology requirements such as access to a smartphone or other video capable device and reliable internet service. On several occasions, the virtual visit had to transition from a video visit to a phone call due to limited broadband access. As pointed out by Julien et al in Circulation

Successful transition to telemedicine requires the intersection of at least 3 key factors: access to broadband internet, an internet-capable device, and sufficient technology literacy to take advantage of the first 2 factors.7

Attention to the “digital divide” will be important to teach as telehealth curricula are developed.

Patient Perceptions

Finally, patients reported a good experience participating in this telehealth model. Although Initial data is limited as very few patients completed the survey, the majority that did respond reported that they would participate again. Additionally, both ED and outpatient faculty verbally reported positive feedback during their debriefs with the patients following the video sessions. Although conclusions from this data cannot be drawn due to lack of power and qualitative nature, this limited survey suggests patients are accepting of the experience and have a general sense of good care in regards to students joining their telehealth appointments.

Limitations

Due to the pandemic and the necessary clinical shifts that occurred, there were not enough shifts for all students to participate and it was not a mandatory experience for MS1 and 3s as it had been in prior years. The sessions were scheduled to coincide with the MS1 schedule which is why they were likely the largest demographic group. Additionally, students were reminded to complete their surveys electronically after their clinical sessions, but completion was not required. As such, our response rate of 59% may have limited the representativeness of the final sample. Finally, given the pilot nature of the study, the limited scope, Covid-19 restrictions, and supervisory requirements, our study only included medical and NP/PA students and not other interprofessional health students.

Conclusion

The COVID-19 pandemic required innovative changes in order to continue providing clinical exposure to health professions students. An in-person clinical experience was successfully reconfigured to be a virtual experience where students continued to care for real patients and be active in the care of their community while balancing the need for safety. As community social distancing restrictions are dynamic, these lessons learned can be valuable to other professional schools who may be seeking to develop an interprofessional clinical experience and are exploring telehealth options.

The experience was “customizable” in that students could set their own goals and focus on certain parts of the patient encounter as they assumed different roles on the team. There was also an interprofessional component to these experiences and students were able to work together, with an emphasis on the similarities between professions within a patient encounter.

It is likely that telehealth will continue to be a platform for the provision of healthcare and thus it is important to incorporate telehealth training into the curricula of interprofessional health students so that the next generation of providers will excel at this mode of health care delivery and expand on its capabilities for improving patient health outcomes. Future needs for telehealth education for health professions students include the development of synchronous and asynchronous learning programs and materials in telemedicine.