Live streaming ward rounds using wearable technology to teach medical students: a pilot study.

Background: The COVID-19 pandemic resulted in a loss of clinical clerkship opportunities for medical students. To address this problem while maintaining patient safety, this pilot study explored the feasibility of using a wearable headset to live stream teaching ward rounds to remotely based medical students.
Methods: Three live streamed teaching ward rounds were delivered to three groups of medical students (n=53) using the Microsoft HoloLens 2 device and Microsoft Teams software, and results pooled for analysis. Feedback was gathered from students and instructors using the evaluation of technology-enhanced learning materials (ETELM). Patient feedback was gathered using the Communication Assessment Tool to explore any impact on interpersonal communication.
Results: The response rate for the ETELM-learner perceptions was 58% (31/53), 100% for the ETELM-instructor perceptions. Students strongly agreed that the overall quality of the teaching session and instructors was excellent. However, 32% experienced issues with audio or video quality and one remote student reported cyber sickness. The statement 'educational activities encouraged engagement with session materials/content' returned the most varied response. Instructors reported technological problems with delivery while using the HoloLens 2 device and environmental noise in the ward was a disruptive factor. Preparation and skilled facilitation were key to delivering a high-quality teaching session. Patients reacted generally favourably to the technology and no negative effects on interpersonal communication were identified.
Conclusion: The experience of live streamed ward rounds was well received by patients, medical students and teaching faculty. However, there remain limitations to the routine use of HoloLens 2 technology in our setting including steep learning curves, hardware costs and environmental factors such as noise and WiFi connectivity. Live streamed ward rounds have potential postpandemic implications for the judicious use of resources, and the possibility for few educationally minded clinicians to teach at scale in a patient-friendly manner. © Author(s) (or their employer(s)) 2021. No commercial re-use. See rights and permissions. Published by BMJ.

Descriptive accounts of live streamed ward rounds for educational delivery during the pandemic have been reported.

Empirical data on the student, instructor and patient experience of live streamed ward rounds.

Objective and practical evaluation of the technological advantages and limitations of the HoloLens 2 augmented reality headset for live streamed ward rounds.

Discussion of advantages and barriers to sustained postpandemic application of live streamed ward rounds.

Introduction

In the first half of 2020, the COVID-19 pandemic significantly restricted undergraduate medical training within the UK, including suspension of medical student clerkships on wards.1 2 The need to continue to deliver training for medical students during an active pandemic that requires social distancing has accelerated the need to explore alternative educational methods and adopt new technology.3 Traditionally, ward-based learning is an essential component of medical education and involves a group of a few students and the medical team jointly reviewing patients at their bedside.4 It is an example of situated learning wherein students learn by being present and involved in the workplace environment. Ward-based teaching involves the modelling of professionalism and the demonstration of clinical decision-making as well as the norms of clinical practice.5–7

Alternatives to ward-based learning such as simulation may not address the complexity of learning required. Medical education is ‘complex learning’,8 which includes understanding physiological systems, developing interpersonal skills and making clinical decisions influenced by many factors within dynamic systems. As a result, whole task learning like ward rounds whereby students are immersed within real-life situations are key for mastery.8

Few studies have published evaluations of remote ward rounds for healthcare delivery.9–11 Additionally, descriptive accounts of live streamed ward rounds for medical education are available in the literature,12 13 using varied devices such as mobile phones and HoloLens 2. However, there is an absence of empirical data from an educational perspective. There has been great enthusiasm in the adoption of virtual teaching during the COVID-19 pandemic,14 but crucially alongside educators, the experience of students15 and patients must be evaluated and their concerns are addressed.

To address the tension between the need for continued ward-based learning and the need for infection prevention, a pilot study was developed using wearable technology to deliver live streamed remote ward rounds. The use of live streaming was hypothesised to provide the most real-to-life learning possible and so supported the need for complex, whole task learning. In particular, live streaming allows students to identify nonverbal cues and, therefore, continue to develop their interpersonal skills. Relevant reporting guidelines were followed.16

The aim of this pilot study was to evidence the feasibility of live streaming teaching ward rounds to remotely based medical students, using the Microsoft HoloLens 2, a wearable augmented reality (AR) headset.

Our objectives were classified into three domains: (1) pedagogy: to assess both the student and instructor learning experiences of a virtual ward round, (2) technology: to assess student and instructor perspectives on comfort and practicality of HoloLens 2 within this educational activity, (3) patient experience: to assess whether the teaching set-up was acceptable to patients.

AR devices such as the Microsoft HoloLens 2 allow integration of real-life working environments with virtual images projected in space. The HoloLens 2 is an untethered wearable AR headset, which is essentially a computer worn around the head17 (figure 1). AR allows the headset wearer and remote participants to see both the real and the augmented world. It also features a high-quality camera, which can be used for hands-free live streaming. The Remote Assist software on the HoloLens 218 is integrated with the end-to-end encrypted Microsoft Teams software, a platform that is formally endorsed by NHS Digital19 and being adopted across the National Health Service (NHS) in the UK. Therefore, we chose this technology to support live-streamed ward rounds as those both align with the ongoing digital changes in the NHS and offer the flexibility of an untethered wearable device.

Figure 1

Microsoft HoloLens 2 worn around the head with an external microphone attached.

Methods

Following the first wave of the COVID-19 pandemic, fourth year (prefinal year) medical students at the University of Leeds recommenced clinical placements in July 2020. At our tertiary Oncology centre, 53 students were assigned to one placement block (July–August 2020), and during this period, they rotated though oncology in three subgroups. Our pilot project was completed with this cohort.

All students joined the ward round remotely using Microsoft Teams on personal 2D devices. The ward round structure was designed jointly by healthcare professionals (HCPs) in oncology and the undergraduate medical education department through an iterative process with trial attempts. The final design was divided into three sections: (1) prepare and discuss: a 15 min case discussion away from the bedside with instructors using Microsoft Teams on a laptop, (2) observe and explore: a 10 min bedside clinical review with an instructor wearing the HoloLens 2 device (figure 2). The compatibility of Remote Assist software with Microsoft Teams supported an uninterrupted teaching session. Students were encouraged to ask questions via the chat function and these were then relayed to the patient by instructors, (3) reflect: a 5 min postreview debrief away from the bedside to help consolidate learning points.

Figure 2

Live streaming using the Microsoft HoloLens 2 at the bedside.

The advantage in the use of the wearable headset was to enable flexibility in changing the focus of the video stream by simple head movements of the HoloLens 2 wearer. The headset is completely untethered allowing unrestricted motion. Thus, remote students were in effect following the gaze of the HoloLens 2 wearer, allowing for a perspective that simulated being physically present and examining the patient in real time. Furthermore, with the use of the Microsoft HoloLens 2, radiological images or blood test results could be easily projected in AR at the patient bedside, thus enabling integration of all clinical information in one place (figure 3). However, practically, this process was deemed to be too cumbersome and, hence, this feature was not used.

Figure 3

Images and results projected in augmented reality around the bed space.

Patients for live streaming were selected by instructors based on the perceived learning value for students and informed consent. Three instructors were present for each teaching ward round during our pilot, one leading the patient consultations, one observing and one individual with expertise in managing the HoloLens 2 who was wearing the headset. Instructors, medical students and patients provided structured feedback.

Feedback was gathered using the evaluation of technology-enhanced learning materials (ETELM) developed by Cook and Ellaway,20 a previously validated tool that focuses on both pedagogic and technological elements of a teaching activity and provides data intended to help improve future sessions. Two questionnaires, the ETELM-LP (learner perceptions) and ETELM-IP (instructor perceptions) were used and are reproduced in online supplemental materials. In both ETELM questionnaires, evaluators rate statements on a Likert scale between 1 (strongly disagree) and 7 (strongly agree). There are also three open-ended questions probing for ideas for improvement and allowing for qualifying statements about the evaluator’s experiences.

Patient feedback was gathered using the Communication Assessment Tool (CAT).21 As the HoloLens 2 is a wearable technology that is prominently visible around an individual’s head, we expected it might impact on nonverbal communication during the consultation. The CAT is able to detect negative clinical faculty–patient interactions and was selected to provide objective evidence of the patient experience. One free text response question, ‘Do you think the HoloLens changed the consultation in any way?’, was added to the CAT to specifically address any communication concerns as a result of the use of technology. The CAT questionnaire used is available in online supplemental materials.

Ordinal data were analysed using descriptive statistics, medians and IQR. Qualitative comments for improvement were used to qualify reported empirical data. This project was reviewed by the Trust’s Research and Innovation review committee and registered as a service evaluation. All data from students were collected anonymously.

A standard operating protocol was developed for information governance purposes. All call members had to use a secure nhs.net email to join the meeting. The information technology team of the hospital liaised with Microsoft and confirmed that data from live streams are secure, encrypted and not stored on devices or in a cloud. Patients with capacity were verbally consented before live streaming and provided with information leaflets. Nearby patients and staff were made aware of the live streaming process to avoid unintentionally live streaming other persons, and recording of these sessions was prohibited. Protocol for accidental recording of the session involved immediate deletion of the material and contacting the deputy Caldicott guardian. While moving between bedspaces where other patients or staff could be inadvertently filmed, video and audio functions were disabled. Remote observers signed a form confirming they would put strong preventative measures in place to ensure unauthorised colleagues could not view or hear the live stream. Watching in a public place was prohibited.

Results

Three virtual ward rounds were completed during the placement block for the first cohort of 53 medical students. 23, 13 and 17 unique students attended the three ward rounds, respectively. The evaluations from all ward rounds in the placement block were pooled for analysis. The overall response rate for the ETELM-LP questionnaire was 58% (31/53). The median age of students was 24 years (range 22–37 years). The three instructors were common for all teaching rounds and the ETELM-IP questionnaire was completed by all for each session returning 9/9 responses or a 100% response rate.

Learner perceptions

The detailed results of the ETELM-LP tool are reported in table 1.

Table 1

ETELM-LP results

ETELM-LP statement	Median score (on a 7 point Likert scale with 7 as strongly agree and 1 as strongly disagree)	IQR
Instructions provided a good introduction to the session.	6	2
Session objectives, expectations and policies were clearly stated.	6	2
The session was well organised.	7	2
Session objectives were relevant to my needs.	7	2
Navigation of the technology-based components of the session was logical, consistent and efficient.	6	2
The session technologies and media supported the learning objectives.	7	1
This session required inappropriately high technology skills.	2	1
I had significant computer/technical problems during this session.	2	1
The educational activities encouraged engagement with session materials/content.	6	3
The educational activities promoted achievement of the session objectives.	6	2
There was a strong instructor presence/personal touch in the session.	7	1
I had sufficient opportunity to assess and reflect on my learning progress.	6	2
I received adequate feedback on my learning progress.	4	2
I had sufficient opportunity to evaluate/provide feedback on the session.	6	2
I received adequate support for any technical issues encountered during this session.	5	2
I received adequate support for any questions or concerns I had about my learning.	6	1
I encountered culture- or language-related problems.	1	0.5
I invested enough time and energy to meet/exceed the session expectations.	6	2
This session will change my practice.	5	3
The overall quality of this session was excellent.	6	1.5
The overall effectiveness of the instructor was excellent.	7	1

ETELM-LP, evaluation of technology-enhanced learning materials-learner perceptions; IQR, Interquartile range.

Students’ responses were overall in strong agreement with positive statements on the questionnaire and the overall quality of the instructors and session achieved high median scores of 6 or 7 (on a 7 point Likert scale where seven was strongly agree). Similar strong agreement was recorded for statements concerning teaching preparation, namely, clearly stated session objectives, session objectives being relevant to learner’s needs, clear instructions and the organisation of the session.

The students attended the session via Microsoft Teams and they reported logical and consistent navigation of technology (median=6) with the technology supporting the learning objectives for the session (median=7). They generally disagreed with statements expressing concern about technology such as need for inappropriately high technology skills or facing significant technical problems. However, support for technical issues for those who did encounter them was felt to be less than adequate (median=5). This correlates well with the open text suggestions for improvement, which highlighted concerns with audio or video quality experienced by 32% (10/31) of students. Disturbance due to background noise on the ward and interrupted video streams due to unstable WiFi connections were the most common problems faced. One student suffered from nausea due to the movements of the video stream in tandem with head movements of the HoloLens 2 wearer.

Interestingly, despite the virtual nature of the ward round, students agreed on a strong instructor presence or personal touch (median=7) during the session. Only 3/31 students commented in free text on a preference for smaller groups for teaching. Students did not report strong feelings (median=4) about whether they received feedback on their learning progress, which is unsurprising as the session was not designed to provide individual feedback.

The statement ‘educational activities encouraged engagement with session materials / content’ (median=6, IQR=3) returned the most varied response, with 29% (9/31) of students disagreeing or unsure (score 2 to 4) and 58% (18/31) expressing strong agreement (score 6 or 7). Free text comments included dissatisfaction with the interactive elements of remote learning.

"It’s far more difficult to engage over video as you can't take part in conversation or discussion."

These difficulties were contrasted by others detailing good participation.

"[The teaching] put in context with how the patient actually looks made it much more realistic. Liked how the presenters asked lots of questions to involve [us] which helped me embed my learning further.

Instructor perceptions

The results of the ETELM-IP questionnaire are detailed in table 2.

Table 2

ETELM-IP results

ETELM-IP statement	Median score (on a 7 point likert scale with 7 as strongly agree and 1 as strongly disagree)	IQR
Instructions provided a good introduction to the remote ward round (eg, how to get started, what to expect).	6	1
Session objectives were relevant to participant needs.	7	1
Navigation of the technology-based components of the session was logical, consistent, and efficient.	5	1
The session technologies and media supported the learning objectives.	6	0
This session required that participants possess inappropriately high computer skills.	2	1
The educational activities encouraged participants' engagement with session materials/content.	6	1
The educational activities promoted participants' achievement of the session objectives.	6	0
I was able to contribute a personal presence/personal touch during the ward round delivery.	6	0
I plan to use learner feedback to improve the session.	7	0
The remote ward rounds will be easy to maintain and deliver again.	6	1
It will be easy to re-use of all or part of the session materials in other, future sessions.	6	1
I had access to needed tools during ward round delivery.	6	1
I had significant computer/technical problems while delivering this session.	5	1
I received adequate support for any technical issues encountered while developing and delivering this session	7	1
I was able to provide adequate support to students for questions or concerns about their learning.	6	1
The ward round was a good use of time and resources.	6	0
The overall quality of this ward round was excellent.	6	1

ETELM-IP, evaluation of technology-enhanced learning materials-instructor perceptions; IQR, Interquartile range.

In contrast to the students, instructor responses indicated there were ‘significant technical problems while delivering the session’ (median=5).

"The HoloLens did have glitches which a webcam wouldn't experience. However, you would loose the flexibility of the HoloLens."

There was agreement on the technology supporting the learning objectives (median=6), but there were some concerns with sustainability where two responses indicated indifference to the statement 'The remote ward rounds will be easy to maintain and deliver again'. Suggestions for improvement largely focused on technological concerns with unexpected technical faults and interruptions while operating the Remote Assist software compounded by variable WiFi connectivity. An instructor noted that,

"[It] can be quite stressful if [the technology] fails during a teaching session."

Furthermore, the issue of environmental noise was also identified as a disruptive factor. Nevertheless, clinical faculty agreed that ‘the overall quality of this ward round was excellent’ (median=6) and that the session ‘was a good use of time and resources’ (median=6, IQR=1).

Patient feedback

7/8 patient CAT questionnaires were returned and all of these reported positive interaction statements including ‘greeted me in a way that I felt comfortable’, ‘paid attention to me’ and ‘showed me care and concern’ which were rated as either very good or excellent (table 3).

Table 3

Communication assessment tool results

Communication assessment tool statement	Median score on a 5 point scale, 5=excellent
Greeted me in a way that made me feel comfortable	5
Treated me with respect	5
Showed interest in my ideas about my health	5
Understood my main health concerns	5
Paid attention to me (looked at me, listened carefully)	5
Let me talk without interruptions	5
Gave me as much information as I wanted	5
Talked in terms I could understand	5
Checked to be sure I understood everything	5
Encouraged me to ask questions	5
Involved me in decisions as much as I wanted	5
Discussed next steps, including any follow-up plans	5
Showed care and concern	5
Spent the right amount of time with me	5
(The doctor’s staff) treated me with respect	5

This provides objective data suggesting the patient–clinician consultation was not adversely affected by the presence of a visible HoloLens 2 headset at the bedside. One patient commented,

"The consultation did not change in any way. It was a benefit that the students could ask questions."

Contrary to expectations, one patient even reported the consultation felt more personal.

"I did not find the HoloLens to be off putting in anyway. It limits the number of people at the bedside which is very important at this current time. This also means it feels more personal during the consultation."

Discussion

Our pilot project demonstrated that live streamed remote ward rounds are feasible from an educational perspective, and medical students valued the experience. Similar to any other educational exercise, preparation was identified to be key to delivering high-quality teaching.22

The perception of engagement with the teaching session was variable between students reflecting the challenge of making students feel involved despite them being remote. Instructors have little control over the physical learning environment around remote students and are limited to organising their own surroundings and the virtual interface. Additionally, in our study, issues with environmental noise in the ward hindered student engagement. The HoloLens 2 device is equipped with an inbuilt noise cancellation feature as a default setting. However, this only allows the voice of the HoloLens 2 wearer to be clearly heard. Thus, it muffles the voice of the patient and, hence, is not suitable for a ward round setting. This limitation has been identified in other AR headsets as well.9 In order to overcome this, a small external microphone needed to be attached to the HoloLens 2, but this in turn resulted in amplification of all surrounding sound. Finally, engagement is a bilateral process requiring agency and reflexivity from the student as well. Thus, some variability in perception when presented with the same teaching exercise is to be expected.23

A fundamental concept for technology supported learning activities is the instructional design of the session.24 Live streamed ward rounds are a novel tool and, hence, the design is not yet established. One student commented, ‘(The) structure of the session was very useful’. Our structure followed the natural flow of our traditional ward round where patient notes and recent investigation results are reviewed by HCPs prior to a bedside review. We attempted to deploy the right technology for the right activity and, hence, we used secure platforms and alternated between a laptop and the HoloLens 2 headset. This also ensured that students could remotely view the patient examination and all the nonverbal communication between physician and patient. Alternative structures suggested in the literature13 limit the video stream to observe the physician only. Learning on ward rounds is not merely participation in a cognitive exercise but includes active observation of communication and the patient–physician interaction.25 Thus, not visualising the patient would expect to result in a reduced sense of participation in workplace activities and a less effective learning environment.

However, fluctuating between devices requires increased technological support. Interruptions due to freezing or sudden restarting of the Remote Assist application were problematic and adversely impacted the instructor and student experience. This issue has also been reported in other wearable headsets.9 We found that it was easy to teach someone how to operate the HoloLens 2, but to gain proficiency that enables troubleshooting required extensive familiarity with the device. Therefore, an individual who was thoroughly familiar with the HoloLens 2 was required on the instructor team for technical support.

We also believe that the learning experience was heavily influenced by the quality of our instructors.26 HCPs leading the ward round were enthusiastic, keen to engage students through questions and challenged them to critically analyse patient presentations. This was likely reflected in the student’s perspective of the teaching.

An important limitation of this evaluation is the difficulty in distinguishing the value of live streaming of inpatient teaching ward rounds from the value of a structured teaching ward round. However, our argument is that a virtual design is a practical and efficient solution to teach at scale. This in turn enables few educationally minded clinicians to design and deliver a structured teaching ward round for large cohorts of students in a sustainable and patient-friendly manner, overcoming traditional barriers of resource and time limitations.27 28 We believe a clearly structured ward round with quality instructors and targeted teaching preparation creates a superior learning experience.29 This would also help minimise the heterogeneity of student experience and expose them to varied patient cases.

Limitations to live streamed ward rounds include inability to practice physical examinations and history taking. Thus, students would lack exposure to important components of their expected ward round learning.30 Medical students need direct patient contact to independently practice these skills. Subsequently, they should be encouraged to present case histories and communicate their clinical reasoning.13 31 It has been suggested that this is best achieved through small group discussions, so that students can receive individual feedback on their performance.32 Other concerns are that large cohorts of students attending virtually at a time can make some students feel uncomfortable in asking questions,33 34 and this was also reflected in our results. However, skilful facilitation can overcome students’ hesitation.33 Additionally, equality and access require attention as students might be differently abled or lack devices with good image resolution or audio quality.35 This can result in differential learning experiences. Finally, unique side effects of AR headsets such as cyber sickness36 or the sensation of nausea for those wearing the headset are difficult to overcome. We also reported a similar sensation experienced by a remote student sharing the HoloLens wearer’s perspective.

The cost of a single HoloLens 2 device is US$3500.37 Licenses to use different software on the device will account for additional recurrent expenditure. For organisations interested in developing a HoloLens 2 or AR programme, a local cost-effectiveness analysis needs to be completed. Alongside initial investment in hardware, the ability to locally develop or purchase educational software, the training of personnel required, the IT infrastructure such as WiFi connectivity, information governance protocols as well as learner and instructor preferences need to be considered. As evidenced by Case Western Reserve University in Ohio, live video streaming only scratches the surface capabilities of the HoloLens 2. Case Western has replaced their wet dissection anatomy programme with anatomy taught through AR applications using the HoloLens 2.38

The use of telemedicine in modern healthcare delivery is rapidly expanding39 and it is imperative that medical students are trained for a digital future. The same technologies used for live streaming ward rounds in a hospital can be used for healthcare delivery or vice versa. For example, telemedicine has already been demonstrated to remotely allow family members to be present on intensive care ward rounds and can cost effectively reduce the need for travel to remote locations.40 In the USA, 60 medical schools include some form of telemedicine training in their clinical clerkships.41 Thus, the digital know-how and infrastructure developing through live streaming during the COVID-19 pandemic has long-term applications.39

In conclusion, we found that using HoloLens 2 with Remote Assist and Microsoft Teams was a feasible option for live streamed ward rounds that delivered a useful educational experience. A summary of tips is provided for others intending to use the HoloLens 2 in this capacity (box 1).

Careful patient selection and preparation of the teaching session are required.

Preferentially select a quieter clinical environment to optimise the acoustics.

Test the internet connectivity in the clinical environment prior to running any live teaching session.

The HoloLens 2 can live stream for 2 hours following a standard charge. A portable battery pack can be used for longer sessions.

A universal serial bus (USB) microphone adjunct to the HoloLens is required to transmit the patient’s voice.

Operator issues with the HoloLens can arise and extra time must be built into the teaching session to allow for this.

Identify very clear learning objectives at the start of the teaching session.

However, there remain several limitations to the routine use of HoloLens 2 technology in our setting including steep learning curves, hardware costs, unpredictable errors with the software and environmental factors such as noise and WiFi connectivity. Importantly, this paper provides objective data on the learner and instructor experience of live streamed ward rounds, highlighting advantages that might encourage educators to develop this as a teaching activity that persists beyond the pandemic. Moving forwards, an important area of future research would be evaluating live streamed ward rounds using alternative lower cost technologies. Importantly, future research should ensure that alongside the perceptions of students and instructors, the patient experience is highlighted and carefully studied.