Real-time online point-of-view filming education for teaching clinical skills to medical students.

PURPOSE: This study aimed to measure the educational satisfaction with and effectiveness of real-time online point-of-view filming (POVF) clinical skills education in medical students.
METHODS: Medical students participated in a 120-minute clinical skills education session. The session consisted of emergency procedures, wound management, and vascular access. The authors provided real-time online POVF using a smartphone. A questionnaire survey was issued to the students after the class, and their satisfaction with education, educational environment, and effectiveness were analyzed.
RESULTS: Responses about satisfaction with POVF education were very positive in all grades. However, approximately half of the students were satisfied with the smoothness of listening to a lecture and the video quality. More than half of the students responded positively to the question about educational effectiveness.
CONCLUSION: In these times of non-classroom teaching brought on by the coronavirus disease 2019 (COVID-19) pandemic, POVF clinical skills education is likely to be a very useful educational tool. If disadvantages such as insufficient feedback or environmental problems can be addressed, it could serve as an alternative method of clinical skills education even after the COVID-19 pandemic.

Introduction

Medical students receive clinical skills education through various methods such as lectures, bedside observation of patients, direct practice activities, and simulation. Among these methods, direct practice and feedback are known to be more effective at enhancing technical skills [1]. However, the appropriate teacher-to-learner ratio in this method is known to be 1:4, which means that a large number of skilled instructors are needed to participate in this form of education [2]. Since the number of learners in each practice session is small, it takes a long time to educate a large group of students. Although most medical schools usually implement direct practice or simulation curriculum for clinical skills teaching, they are experiencing difficulties due to the imbalance in the number of teachers and students. This problem may be particularly prominent in skills teaching that requires detailed observation or manipulation (e.g., verifying the vocal cord during endotracheal intubation) and may lead to a decrease in the effectiveness of and satisfaction with clinical skills education.

The coronavirus disease 2019 (COVID-19) pandemic has almost suspended face-to-face medical education and clinical skills education has been negatively affected. Clinical clerkship has been cancelled or moved online, and the opportunities for clinical skills observation or activities have considerably decreased since the start of the pandemic [3]. Online teaching and social distancing cause difficulties in direct clinical practice training. While faculty members and students have tried to adapt rapidly to this situation, both medical educators and students have become concerned about educational gaps in clinical skills.

A previous study revealed that video-based e-learning was effective in teaching certain practical clinical skills [4]. As video educational materials such as YouTube videos or massive open online courses were in use before the pandemic, educators were able to apply them to replace those methods most affected by the pandemic [5]. Teleconferencing platforms like Zoom or Microsoft Teams were also introduced as alternative methods. However, these alternatives have definite limitations as it is difficult for students to observe the clinical skill processes through these media. Accordingly, teachers must creatively reconstruct educational methods in the online environment.

The main aims of newly developed clinical skill educational methods are to achieve the following goals: (1) replace current teaching methodology in situations where face-to-face education is impossible; (2) observe clinical skill processes in greater detail than in direct practice activity; and (3) bridge the gap created by the pandemic in teaching clinical skills effectively.

The authors considered that real-time online clinical skills education using point-of-view filming (POVF) with a smartphone would be appropriate in the pandemic environment since one educator can readily facilitate education for large cohorts of students. We expected that students would be able to observe the relevant skills in detail because the educator could easily perform camera manipulations such as zooming in/out of the scene. Standardization of skills education would be also be achieved through the maintenance of consistency, because only one educator would be involved.

Few studies have explored the practicality and effectiveness of one-person POVF in medical education [3,6]. This study aimed to evaluate the educational effects and satisfaction for medical students after real-time online clinical skills education using POVF with a smartphone by grade and to identify the correlations among all variables. Based on the results, the authors attempted to explore the possibility of this method as a substitute for existing face-to-face skills education.

Methods

1. Construction of the education session

A total of 220 M5 (3rd grade) and M6 (4th grade) medical students (110 students in each grade) attending the School of Medicine were included in this study. The session runtime was 120 minutes, and the lecture was progressed for each grade. A clinical skills expert with more than 10 years of clinical skills training experience participated in the lecture, and another educator participated for video filming. An Android smartphone was used for POVF, and Zoom (Zoom Video Communications Inc., San Jose, CA, USA) was utilized as a real-time online teaching tool (Fig. 1).

Fig. 1

Real-Time Online Clinical Skills Education Using Point-of-View Filming with a Smartphone and Video Shown to Students

(A) With a smartphone and (B) video shown to students.

Clinical skills contents included (1) emergency procedures (adult basic life support, defibrillation, and endotracheal intubation), (2) wound management (wound dressing, local anesthesia, and sutures), and (3) vascular access (arterial puncture, venipuncture for blood culture, and blood transfusion). For reference, M6 students had already learned these skills during their clerkship, whereas M5 students had not yet been taught emergency procedures.

We used an educational module devised from a modified Peyton’s four-stage approach [7]. The process consists of four steps: (1) demonstration, (2) deconstruction, (3) comprehension, and (4) execution. However, we could not apply the last execution step to this session because of online distance learning. Before the class, we employed self-directed learning using a pre-learning video. In the class, the educator demonstrated the clinical skills without any comments. During the deconstruction and comprehension steps, the educator demonstrated the same processes with additional explanation and comments. Students could ask questions or give comments any time during the class.

2. Data collection and analysis

After POVF education, a questionnaire survey was issued to the students who took the class. The survey was conducted from December 2021 to February 2022, and all participants were voluntarily surveyed. The survey was based on the questionnaire established by the authors and consisted of four categories as detailed below. The answers were assessed using a 5-point Likert scale (1=very unsatisfied to 5=very satisfied) and through open-ended questions. The four categories comprised the following components:

1) Satisfaction with education (5+2 questions, Cronbach’s α=0.75)

It includes (1) difficulty level of pre-learning materials, (2) adequacy of educator attitude, (3) satisfaction compared to face-to-face procedural education, (4) satisfaction compared to self-directed learning, (5) adequacy of feedback, (6) adequacy of teaching time (1=very short, 3=appropriate, 5=very long), and (7) difficulty level of lecture (1=very easy, 3=appropriate, 5=very difficult). Difficulty level and teaching time questions were excluded in the process of summing the final satisfaction.

2) Educational environment (4 questions, Cronbach’s α=0.69)

It includes (1) ease of network access, (2) difficulty in listening to a lecture, (3) video quality, and (4) skillfulness of smartphone camerawork.

3) Educational effectiveness (3 questions, Cronbach’s α=0.71)

It includes (1) usefulness for acquiring clinical skills, (2) usefulness for preparing for Korean Medical Licensing Examination, and (3) predictive educational effectiveness compared to face-to-face procedural education after COVID-19.

4) Students’ feedback for clinical skills education using POVF (open-ended questions)

Of the 220 participants, 186 (84.5% response rate) completed the questionnaire. The score differences between the grades were analyzed using an independent two-sample t-test. The analysis was conducted using IBM SPSS ver. 26.0 (IBM Corp., Armonk, USA) software, and statistical significance was set at p<0.05 and p<0.01.

3. Ethical considerations

This study was approved by the Institutional Review Board of Kyung Hee University (KHSIRB-21-483[NA]). Informed consent was obtained from all participants.

Results

1. Satisfaction with education

Satisfaction with education was positive in the case of students of all grades (M5: 22.0±2.558, M6: 22.4±2.379). Most students responded positively regarding satisfaction with pre-learning materials (4.45±0.641), educator’s attitude (4.70±0.546), self-directed learning with video clips (4.10±0.937), satisfaction compared to face-to-face education (4.31±0.778), feedback (4.63±0.537) (Table 1), education running time (appropriate 81.7%), and the level of the lecture (appropriate 90.9%). However, M6 students responded that the education running time of 120 minutes was a little short.

Table 1

Analysis of Survey about Effect of Clinical Skills Education by Grade

Variable	M5 (N=93)	M6 (N=93)	M5+M6 (N=186)	p-value
A
Satisfaction of pre-learning materials	4.38±0.624	4.53±0.653	4.45±0.641	0.110
Adequacy of educator’s attitude	4.73±0.534	4.67±0.558	4.70±0.546	0.422
Comparative satisfaction with self-directed learning with video clips	4.05±0.925	4.14±0.951	4.10±0.937	0.533
Comparative satisfaction with face-to-face procedural education learned during clerkship	4.24±0.813	4.39±0.738	4.31±0.778	0.188
Adequacy of feedback during teaching	4.56±0.598	4.70±0.461	4.63±0.537	0.076
Total	22.0±2.558	22.4±2.379	22.2±2.474	0.409
B
Ease of network access	4.46±0.700	4.45±0.773	4.46±0.736	0.921
Smoothness of listening to a lecture (disconnection or buffering)	3.59±0.850	3.63±0.953	3.61±0.901	0.746
Video quality (sharpness)	3.70±0.857	3.55±1.016	3.62±0.941	0.276
Skillfulness of smartphone camerawork (zoom in/out)	4.01±0.759	3.90±0.795	3.96±0.777	0.347
Total	15.8±2.230	15.5±2.530	15.7±2.430	0.527
C
Usefulness of acquiring clinical skills	4.30±0.688	4.32±0.611	4.31±0.649	0.822
Usefulness of preparing for Korean Medical Licensing Examination	4.25±0.670	4.25±0.670	4.30±0.669	0.325
Predictive educational effectiveness compared to face-to-face procedural education after coronavirus disease 2019	3.40±1.252	3.86±1.099	3.63±1.198	0.008[**]
Total	11.95±2.060	12.53±1.851	12.24±1.975	0.045[*]

Data are presented as mean±standard deviation. The p-values are represented by independent two sample t-test.

A: Satisfaction with education (score range, 5–25), B: Satisfaction of educational environment (score range, 4–20), C: Educational Effectiveness (score range, 3–15).

p<0.05,

p<0.01 (statistically significant).

2. Educational environment

Satisfaction with educational environment was positive in the case of students of all grades (M5: 15.8±2.23, M6: 15.5±2.53). Students answered that network access was easy (4.46±0.736) and the skillfulness of smartphone camerawork (zoom in/out) was good (3.96±0.777). However, items related to the smoothness of listening to the lecture such as disconnection or buffering (3.61± 0.901) and video quality (3.62±0.941) showed low relative satisfaction (Table 1).

3. Educational effectiveness

Effectiveness of education was positive in the case of students of all grades (M5: 11.9±2.1, M6: 12.5±1.9). Responses about the usefulness for acquiring clinical skills (4.31±0.649) and for the practical exam for the Korean Medical Licensing Examination (4.30±0.669) were very positive, and there was no difference by grade. More than half of the students responded positively to the question about whether it would be possible to apply the test methodology instead of face-to-face education after COVID-19 (3.63±1.198), although a statistically significantly higher negative response was observed in M5 students (p=0.008) (Table 1).

4. Other comments

The most commonly reported merit of this educational exercise was that students were able to watch the key scenes in detail through the camera compared to face-to-face skills practice. There were some positive comments vis-à-vis all students being able to receive the same education at the same time. However, the fact that they could not perform or receive feedback on the practice directly was noted to be a significant limitation. There were requests about improving video quality and the POVF technique (Table 2).

Table 2

Students’ Feedback for Clinical Skills Education Using Point-of-View Filming Education

Answer	No. of answers
Best lecture	3
Camera work	4
Video quality	8
Repeat playback	7
Disconnection or buffering	2
Network access	1
Running time	1
Preference for face-to-face classes	18
Curriculum change	2
Convenience of feedback	6
Benefits of non-face-to-face	12
Composition of education	4
Contents of education	4
Progress of education	3

5. Correlation between variables

There were positive correlations among all variables (satisfaction with education, educational environment, educational effectiveness). In particular, satisfaction with education and educational effectiveness showed a correlation coefficient of 0.617 (p<0.000), and it was found that students with high satisfaction saw more effectiveness in learning (Table 3).

Table 3

Correlation between Educational Satisfaction and Educational Effectiveness after Point-of-View Filming Education (N=186)

	Satisfaction education of clinical skills		Educational environment of clinical skills education		Educational effectiveness of clinical skills education
	r	p-value	r	p-value	r	p-value
Satisfaction of clinical skills education	1	-	-	-	-	-
Educational environment of clinical skills education	0.371	>0.000[**]	1	-	-	-
Educational effectiveness of clinical skills education	0.617	>0.000[**]	0.270	>0.000[**]	1	-

The p-values are represented by Pearson’s correlation coefficient.

p<0.01 (statistically significant).

Discussion

As the COVID-19 pandemic persists, students continue to voice their growing concern regarding reductions in their face-to-face clinical skills education and the effect this is likely to have on their learning and subsequent clinical performance outcome. Many medical educators are also worried about these problems. This study proposes a complementary mode of education. Though teaching clinical technology remotely can be a considerable challenge, we attempted to conduct the study using a real-time online POVF method using a smartphone.

This method has been investigated previously in relation to medical education. A UK-based study explored POVF as a tool to clinically teach medical students [8]. In 2014, a global, live-streamed surgical teaching session was conducted for surgical students and healthcare professionals using Google Glass [9].

There are two types of POVF education. The first uses fixed cameras to film the lecture while, in the second, educators teach using wearable devices. The method with wearable devices requires purchasing new equipment such as action cameras or microphones, and it is difficult to depict a scene accurately because the operation of wearable devices and their features, such as expanding movement, are not easy to maneuver. Education using a smartphone does not incur additional costs, and any educator can easily take a video. The educator may operate a smartphone with one hand or may need an assistant during class, but the education method using smartphones has an advantage because smartphones are very familiar devices. Therefore, we chose to implement real-time online POVF clinical education using a smartphone. The results of a virtual teaching study on physical exam demonstration helped us decide on this plan [10].

In a survey conducted after the lesson, overall satisfaction with this method of education, online environment, and effectiveness was positive. There was no statistically significant difference in satisfaction with education and educational environment by grade, but there was a statistically significant difference in educational effectiveness by grade. In particular, there was a substantial difference in the questions related to COVID-19. As M6 students were about to take the Korean Medical Licensing Examination, they may have evaluated the predictive educational effectiveness of POVF education more positively.

Although the smoothness of listening to the lecture and video quality were mentioned as disadvantages in the real-time online environment, these obstacles can be overcome with relative ease. From an educational point of view, this method has various advantages. POVF can readily facilitate education for large cohorts of learners and is easy to implement without using special equipment. It is also possible to record videos from a close distance and provide detailed videos to learners. However, as the students pointed out, there was a distinct disadvantage as learners could not practice and receive feedback directly.

This study had some limitations. First, it was conducted at one medical college and, thus, it is not readily generalizable. Second, as the running time was short, there was not enough time for the students to learn multiple clinical skills. Third, it was not confirmed through a pre/post-test how much the students’ competency increased. Therefore, further research is needed that takes these factors into account.

In conclusion, the POVF method was perceived by medical students as a flexible and useful online education mode. Although this method cannot completely replace face-to-face education, it could be useful if used to complement face-to-face education. In addition to the situations where face-to-face education is impossible, it could also play a significant role in pre- or post-training education after COVID-19. It is necessary to investigate the effects of POVF education using other devices and/or on education for other health professions.