An evaluation of medical student-led podcasts: what are the lessons learnt?

BACKGROUND: Student-led podcasts were developed by 5th year Peninsula Medical School students as part of an educational grant. The students completed 35 video podcasts using PREZI software, and based on clinical indicative presentations of the Peninsula Medical School curriculum.
METHODS: Third, 4th and 5th year medical students were invited to complete the evaluation of the indicative presentation video podcasts they watched. Both quantitative and qualitative data were collected through anonymized questionnaires. A thematic analysis of qualitative data was carried out.
RESULTS: Seven hundred and fifty students were invited to evaluate the podcasts of which 142 responded to the email. One hundred and forty-two students were assigned podcasts, of whom 122 completed the podcast questionnaire (85.9%), with 20 students dropping out for unknown reasons. The majority of the students found the podcasts to be clear, of an appropriate length, targeted at the right academic level and providing a good method of learning. However, there were mixed views in relation to the preference of podcasts over conventional learning methods. The thematic analysis identified positive comments and areas of improvement for the podcasts.
CONCLUSION: Podcasts conducted in an interview style with an engaging voice and images are thought to help maintain student engagement from their perspective. Further evaluation/research is required to help establish the correct depth and breadth of information to be included in podcasts.

Background

Medical education is currently undergoing large scale changes. There is an increasing demand to evolve and adapt delivery methods for learning, teaching and assessment in this ever-changing field. Traditional teacher–student based learning schemes have gradually given way to the concept of the independent learner. With constant developments in information technology, resources can be shared globally through reusable learning objects (RLOs).

A digital learning object is one that can be electronically stored and may utilize text, animations, audio, and videos to support and enhance learning. Due to their electronic nature,1 video podcasts have an added advantage of engaging students in multiple learning styles. The role of podcasts as the primary source of knowledge has been questionable in the past. In combination with other methods, podcasts have been shown to improve results.2

One study suggests that podcasts in conjunction with quizzes improve knowledge retention compared to traditional live lectures.3 A study from Plymouth, UK compared the use of podcasts with more traditional handouts in self-directed education across three common ear, nose, and throat (ENT) conditions namely epistaxis, otitis media, and tonsillitis.3 The authors developed written handouts and video podcasts for these conditions. Participating students were assessed by multiple choice questions (MCQ) test as a base line and 1 month after the intervention. The result of this randomized controlled trial demonstrated there was no significant statistical difference in the test scores of both groups though a type II error could not be excluded. The students who were randomized to the podcasts resource enjoyed using it and felt it would be advantageous to use it more in the curriculum.3

Very few studies have directly compared the effect of podcasts with other self-directed learning (SDL) tools, in terms of learning topics in medicine.2,3

The aim of this project was for medical students to evaluate the indicative presentation podcasts and identify areas for improvement when developing further video podcasts.

Methods

Student-led podcasts were developed by 5th year Peninsula Medical School students as part of an educational grant obtained by the senior author (HK) from the school’s Division of Medical Education. The project was approved by the Curriculum and Learning group. The students completed 35 video podcasts based on clinical indicative presentations, part of the Peninsula Medical School curriculum. The video pod-casts were developed using PREZI software and all students received technical advice on developing the video podcasts by the technology-enhanced learning team. The podcasts ranged from 3 minutes and 23 seconds to 13 minutes 49 seconds.

All podcasts followed the scheme of the indicative presentations with a clinical presentation, underpinning basic sciences, diagnostic approach and evidenced-based treatment lines. A feedback questionnaire for the learner to complete anonymously after watching the video podcast was also incorporated. The video podcasts have been in use for over 2 years and are available through the school’s digital learning environment.

Third-year Peninsula Schools of Medicine and Dentistry students, and 4th and 5th year Peninsula Medical School students were invited to complete the evaluation of the indicative presentation video podcasts they watched, through an email sent by two students at the time (coauthors of the article). It was highlighted that participation in the evaluation of the project was voluntary and with the aim of improving future podcasts. Voluntary completion of the anonymous survey was deemed as giving consent to participate.

Clinical year students were chosen as the target audience due to their previous exposure to a variety of teaching methods earlier on in the curriculum. The students were given three days to respond to the email. The podcasts were split into three different categories depending on their duration, namely short, medium and long. Podcast sets were produced, which included a podcast from each of the three duration categories. For the purpose of the evaluation, each student completed the one podcast set they were given access to. Students were given 5 days to watch the allocated podcasts.

Student survey

Participants were asked to respond to five key statements, which were used to collect quantitative data. A further three questions were used for qualitative analysis (Table 1).

Table 1

The podcast questionnaire

Quantitative aspects	Qualitative aspects
The podcast was clear	What was the best thing about the podcast?
The podcast was of an appropriate length	What was the worst thing about the podcast?
The content of the podcast was appropriate for my level	What could be improved upon in the resource?
The podcast provides an efficient way to learn
I prefer the podcast to conventional learning resources

Analysis of results

Qualitative data were analyzed using NVivo software. Potential themes in the feedback were identified. A thematic analysis was conducted to identify and extract common recurring themes. Two positive and negative themes were chosen for further exploration. A code was given to the feedback that gave a direct answer to the question. The quantitative data was tabulated.

Results

Seven hundred and fifty students were invited to evaluate the podcasts of whom 142 responded to the email. One hundred and forty-two students were assigned podcasts of whom 122 completed the podcast questionnaire (85.9%), with 20 students dropping out for unknown reasons.

The majority of the students found the podcasts to be clear, of an appropriate length, targeted at the right academic level and providing a good method of learning (Figures 1–4). However, there were mixed views in relation to the preference for podcasts over conventional learning methods (Figure 5).

Figure 1

Clarity of podcast.

Figure 2

Appropriateness of length of podcast.

Figure 3

Appropriateness of content.

Figure 4

Efficiency of learning

Figure 5

Preference of podcasts over conventional learning methods

Maximum satisfaction was demonstrated by the small and medium length podcasts, with the majority of the dissatisfaction arising from the longer duration podcasts

The thematic analysis elicited positive and negative aspects of podcasts as a learning resource. The feedback from participants highlighted “overview of the information” and “use of images” as the strengths of the resource. Students in the 4th and 5th years highlighted the importance of integrating specialist knowledge and National Institute of Clinical Excellence (NICE) guidelines within podcasts. Areas of further development included the tone and voice of the presenter, and individual explanations. Examples of student comments that represent recurring themes are illustrated in Table 2.

Table 2

Positive comments and areas for improvement from the thematic analysis

Positive comments	Areas for improvement
These podcasts are the best thing ever. It makes the learning of the indicative presentations so much easier and interactive. A brilliant idea.	The voice was clear but could do with a little more enthusiasm to improve engagement.
Very good use of flow starting with anatomy and epidemiology before moving onto the specific aims of the podcasts with good resources at the end.	The presenter spoke very clearly, but more variation in his tone would have made the podcast easier to listen to – a personal touch every now and again throughout the podcast would have helped maintain the viewers’ attention.
Very clear and good pace of the video. I have always struggled to find a way to learn about diabetes. I think this video is the best way to learn about it that I’ve come across when you don’t have the time to spend hours on it.	I think that the video could have been split into two videos, for example one on acute renal failure and another on chronic renal failure.
Full disease framework employed, brief and concise for all the discussed areas.	Hemodialysis should again be a separate video – the viewer is never going to retain any of the information because it is so overwhelming.
Good level of detail within the podcast and liked the layout of the session, going through causes, investigations and management was really useful for clinical practice.	Should be accompanied by a video/pictures to show the examination being done and/or the abnormalities being talked about.
Good diagram of the visual pathway and good use of colors to explain lesions.	Use of a clinical scenario that remained consistent throughout the video to demonstrate some of the symptoms, tests to be done etc.

Discussion

Students recognize podcasts as an effective self-directed learning method.

The evaluation of the student-led podcasts demonstrates that podcasts should be for an adequate length of time. Long, narrated podcasts are thought to be difficult to follow, with students having to listen to the entire podcast, without being able to skip sections. We postulate that preference for short and medium length podcasts may have also been influenced by the students’ overall experience, which included, but was not limited to, the tone of voice, use of images, and simplification of explanations. Furthermore, the longer podcasts attempted to cover greater details about the topics, resulting in reduced concentration. It was harder for students to stay engaged with the podcasts that had a longer duration.

One of the key aspects identified by the thematic analysis is the presentation of the podcast. Students are affected by the tone and voice of the presenter. The feedback highlighted the importance of an enthusiastic tone throughout the presentation, which ensured student concentration was maintained. A systematic review into general medical podcasts has shown the mode of delivery greatly affects a student’s experience.4 A podcast conducted by a professional interviewer creates a more natural sounding conversation making it easier to stay engaged. The presentation can be seriously affected by the presenter reading off a script, and should be avoided3.

The 35 podcasts evaluated covered a variety of topics based on the core Bachelor of Medicine, Bachelor of Surgery curriculum. Based on this we suggest that students in their preclinical years may benefit from podcasts based on anatomy, physiology, and pathophysiology, which can be linked to further podcasts that focus on clinical examinations, investigations and management of diseases.

There were differences between 3rd-, 4th-, and 5th-year responses, with 4th and 5th year students demonstrating a more positive response to podcasts. This may be a result of personal interest demonstrated by 4th and 5th year students with fast approaching final medical school exams.

The main limitation of this evaluation relates to students being able to watch the podcasts within their own time. As a result, extrinsic factors could greatly alter the feedback from students. This may explain the low response rate of the evaluation. Furthermore, it would be hard to determine whether students watched the entire podcast. There is also an unequal split between 3rd, 4th and 5th year students; this may be influenced by assessments taking place within the year group. There is a need for a validation study of podcasts versus gold standards of medical education instruction.

Conclusion

The design of podcasts can greatly influence the learning experience of medical students. Podcasts conducted in an interview style with an engaging voice and images are thought to help maintain student engagement from their perspective. Further evaluation/research is required to help establish the correct depth and breadth of information to be included in podcasts.