Podcasting in medical education: a review of the literature.

Podcasts are increasingly being used for medical education, both within teaching institutions and on an international scale by major journals. To date, there are no evidence-based guidelines for the development of educational podcasts. To review the state of the literature, MEDLINE, SCOPUS, and ERIC were searched in May 2016 for articles describing audio format podcasts used in medical education. Eighty-four articles met inclusion criteria. A qualitative synthesis of the evidence was done using Kirkpatrick's model for evaluating outcomes. Twenty-four articles described reaction outcomes, eleven described learning outcomes, and one described behavioral outcomes. None measured patient impact. The literature demonstrates that podcasts are both feasible and accepted by learners. The mean length of reported podcasts was 18 minutes, which falls within the recommended range in at least one paper, and is consistent with reported listener preference. Interview format, clear disclosures, and accurate information were reported as desirable. There is limited evidence showing the efficacy of podcasts as teaching tools, or regarding best practices in making podcasts. More rigorous studies evaluating efficacy, changes in behavior, and changes in patient outcomes need to be performed in order to prove podcasts' value and to justify production costs.

Introduction

The use of audio recordings for medical education has been documented in the literature as far back as 1968, when they were used for asynchronous learning in histology classwork [1]. Podcasting in its current form, as a method of distributing audio content, is much more recent. The first use of the term podcast was in 2004 in The Guardian, and prior to that the term “audio blogging” was used for digital audio content being distributed online [2]. Podcasts are on the rise in medical education and continuing medical education, at least in the fields of critical care and emergency medicine [3]. Many major journals produce podcasts and significant cost and effort is being applied to these products. Practical guidance for the development of podcasts for medical education exists in the medical literature, though evidence-based guidance recommending specific educational techniques does not [4,5].

This literature review will examine the efficacy of podcasts for medical education and seek evidence for best practices in developing medical education podcasts. For the purpose of this review, we define a podcast as: audio (or audio with visual enhancements) broadcasts distributed through the Internet and able to be consumed via platforms: web pages, iTunes, handheld devices, and so forth.

Methods

On May 31, 2016 searches that used a combination of keywords and MeSH terms were run in three common medical databases: MEDLINE, SCOPUS, and ERIC. The MEDLINE search string was ((“Webcasts as Topic” [Mesh]) OR (audioblog OR audio-blog OR podcast OR podcasts OR podcasting OR vodcast OR vodcasts OR vodcasting)) AND “Education, Medical” [Mesh]. This yielded 123 results. The SCOPUS search string was ((audioblog OR audio-blog OR webcast OR webcasts OR webcasting OR podcast OR podcasts OR podcasting OR vodcast OR vodcasts OR vodcasting) AND medical education). This yielded 297 results. The ERIC search string was ((audioblog OR audio-blog OR webcast OR webcasts OR webcasting OR podcast OR podcasts OR podcasting OR vodcast OR vodcasts OR vodcasting) AND medical education). This yielded 20 results. All citations were added to Endnote (Thomson Reuters, New York, USA). Fig. 1 shows the search, identification, screening, and review process as a PRISMA diagram. Screening by abstract and title was performed by two independent reviewers (authors MC and JE). Full texts were reviewed by all authors. Articles were included if they announced the presence of a podcast or reported data on prevalence of use, feasibility of production, or outcomes of the use of audio podcasts for the education of physician track learners (undergraduate medical education in the European system, medical students, residents, fellows, and attending physicians). Podcasts accompanied by images or short videos were included. Excluded were any articles using video-based podcasts, webcasts, or other web-based educational tools. Podcasts indexed as articles were also excluded. Data extraction methodology was applied systematically to all included papers. Articles that included educational outcomes were further categorized based on Kirkpatrick’s outcomes hierarchy [6].

Fig. 1.

PRISMA Flow Diagram

Results

Our initial literature search generated a total of 440 citations. This was reduced to 324 articles after automated and manual de-duplication. Seventeen additional papers were identified through other sources, for a total of 341 records. Initial screening by title and abstract yielded 115 articles. Full text was available and reviewed for 113 of these articles, of which 84 met inclusion criteria. Supplementary Table 1 contains all 84 references, data extraction, and notes.

Fifty-eight of the included articles described prevalence of use, format of existing podcasts and feasibility of production. Of these, 25 describe current usage, 27 have descriptions of existing podcasts, five describe how to make podcasts, and one review addressed feasibility. Table 1 details the content of these papers.

Table 1.

Characteristics of Podcasts Described

Characteristic	No. (%)
Type of article (n=84)
Current usage	25 (30)
Description of existing podcasts	27 (32)
Review ―how to make podcasts	5 (6)
Review ―feasibility	1 (1)
Review ―existing literature	4 (5)
Outcome study―attitudes	11 (13)
Outcome study―knowledge retention	9 (11)
Outcome study―behavior change	1 (1)
Outcome study―patient impact	0
Survey of preferred learning method	2 (2)
Editorial	1 (1)
Specified learner (n=64)
Medical student	39 (61)
Resident	29 (45)
Fellows	11 (17)
Attendings	24 (38)
Format (n=23)
One lecturer/host	11 (48)
One lecturer/host with interview guest	8 (35)
Two lecturers/hosts	4 (17)

Percentages expressed in terms of those explicitly reporting data for a given category.

1. Prevalence and format

Podcast use was reported in a diverse range of fields in medical education in 59 papers, including basic sciences (anatomy, pharmacology) (11), internal medicine (7), surgery (8), emergency medicine (6), pediatrics (5), neurology (5), anesthesiology (3), dermatology (3), family medicine (2), radiology (2), procedural skills (e.g., ultrasound, airway) (2), critical care (2), pathology (1), obstetrics and gynecology (1), and psychiatry (1). Sixty-four papers explicitly stated who a podcast’s target learner was, including medical students, resident physicians, fellows, and attending physicians. Twenty-three reported podcasts created by educators, both classroom and clinical, while nine described podcasts created by professional services. Five podcasts were created by medical learners as part of their own education, or explicitly to teach their peers.

Nineteen papers described audio-only podcasts, while 20 described podcasts with audio and some kind of visual cues, which included still images or short video clips. Again, video-only podcasts were explicitly excluded from this review.

The length of the podcast used was reported in 17 papers and typically ranged from 10 to 20 minutes with a mean of 18 minutes (Fig. 2). Feedback specifically on length was reported to be uniformly positive in one study using 15- to 20-minute recordings [7]. A survey of Canadian anesthesiology residents reported that they preferred a 15- to 30-minute format for didactic talks and a 5- to 15-minute length for other modalities (case discussions, journal article summaries, procedural skills, debates, or discussions). There was almost no interest in content beyond 45 minutes in this group [8].

Fig. 2.

Reported Podcast Lengths

Based on 17 papers that reported podcast duration. Midpoint of range used for articles that reported a range of lengths. For example if article reported podcasts ranged from 10 to 15 minutes, a length of 12.5 minutes was used for this figure.

Both monologue and interview format podcasts were reported. Most dialogue and interview participants were designated as experts, clinicians or faculty, but in four papers at least one of the participants was a learner [9-12]. One study reported positive feedback from learners specifically in regards to a format with a foundation-level doctor asking questions of a more senior physician [13]. Another review of existing podcasts described improved listening experience with interview format [14]. Podcasts structure was rarely reported, but some authors used a modular format including a “radio-drama” with sound effects and atmospheric noises that were applied to each topic [11].

2. Feasibility and cost

Twenty-one papers had commentary on the feasibility of producing podcasts. Common challenges included cost, time commitment, technical skills, sustainability, and copyright concerns. Costs were related to equipment, software, web hosting services, subscription services, and production. Production costs were described in some fashion in six papers [10-12,14-16]. Reported costs were variable (hardware costs ranged from 56 to 800 US dollars) though limited details were provided. The production of podcasts requires at minimum a computer, a microphone, and recording/editing software. Free editing software is available (such as Audacity, http://www.audacityteam.org/), and online hosting and distribution was described either as free through certain portals (i.e., iTunes, YouTube) or available at low cost (as little as 6 US dollars/mo). In terms of cost to the listener, several podcasts were available to be downloaded for free, whereas some were included with a paid subscription to a website, journal, or academic course. Podcast listeners need access to the Internet and a device to play back the content.

Several articles detailed the time required to produce podcasts [17-19]. In three papers, it took an average of 2 to 3 hours to produce a podcast of length ranging from 8 to 10 minutes, with one article not reporting the length.

Podcast creators were found to have a wide range of technical skills, from professional services to educators without formal audio training. Some papers detailed opportunities for students to create their own podcasts for local peer to peer learning; one article described a peer review process and broader distribution [10].

3. Utilization measures

Eleven papers reported some type of podcast utilization (Table 2) [9,11,12,16,20-26]. The most common metrics were total downloads, views, and website traffic. One podcast directed at medical students on surgical rotation reported 100,000 downloads in the first 16 months [26]. No papers measured listener retention within a single episode (the fraction of an episode to which a learner listened).

Table 2.

Characteristics of Podcasts Described

Citation	Utilization measure	Results	Reference summary
Evans [20] (2011)	Downloads	Downloads (67–362)	Study examines the use of and student reaction to a series of screencasts produced to accompany embryology lectures.
	No. of students downloading	No. of students downloading (55–111)
	Average no. of downloads per student	Average no. of downloads per student (1.2–3.7)
	Maximum no. of downloads per student	Maximum no. of downloads per student (4–16)
	Most popular hour of the day for downloading	Most popular hour of the day for downloading (8–9 PM)
	Most popular day of the week for downloading	Most popular day of the week for downloading (Wednesday–Thursday)
	Percentage of total downloads on day before examination	Percentage of total downloads on day before examination (19%–29%)
Heydarpour et al. [21] (2013)	No. of students who downloaded podcast	28 Students (46% of 62 students)	Medical school creates podcast for students.
Lichtenheld et al. [12] (2015)	No. of downloads	148.5 Downloads per month in first 20 weeks of EMIGcast (698 total downloads) with consistent increase in monthly downloads	Emergency Medicine Student Interest Group produced EMIGcast to educate emergency medicine bound medical students.
McHugh et al. [22] (2010)	No. of times accessed	Over 6 weeks, podcasts only accessed 209 times (2.6% of overall use of website). No. of times accessed via iTunes not measured	Podcasts produced as part of hospital audit to decrease surgical site infections.
Moriarity & Burns [23] (2012)	No. of downloads	6,000–13,000 Downloads each week; more than 200,000 downloads in total	Podcast created as part of neurology journal for continuing medical education.
MCinch-Harrach et al. [11] (2013)	No. of times accessed	8 Different podcasts were accessed between 2,000–18,000 times each	Eight podcasts produced for biochemistry courses.
Petrovic et al. [9] (2008)	Listens	Podcasters (6), listeners—online (55), listeners—downloads (10), visitors to website (28), commenters (8)	Pilot study investigating how student-generated podcasts can be used.
	Downloads
	Website visits
	Comments posted
Rowell et al. [24] (2006)	Requests for download	3,268 Requests for Real Simple Syndication download within 5 month period	Weekly podcasts produced for practicing radiologists.
Savel et al. [16] (2007)	No. of downloads	Average of 664 downloads per podcast; podcast feed accessed 68,000 times in first 7 months	Society of Critical Care Medicine produced an educational podcast on critical care medicine.
Shantikumar [25] (2009)	No. of downloads	5,394 Episodes downloaded over 2 month period at average of 93/day	Series of 13 podcasts produced to cover general surgery syllabus for 5th year medical students.

4. Educational outcomes

Several articles reported educational outcomes. These were stratified according to Kirkpatrick’s outcome hierarchy (Fig. 3) and are further described below (Table 3) [17,20,27-35].

Fig. 3.

Kirkpatrick’s Outcomes Hierarchy

Donald Kirkpatrick’s Four Level Evaluation Model is one of the best known evaluation methodologies for educational processes. The number of citations in this study in each category is in parenthesis.

Table 3.

Summary Educational Outcomes

Citation	Control group (N)	Intervention group (N)	Study design	Outcome
Alam et al. [27] (2016)	Podcast only (15)	Podcast and video clips (15), podcast and mental practice (15), podcast, video and mental practice (15)	Pre- and post-training quiz after 1 week	Combined mental practice and modeling group had higher knowledge retention.
Alla & Kirkman [28] (2014)	None	Podcast (28)	Pre- and post-training quiz	Correct answer percentage rose in two topics post-podcast (86% to 92% and 69% to 92%).
Bensalem-Owen et al. [17] (2011)	None	Podcast (10)	Post-podcast quiz	Mean score increased from 9.5 to 13.4 (of 25 questions).
Bhatti et al. [29] (2011)	Lecture (73)	Web-learning with podcast (75)	Pre- and post-training quiz	Podcast group showed significantly more improvement (19.13/25 vs.18.23/25).
Evans [20] (2011)	Traditional lecture (historical comparison)	Podcast (129)	Short answer question examination	Mean scores compared from two consecutive academic years (2nd year had podcast as part of curriculum); mean scores of two subjects increased from 34.3/50 to 36.6/50 and 16.5/25 to 19.4/25, and one subject fell from 18.1/25 to 17.4/25.
Florescu et al. [30] (2015)	Lecture (21) and control (23)	Podcast (20)	Pre- and post-training quiz	Differences among training groups not significant.
Geyer et al. [31] (2008)	Non-podcasted lecture topics (148)	Podcasted lecture topics (148)	Comparison	Students performed better on the questions covered by the podcast than those not covered.
Kurien et al. [32] (2013)	None	Podcast (134)	Post-podcast quiz	28 Students randomly selected to have pre- and post-teaching session quizzes; scores improved from average of 2.75/5 to 4.00/5.
O'Neill et al. [33] (2010)	None	Podcast (138)	Pre- and post-podcast quiz	When analyzing all quiz results there was an average increase of 18.5% in the post-podcast quiz, and in all but one quiz series there was an increase in the students' scores in the post-podcast quiz.; 4.94/18 pre-podcast, 6/18 post-podcast.
Tigges et al. [34] (2016)	Traditional radiology clerkship (1,492)	Vertically integrated radiology course including podcast components (261)	Computer-based radiology test	No significant difference between virtual clerkship and traditional.
Vasilopoulos et al. [35] (2015)	Standard didactic (24)	Podcast (33)	Pre- and post-training quiz	Podcast group had small but significant increase in test scores (mean change 6.21 vs. 5.08); those with more prior podcast experience had larger gains.

1) Reaction—satisfaction and preference

Reactions of learners to podcast interventions were described in 24 articles (Supplementary Table 1). These were assessed primarily through surveys using Likert scale items, multiple choice, and open-ended questions. No papers used standardized validated tools for gauging learner reactions.

Most learners reported podcasts to be useful for learning and easy to use. With respect to learning, listeners found podcasts either no different from classroom education or better than traditional lectures. Many people liked podcasts because of repeatability and convenience, both of which they felt contributed to improved learning. Learners found podcasts to be more effective when visual aids were used [25]. Learners utilized the podcasts differently: some chose to listen to the podcasts prior to a classroom lecture, while others used the podcast to review material following a didactic session. Several papers mentioned that podcasts allowed for greater interaction or connection between teachers and learners [15,19,36]. Conversely, one paper indicated that students disapproved of podcasts because they did not allow learners to ask questions of faculty [37]. Comments supporting the use of humor, personal anecdotes and the “personality” of a podcast were also reported [26].

2) Learning—attitudes, skills, and knowledge

Eleven articles discussed the efficacy of podcasts in affecting learning outcomes. This assessment was usually accomplished through quizzes covering a specific subject within a medical specialty.

One study detailed that experienced educational podcast users (defined as having used four or more educational podcasts previously) benefited more from a new podcast intervention when compared to those with less podcast experience [35]. Nine papers detailed an improvement in test scores when podcasts were utilized, though few studies used a control group for comparison.

3) Behavior—impact on clinical practice

Changes in physician behavior were measured in one study. Quitadamo et al. [38] showed that physicians were found to prescribe fewer proton pump inhibitors after utilizing the podcast than prior to the podcast. However, the extent of the behavior change correlated to the podcast was not statistically significant when compared to conventional training via written synopses [38].

4) Results—impact on patients

No articles described patient outcomes.

Discussion

Podcasting has become mainstream in the medical field, with major medical journals (The New England Journal of Medicine, The Lancet, Journal of Clinical Oncology, Journal of the American Medical Association, Circulation) and journals in medical education (Medical Education, The Clinical Teacher) now producing podcasts. A similarly rapid rise in available online resources, including podcasts, in the fields of emergency and critical care medicine from 2002 to 2013 has been documented [3]. Publication on the topic is also newly common in a variety of fields; a basic search of MEDLINE for the term “podcast*” yields 447 results, with the number of publications steadily increasing from 2005 to 2016. By comparison, this literature review found articles meeting our inclusion criteria being published 4 to 11 times annually from 2006 to 2016. Fig. 4 shows the number of citations per year in our generic MEDLINE podcast search compared to the publications included in this review. It appears that while publications that attempt to more rigorously evaluate podcasts from an educational standpoint have been relatively constant over the past decade, citations discussing podcasts in general increase every year.

Fig. 4.

Number of Citations per Year

A comparison of the citations per year in this study versus all PubMed citations that result from searching the term "podcast*".

The rise in medical podcasting is in keeping with a general societal trend of greater proliferation of popular podcasts, as well as the authors’ personal experiences listening to, sharing, and making podcasts. This also suggests another trend: podcasts are being used in the medical field without evidence-based methodologies and without significant evaluation of their impact, value, or role in medical education. The cause is likely multifactorial: the desire to stay current and engage millennial learners, the overall increase in the supply of and demand for podcasts, and the general impression that podcasts are helpful to medical learners.

1. Feasibility, acceptance, and quality

The broad uptake of podcast production by medical journals reflects feasibility for large institutions, and this review finds many articles reporting low costs and short production times for individuals or groups of individuals producing podcasts. The authors’ own experience is that high quality audio and content requires more time and equipment cost than are reported in the literature. An early review of the quality of podcasts in the medical field found them to be of inferior quality compared to audio recorded for radio [14].

There is little evidence to support particular standard practices when developing a podcast. Wide ranges of lengths are reported; 15 to 20 minutes is a common length and may be a reasonable starting point. Ahn et al. [4] recommended a 10- to 20-minute length. However, no one has reported a comparison of one length to another with regards to outcomes. Formats also vary, with monologue, dialogue and expert interview formats all being reported. A visual component was commonly reported.

Markers of high-quality podcasts were identified in one article based on modified Delphi criteria. They include: disclosing conflicting interests, clearly identifying authors, clear distinction of fact versus opinion, accurate information, and accessibility [39]. More recent publications have reported the development of scoring systems to better assess the quality of online educational resources, including podcasts. These include the ALiEM (Academic Life in Emergency Medicine) AIR (Approved Instructional Resources), Medical Education Translational Resources: Impact and Quality (METRIQ)-8, and METRIQ-5 scoring systems, which emphasize developing concise, relevant, accurate, evidence-based content that is adequately cited for both sources and authorship [40,41]. These criteria, while intended to evaluate existing materials, can be helpful in guiding the development of new podcasts. Suggestions on manners of speaking in audio developed for medical education was not discussed in the literature, but general guidance can be found in Medical Sound Recording, including avoiding jargon, using shorter instead of longer words, and using active vocabulary [42]. Other guidance from this book that can be applied to podcasting is the suggestion of improvising a preliminary audio recording for use in script development.

Learner acceptance of podcasting appears to be well established, with multiple reports of students successfully accessing podcasts and reporting satisfaction, especially with the ability to replay content and convenience of the delivery method. This generally positive attitude toward the medium, albeit with limited data on learning outcomes, appears similar to the literature on the use of social media [43].

2. Evaluating the impact of podcasts on medical education

There is controversy in the literature about adopting this new educational modality without evidence of efficacy [44]. As of this review, the learning outcomes that have been reported are mixed. Some studies showed improvement over traditional modalities and others showed similar efficacy. Most studies were uncontrolled, or podcasting was incorporated into a larger package of interventions. Overall evidence is weak for podcasts as educational tools. However, the same could be said for many traditional teaching techniques in medicine, and medical education research suffers from a lack of major academic currencies: prestige, perceived importance, and funding [45,46]. More rigorous studies may be able to demonstrate how podcasts compare to traditional educational modalities in terms of learning outcomes.

Beyond learning outcomes, there is a single study showing change in physician behavior after a single podcast, with no difference seen between podcast and a written synopsis. No study of change in patient outcomes was found, again underlying the need to better understand the impact of educational tools in general, and podcasts in particular.

Conclusion

The low cost, ease of production, rapid distribution, and general appeal of podcasting has made it more common in medical education. Several studies have demonstrated that implementing podcasts is both feasible and acceptable to a wide variety of medical learners. Though there are no formal guidelines, there seems to be some consensus that these podcasts should be relatively concise (perhaps under 20 minutes) and meet certain quality standards with regards to attribution, evidence, and accuracy. Despite many publications, however, little is known about its efficacy as a teaching method or best practices in terms of content development. The medium is promising as an adjunct to traditional training methodologies, but before institutions commit additional time and resources, educators should focus on developing evaluation metrics for this new tool, assessing its impact on medical learners, and generating evidence-based guidelines for creating new podcasts.