Can cardiac auscultation accuracy be improved with an additional app-based learning tool?

INTRODUCTION: Many institutions use simulation 'events' to instruct cardiac auscultation. Research shows that these 'one and done' events limit repetition, are costly and do not incorporate learning science techniques, such as spaced learning and retrieval practice. The Littmann Learning™ mobile app, which has unlimited access to a large library of real patient heart sounds, is a cost-effective tool that we considered could be leveraged by educators to provide this training.
METHODS: This was a quasi-experimental pre- and post-design consisting of an intervention group (PA22) and a non-equivalent comparator group (PA21). The intervention group used a novel mobile app cardiac auscultation curriculum (MACAC), while the comparator group received standard didactic instruction. One-way analyses of variance were used to analyse the data.
RESULTS: A total of 174 PA students participated in the study. There was a significant (p < 0.001) difference in knowledge and auscultation scores between those who did and did not complete the MACAC. PA22 didactic year knowledge scores were 4.11 and 2.96 points higher than PA21 didactic and clinical year knowledge scores (p < 0.001, d = 1.61 and p < 0.001, d = 1.32), respectively. On average, PA22 didactic year auscultation scores were 0.83 points higher than PA21 clinical year scores (p < 0.001, d = 0.6).
CONCLUSION: Results indicate that students in their didactic year achieved proficiency in clinically identifying heart sounds, despite not having access to a mannequin simulator and not having an opportunity to identify these sounds bedside. Overall, a MACAC may be an effective method to teach cardiac auscultation to medical learners.

INTRODUCTION

Valvular heart disease (VHD) is a major contributor to loss of physical function, quality of life, and longevity. The epidemiology of VHD varies substantially around the world, with a predominance of functional and degenerative disease in high‐income countries and a predominance of rheumatic heart disease in low‐income and middle‐income countries. Although VHD can be diagnosed at any age, prevalence of VHD increases with age. Factors such as the underuse of cardiac auscultation and a low skill base for auscultation among providers contribute to late diagnosis of VHD.

Many institutions use simulation ‘events’ to instruct cardiac auscultation using computerised mannequins. , , , However, the equipment needed for these events can cost institutions upwards of $60,000 USD and do not allow for additional student practice beyond that of the event. Research shows that these isolated simulation events do not allow students to listen to the heart sounds at the numbers of times truly needed to correctly identify them in clinical practice, and they do not incorporate science of learning techniques, such as spaced learning and retrieval practice. , , , Cardiac auscultation remains one of the most valuable bedside diagnostic tools that a clinician can use to detect VHD. , , Therefore, improving students' and clinicians' ability to accurately assess cardiac pathology through physical exam alone, without the use of more expensive imaging techniques, and before significant VHD progression, is imperative to early detection, as well as reduction in mortality and health care costs. ,

Simulation events do not allow students to listen to the heart sounds at the numbers of times truly needed to correctly identify them.

As technology continually and rapidly improves, there are significant opportunities for medical educators to ensure high‐quality, accessible learning such as digital stethoscopes, interactive online materials and medical apps on mobile devices. , , Because 98% of Generation Z own a smartphone, mobile apps provide an opportunity to make cardiac auscultation instruction more accessible to today's learners and still incorporate techniques that are consistent with the science of learning. The Littmann Learning™ mobile app, which has been validated by board‐certified cardiologists, is a promising, relatively inexpensive tool (~$60 USD for a 1‐year subscription) that could be leveraged by educators to provide this training. This app gives students unlimited access to a large library of real patient heart sounds, which provides learners the opportunity for spaced learning and retrieval practice. The app has both learner and teacher modes, enabling faculty to design a curriculum that students can work through in‐class and remotely.

Mobile apps provide an opportunity to make cardiac auscultation instruction more accessible to today's learners.

Provides learners the opportunity for spaced learning and retrieval practice.

Physician assistants (PAs) practice internationally in every subspecialty of medicine, thus highlighting the importance of cardiac training during PA education. Almost all of what is known about auscultation education has been conducted primarily with medical students; thus, generating knowledge in PAs regarding cardiac auscultation is important. The objectives of this study were to assess the impact of this mobile app cardiac auscultation curriculum (MACAC) on PA students' knowledge and skills in recognising abnormal heart sounds.

METHODS

This Institutional Review Board approved this study (#IRB00061264).

Study participants

This study used a cross‐sectional, quasi‐experimental design involving two groups: (1) a MACAC user group (PA22) and (2) a comparator group (PA21). The groups were an opportunity sample of two student class cohorts enrolled in a PA programme at one institution. The first cohort was students who will graduate in the spring of 2022 (PA22). These students were in their first (didactic) year of the programme and completed the MACAC. The second cohort of students was second (clinical) year students who graduated in the spring of 2021 (PA21). The PA21 cohort did not complete the MACAC. Although participating in the curricular and assessment activities was mandatory, the pre‐ and post‐questionnaires were optional.

During their didactic year, both classes received the same instruction on the cardiac exam, which included two lectures on cardiac auscultation and a corresponding hands‐on physical exam lab. Objectives for these sessions focused on recognising anatomy, history taking, documentation and basic cardiac exam mechanics. This portion of cardiac instruction is unchanged from prior cohorts and occurs near the mid‐point of the didactic year of training. Although PA21 did not complete the MACAC, they did complete five clerkship rotations where they interviewed and examined patients daily. Further, they participated in a physical exam review session and attended a cardiac refresher lecture in the same week of their final assessment.

MACAC Part 1: Littmann Learning™ mobile app modules

All PA22 students purchased the Littmann Learning™ mobile app. These students independently completed 14 listening modules that were self‐paced but corresponded with weekly learning objectives. Each module contained 5–20 tasks, and each task gave opportunity to listen to up to four sounds each. When an answer was chosen for the task, information about that sound was displayed, regardless of accuracy. Students could attempt each task as many times as they wished; however, a correct answer had to be chosen before attempting the next task. The modules were completed when all tasks are complete. Students could repeat modules as many times as they wanted. Table 1 shows the MACAC curriculum and objectives. Faculty were able to monitor completion of the modules as well as engagement with the app through a faculty‐only downloadable transcript provided by the company.

TABLE 1

Mobile app cardiac auscultation curriculum

Week	Module/listening session	Objectives for the week
1	Intro to Littmann Learning (tutorial)	Become familiar with the Littmann Learning app.
		Know how to locate and complete the modules.
		Once inside the module, know how to locate information about the heart sound you are listening to.
2	Fundamentals 1:1, 1:2	Become familiar with what a normal heart sound without pathology sounds like.
3	Fundamentals 1:3, intermediate 1:1 Listening session 1	Become familiar with what a heart sound that has a murmur sounds like.
		Listen to the following heart sounds and apply them to a clinical scenario: Normal, Tachycardia, Bradycardia, Aortic Stenosis
4	Intermediate 1:2, 1:3	Be able to discern a heart sound with any kind of murmur between a heart sound without any pathology.
		Listen to the following heart sounds and apply them to a clinical scenario: MVP, MS, MI, AS, AI, S3, S4 and Split S2.
5	Intermediate 3:1 Listening session 2	Know what types of heart murmurs are pathologic and what types are innocent (written).
		Know which heart murmurs are diastolic and which heart murmurs are systolic (written).
		Listen to the following heart sounds and apply them to a clinical scenario: PS, AI, VSD and PDA.
6	Intermediate 3:2, 3:3	Recognise the sound of a pathologic heart murmur.
7	Fundamentals 3:1, 3:2 Listening session 3	Distinguish between a normal heart sound with and a heart sound with a click, split or S4.
		Distinguish between a heart sound with and without an arrhythmia.
		Listen to the following heart sounds and apply them to a clinical scenario: PVC, MI and ASD.
8	Term break week	By this point in the curriculum, students should be able to:
		Distinguish between a heart sound that does or does not have a murmur.
		Distinguish between a heart sound that does or does not have an abnormal rhythm or extra sound.
		Know which heart murmurs are systolic and which are diastolic.
		Know what types of heart murmurs are pathologic and what types are innocent.
9	Advanced 1:1 Listening Session‐ 4	After listening to a heart sound, determine if the heart murmur is diastolic or systolic
		Listen to the following heart sounds and apply them to a clinical scenario: MS, MVP (Review‐ AS, AI, VSD)
10	Advanced 1:2, 1:3	After listening to a heart sound, begin to distinguish between a pathologic and innocent murmur
11	Term break week	Term break week
12	Listening session 5	By this point in the curriculum, along with midterm objectives, students should be able to listen to a heart sound and:
		Distinguish between a heart sound that does or does not have a murmur.
		Distinguish between a heart sound that does or does not have an abnormal rhythm or extra sound.
		Distinguish between a systolic and diastolic heart murmur

Students could repeat modules as many times as they wanted.

MACAC Part 2: Listening sessions

All PA22 students also attended five ‘listening sessions’ (LS), run by two authors (SG and NB). Each session was 45 minutes in length and delivered synchronously using PowerPoint™ and videoconferencing software. For each LS, students utilised their own Littmann Learning™ mobile app to digitally auscultate approximately four different pre‐selected heart sounds. For each individual pre‐selected heart sound, students were first asked to independently digitally auscultate and identify whether the sound was normal or abnormal. A supporting clinical vignette was then presented on a slide. Through digital auscultation on their mobile app, students listened to the heart sound again, wrote down a description of what they heard and shared it through the web‐based chat. The next slide presented an anatomic model of the heart. Students traced the flow of blood and identified potential areas of turbulence in relation to the heart sound while simultaneously auscultating through the app. Faculty then used a slide pen to draw and visually demonstrate the accurate pathology. The last slide presented an accurate written description as well as final interpretation of the heart sound so that students could compare against their own. Using their app, students virtually auscultated one last time while viewing the final diagnosis presented on the slide, bringing the total number of repetitions for each pre‐selected heart sound to four. The fifth LS was a review of previously played heart sounds. Through completion of the five LS, approximately 13 distinct heart sounds were played 5–6 times each.

Measures

Prior to receiving any cardiac exam instruction, students in the PA22 cohort completed optional pre‐MACAC questionnaires including demographic (gender, race, age group and cardiac experience) and knowledge items (Table 2). The knowledge assessment asked students to identify 12 heart sounds through text descriptions. The 12 cardiac sounds on the questionnaire are commonly diagnosed in clinical practice as well as expected sounds from conditions tested on PA certifying examinations.

TABLE 2

Knowledge assessment

From the sound bank listed below, please write in the most correct answer for the following heart sound descriptions:
Mitral regurgitation	Aortic stenosis	Atrial fibrillation
Pulmonic stenosis	S3/S4	Atrial septal defect
Mitral valve prolapse	Ventricular septal defect	Patent ductus arteriosus
Mitral stenosis	Aortic regurgitation	Tricuspid regurgitation
A mid‐systolic click.
A pansystolic blowing murmur heard best at the left lower sternal border which radiates to the right sternum and xiphoid.
A blowing, diastolic decrescendo murmur heard best at the 2nd‐4th Left ICS, radiating to the apex and right sternal border.
A continuous machine like murmur with a wide pulse pressure.
Extra heart sounds heard during diastole.
An irregularly irregular heart rhythm.
A low pitch, mid‐diastolic murmur heard best at the apex.
A mid‐systolic murmur heard best at the second right ICS, which radiates to the neck and left sternal border.
A pan‐systolic, blowing murmur heard best at the apex, which radiates to the left axilla.
A systolic crescendo decrescendo murmur heard best at the second and third left ICS, radiating to the left shoulder or neck.
A systolic ejection murmur heard best at the second left ICS with an early to mid‐systolic rumble.
The most common congenital systolic murmur heard best at the left lower sternal border.

The post‐MACAC assessment repeated the 12‐item knowledge section (Table 2) where students identified an abnormal heart sound through a text description. Students also completed a 16‐item clinical skills (auscultation) section where they digitally auscultated and selected either (1) no murmur or murmur, (2) normal or abnormal rhythm or (3) systolic or diastolic murmur. Students in the PA22 cohort completed the knowledge and auscultation assessments 1 week after completion of the entire MACAC. For each heart sound (text description or sound bite), a dichotomous variable was created to indicate whether the student correctly identified the sound (i.e., accuracy) (1) or incorrectly identified the heart sound (0). Individual item and total assessment scores were analysed.

Students in the PA21 cohort did not complete the MACAC. However, during their didactic year, as a baseline, and prior to receiving any cardiac exam content, PA21 completed the same demographics and knowledge questionnaires that the PA22 students completed during their didactic year. Along with the addition of the auscultation assessment (described above), students in the PA21 cohort completed these questionnaires and assessments again in their clinical year during protected time of an on‐campus event, approximately half way through their clinical year rotations. Although sound bites from the clinical skills section were played from the Littmann Learning™ app, none of the assessment sound bites were used during instruction/practice, so PA22 students did not have an unfair advantage over PA21 students.

Statistical analysis

We used SAS v9.4 (Cary, NC) to conduct the analyses. Descriptive statistics of the two cohorts for both the pre‐ and post‐MACAC assessments were generated. One‐way analysis of variance (ANOVA) was conducted. One analysis compared the two groups at similar time points in their didactic years. The other compared the treatment group at the end of the intervention and the comparison group in the middle of their clinical year. Cohen's d, a measurement of effect size, was generated to evaluate practical importance of the group differences. A priori, an alpha of less than 0.05 was used to indicate statistical significance.

RESULTS

A total of 174 PA students completed the assessments and questionnaires. Table 3 presents the PA cohort student characteristics. Useable demographic data were collected on 72 PA22 and 85 PA21 PA students, respectively (80.9%, 100% response rate). The majority of students who completed the optional questionnaire identified as female (82.80%, n = 130), White (76.43%, n = 120), between the ages of 21–25 years (50.96%, n = 80) and reported no prior cardiology/EKG experience (78.98%, n = 124). Similarities between the two groups can be seen in Table 3.

TABLE 3

PA student demographics

Student characteristic	Combined percent (n)	PA22 percent (n)	PA21 percent (n)
Gender
Female	82.80% (n = 130)	87.5% (n = 63)	78.82% (n = 67)
Male	15.92% (n = 25)	12.5% (n = 9)	18.82% (n = 16)
Prefer not to say	1.27% (n = 2)	0% (n = 0)	2.35% (n = 2)
Race
White	76.43% (n = 120)	77.78% (n = 56)	75.29% (n = 64)
Non‐White	19.76% (n = 31)	20.83% (n = 15)	18.82% (n = 16)
Prefer not to say	3.82% (n = 6)	1.39% (n = 1)	5.88% (n = 5)
Age (in years)
21–25	50.96% (n = 80)	58.33% (n = 42)	44.71% (n = 38)
26–30	38.22% (n = 60)	29.17% (n = 21)	45.88% (n = 39)
31–35	4.46% (n = 7)	8.33% (n = 6)	1.18% (n = 1)
36–40	3.18% (n = 5)	1.39% (n = 1)	4.71% (n = 4)
Prefer not to say	3.18% (n = 5)	2.78% (n = 2)	3.53% (n = 3)
Formal EKG/cardiac training or prior listening experience
No	78.98% (n = 124)	72.22% (n = 52)	84.71% (n = 72)
Yes	8.28% (n = 13)	15.28% (n = 11)	15.29% (n = 13)
Other	12.74% (n = 20)	12.5% (n = 9)	0% (n = 0)

Knowledge assessment

Table 4 presents the PA cohort knowledge data.

TABLE 4

Knowledge and clinical skills results

Knowledge (written test)
Cohort (n)	Mean score; SD	p value
PA22 (pre‐MACAC); (n = 67)	2.73; 1.98
PA22 (didactic year post‐MACAC); (n = 89)	11.78; 0.78
Growth of PA22 from pre‐ and post‐MACAC	9.05 points
PA21 (didactic year no MACAC); (n = 83)	7.67; 2.04
PA21 (clinical year no MACAC); (n = 85)	8.82; 2.12
Growth of PA21 from didactic to clinical year (no MACAC)	1.15 points
PA22 (didactic year post‐MACAC); (n = 89)	11.78; 0.78
PA21 (didactic year no MACAC); (n = 85)	7.67; 2.04
Average difference between groups	4.11 points	<0.001*
		ES = 1.61
PA22 (didactic year post‐MACAC); (n = 89)	11.78; 0.78
PA21 (clinical year no MACAC); (n = 85)	8.82; 2.12
Average difference between groups	2.96 points	<0.001*
		ES = 1.32

Abbreviations: ES, effect size; M, mean; MACAC, mobile app cardiac auscultation curriculum; SD, standard deviation.

Statistically significant.

PA22 students

Pre‐ and post‐MACAC knowledge data were collected on 67 and 89 PA22 didactic year students, respectively (75.3% and 100% response rate). On the 12‐item pre‐MACAC knowledge assessment, the number of correct items ranged from 0 to 7 (22.75% accuracy, mean = 2.73, standard deviation [SD] = 1.98). On the 12‐item post‐MACAC knowledge assessment, scores ranged from 8 to 12 (98.17% accuracy, mean = 11.78, SD = 0.78).

PA21 clinical year students

Knowledge assessment data were collected on 83 PA21 students in the didactic year and 85 students in the clinical year (96.6% and 100% response rate). For PA21 students in their didactic year, out of 12 items, the number of correct items ranged from 1 to 12 (63.9% accuracy, mean = 7.67, SD = 2.04). For PA21 students in their clinical year, out of 12 items, scores ranged from 5 to 12 (73.5% accuracy, mean = 8.82, SD = 2.12).

Comparison

Comparing PA22 students to PA21 students at the same point in time in their respective didactic years, PA22 students who completed the MACAC had a higher knowledge score (mean = 11.78, SD = 0.78) than PA21 students who did not complete the MACAC (mean = 7.67, SD = 2.04). (p < 0.001, d = 1.61). Comparing PA22 didactic year students to PA21 clinical year students, PA22 students who completed the MACAC had a higher knowledge assessment score (mean = 11.78, SD = 0.78) than PA21 students who did not complete the MACAC (mean = 8.82, SD = 2.12; p < 0.001, d = 1.32). The two group differences are not only statistically significant but also are large in size and of practical significance.

Clinical skills (auscultation) assessment

Table 4 presents the PA cohort auscultation data. Auscultation data were collected on 89 PA22 students (100% response rate). Out of 16 items, scores ranged from 8 to 16 (91.9% accuracy, mean = 14.7, SD = 1.31). The same auscultation data were collected on 85 PA21 students in their clinical year (100% response rate). Scores ranged from 11 to 16 (86.7% accuracy, mean = 13.87, SD = 1.39). Comparing PA22 didactic year students to PA21 clinical year students, PA22 students who completed the MACAC had a higher auscultation assessment score (mean = 14.7, SD = 1.31) than PA21 (clinical year) students who did not complete the MACAC (mean = 13.87, SD = 1.39; p < 0.001, d = 0.6). The effect size in this case is medium in size and indicates the group difference is of practical significance.

PA22 students who completed the MACAC had a higher auscultation assessment score.

DISCUSSION

As noted by the large effect size for the comparisons and the medium effect size for auscultation, we found that didactic year students who used the MACAC significantly outperformed the clinical year students both at the same point in time in their didactic curriculum the year before, as well as after the clinical year students had completed half of their clerkship rotations. Our study aligns with Barret et al. (2009), demonstrating that a curriculum which includes repetition in the learning instruction proves to be successful in mastering the detection of heart murmurs. Further, similar to the 2015 study by Multak and Spear, we were able to demonstrate improvements in cardiac knowledge and proficiency in cardiac auscultation through simulation training. However, although Multak and Spear were able to demonstrate post‐auscultation accuracy, this study may have been testing immediate recall, rather than true proficiency, as they tested within a short timeframe of the conclusion of the event. Our research builds on this study, demonstrating cardiac auscultation competency following a longitudinal curriculum, rather than a singular event.

A curriculum which includes repetition in the learning instruction proves to be successful in mastering the detection of heart murmurs.

Even though long‐term competency was not evaluated in the MACAC group, we were able to show that didactic year students were prepared to enter their rotations with greater knowledge and accuracy of heart sounds compared with students who had already undergone bedside training. Additionally, because the cardiac simulation is provided through students' individual mobile device, this programme allows learners to continue improving their ability to detect challenging heart sounds during their clinical rotations. We also envision clinical preceptors and assessors leveraging the technology during clinical rotations for assessments. Equipping students with greater knowledge and accuracy in detecting VHD during their didactic and clinical years may improve patient outcomes later by enabling clinicians to make a diagnosis sooner, reducing their reliance on expensive imaging studies.

Didactic year students were prepared to enter their rotations with greater knowledge and accuracy of heart sounds.

This programme allows learners to continue improving their ability to detect challenging heart sounds.

Limitations

We identified three limitations to this study. First, this was a single institution study with only two cohorts of students. For PA22, the pre‐MACAC knowledge test may have highlighted items likely to be re‐tested, motivating students to spend time studying unfamiliar areas. Additionally, auscultation accuracy also assumes that students would correctly place the stethoscope and perform the cardiac exam in a manner that would elicit the heart sound being played. Finally, we did not conduct a comparative effectiveness study using the MACAC and mannequins. Future research could directly compare whether the MACAC is superior to mannequin education in regards to advantages, outcomes and cost.

CONCLUSION

The educational significance of this study is that we demonstrated that use of a mobile app to create individualised, repetitive and spaced learning facilitated knowledge building and retrieval practice of cardiac auscultation skills. Because the curriculum was delivered through the app, students could review as much or as little as they wanted, and on their own time, demonstrating the importance of spaced learning and repetition for knowledge acquisition. Our programme will continue to utilise the MACAC for future cohorts; however, in order to allow listening practice to be more self‐driven and specific to the needs of the learner, subsequent cohorts will have fewer mandated modules to complete. Instead, learners will be encouraged to focus on mastery of the final learning outcomes through completion of as many modules as they individually need in order to demonstrate competency.

Finally, this curriculum model is seemingly transferable. Using appropriate educational tools, we plan to apply the same concepts of repetition, spaced learning and individual practice to other instruction such as pulmonary auscultation and electrocardiogram interpretation. Next steps for this project include evaluating whether completion of the MACAC is associated with greater ability to detect abnormal heart sounds at later points in training and practice. In summary, medical educators who teach cardiac auscultation should consider approaches beyond mannequins and singular simulation events to enhance student access to heart sounds and improve their clinical skills.

CONFLICT OF INTEREST

The authors do not identify any conflicts of interest or have any financial disclosures to report. Specifically, the authors do not have any marketing or financial agreements with Littmann Learning™.

FUNDING INFORMATION

Not applicable.

ETHICS STATEMENT

This Institutional Review Board of the host institution approved this study (#IRB00061264).