[Knowledge retention after focused cardiac ultrasound training: a prospective cohort pilot study].

BACKGROUND AND OBJECTIVES: Focused Cardiac Ultrasound (FoCUS) has proven instrumental in guiding anesthesiologists' clinical decision-making process. Training residents to perform and interpret FoCUS is both feasible and effective. However, the degree of knowledge retention after FoCUS training remains a subject of debate. We sought to provide a description of our 4-week FoCUS curriculum, and to assess the knowledge retention among anesthesia residents at 6 months after FoCUS rotation.
METHODS: A prospective analysis involving eleven senior anesthesia residents was carried out. At end of FoCUS Rotation (EOR) participants completed a questionnaire (evaluating the number of scans completed and residents' self-rated knowledge and comfort level with FoCUS), and a multiple-choice FoCUS exam comprised of written- and video-based questions. Six months later, participants completed a follow-up questionnaire and a similar exam. Self-rated knowledge and exam scores were compared at EOR and after 6 months. Spearman correlations were conducted to test the relationship between number of scans completed and exam scores, perceived knowledge and exam scores, and number of scans and perceived knowledge.
RESULTS: Mean exam scores (out of 50) were 44.1 at EOR and 43 at the 6-month follow-up. Residents had significantly higher perceived knowledge (out of 10) at EOR (8.0) than at the 6-month follow-up (5.5), p=0.003. At the EOR, all trainees felt comfortable using FoCUS, and at 6 months 10/11 still felt comfortable. All the trainees had used FoCUS in their clinical practice after EOR, and the most cited reason for not using FoCUS more frequently was the lack of perceived clinical need. A strong and statistically significant (rho=0.804, p=0.005) correlation between number of scans completed during the FoCUS rotation and 6-month follow-up perceived knowledge was observed.
CONCLUSION: Four weeks of intensive FoCUS training results in adequate knowledge acquisition and 6-month knowledge retention.

Introduction

Point-of-Care Ultrasound (POCUS) has become a key diagnostic tool for several medical specialties including anesthesiology, emergency medicine, critical care, and internal medicine.1, 2, 3 POCUS training is now an integral component of undergraduate medical curricula in several schools.4, 5 Focused Cardiac Ultrasound (FoCUS), one of the POCUS modalities, is a valuable diagnostic tool in a variety of clinical settings. For instance, FoCUS-guided therapy in subacute shock is associated with improved survival. When performed by anesthesiologists, FoCUS can be instrumental in the clinical decision-making process, providing invaluable information that changes perioperative care during both elective and emergency situations.8, 9, 10, 11, 12

Training residents to perform and interpret FoCUS is both feasible and effective.13, 14, 15 Accordingly, a FoCUS rotation has recently been introduced into the anesthesia residency training program at Queen's University, providing residents with an opportunity to integrate this diagnostic modality into their clinical armamentarium. However, the degree of knowledge retention after this FoCUS training has yet to be determined. This is important because while many educational interventions have shown immediate gains in knowledge and skills, these gains are often subject to subsequent decay over a period of time. We hypothesized that 4 weeks of intensive FoCUS training would allow trainees to maintain knowledge and interpretation skills 6 months later. The objective of this pilot study, therefore, was to assess the knowledge retention (i.e., theoretical knowledge, interpretation and clinical correlation skills) at 6 months among anesthesia residents following the FoCUS rotation. Additionally, we sought to provide a thorough description of our current curriculum (and the solutions we have found over time to major barriers to its implementation and maintenance), in the hopes that it may serve as a template to institutions willing to establish a FoCUS training program.

Structure of rotation

The anesthesia residency training program at Queen's University has a well established 4-week FoCUS rotation that allows its residents to pursue Level I (basic) competence in FoCUS.13, 17 Learning objectives of the rotation include residents obtaining and displaying an understanding of Ultrasound (US) basic principles, showing consistently the ability to acquire and interpret images from 3 Windows (parasternal, apical and subcostal, including assessment of the inferior vena cava), and most importantly, showing the ability to integrate the acquired information into the clinical context. Given the current lack of formal training guidelines for FoCUS in anesthesiology, the structure and learning objectives of our curriculum were adapted from the American College of Chest Physicians statement on basic critical care echocardiography.

The rotation (Fig. 1) consists of weekly didactic teaching sessions with staff anesthesiologists trained in echocardiography, a commercially available online course, scanning practice sessions on an echo simulator, as well as supervised and independent scanning of perioperative patients. The rotation is divided in three phases.

Figure 1

Outline of FoCUS rotation. CAE, “Focused Transthoracic Cardiac Echo” online course by CAE Healthcare; PLAX, Parasternal long axis view; PSAX, Parasternal short axis view; Apical 4/5C, Apical 4-chamber and 5-chamber views; SC/IVC, subcostal and inferior vena cava views.

In Phase 1, trainees are required to complete “Part 1: Basic Principles and 2-D Echo” and “Part 2: Color Doppler, M-Mode and Harmonic Imaging” of a 6-part online course (CAE Healthcare (Montreal, Canada) ICCU “Focused Transthoracic Cardiac Echo” – 7.5 hour course). Successful completion of Parts 1 and 2 is achieved by obtaining a set pass score.

In Phase 2, residents complete the remaining modules of the online course – “Parasternal long-axis”, “Parasternal short-axis”, “Apical four- and five-chamber”, and “Subcostal four-chamber, IVC, aorta”, and supplement it with didactic mini-seminars of basic topics (US principles, knobology, basic views, left and right ventricles – structure and function, Doppler, bedside hemodynamic and volume assessment, and aortic stenosis – although briefly discussed throughout the rotation, formal training in other valvulopathies is not part of the curriculum) delivered throughout the rotation by attending anesthesiologists trained in echocardiography. Residents also begin scanning practice sessions on a simulator (CAE Vimedix US simulator), followed by supervised bedside scanning on patients. After 10 supervised exams, residents begin Phase 3 – independent scanning of perioperative patients with weekly preceptor debriefing.

During Phase 3, residents electively scan pre- and post-operative patients for educational purposes, and perform the initial perioperative scan of patients with a clinical indication. Images and/or video clips are recorded for subsequent review by a certified staff supervisor. Residents complete a report on all patients they have scanned. They also have at least 6 opportunities to shadow a sonographer in the cardiology echocardiography lab (Fig. 1) to further improve scanning technique. On 6 occasions a staff anesthesiologist supervisor reviews the scans performed in the preceding days and completes a feedback form – detailing strengths and areas for improvement. Additionally, trainees use these review days for proctored scanning, with direct real-time feedback from the supervisor.

In order to be deemed successful in the FoCUS rotation, trainees are required to (1) complete a minimum of fifty FoCUS scans reviewed by a staff supervisor, (2) complete the online course along with an acceptable “pass-score” (80%) in its final exam, (3) complete a written (multiple-choice) and practical (video clip-based) exam at the End Of the Rotation (EOR) with a minimum score of 80%, (4) complete an observed practical bedside FoCUS exam at the EOR with adequate image acquisition and interpretation as assessed by a staff supervisor, and (5) complete a Case Management Rounds presentation at the EOR, outlining the specific FoCUS findings and its impact on perioperative patient management.

Methods

Study design and participants

Following approval from the Queen's University Research Ethics Board, senior anesthesiology residents (Postgraduate Years – PGY: 3–5) were recruited to the study on a voluntary basis and gave written informed consent. The intervention was the Queen's University Anesthesiology FoCUS rotation. A prospective analysis was undertaken from July 2013 to June 2015.

Data collection

At the EOR, participants completed a brief questionnaire including PGY of training, number of scans completed, resident's self-rated knowledge (10-point scale) pre- and post-FoCUS rotation, and comfort level (yes/no) with FoCUS. In addition, they completed an exam consisting of 50 multiple choice questions (37 written- and 13 video-based) that assessed US fundamentals, anatomy, imaging windows, and interpretation of static images and video clips of normal and abnormal findings. Six months later, residents completed a follow-up questionnaire (aiming to collect similar data in addition to barriers to the application of FoCUS in clinical practice) and an exam consisting of similar content and questions. Self-rated knowledge and exam scores were compared between EOR and 6-month follow-up.

Statistical analysis

Data were analyzed using the Statistical Package for the Social Sciences (SPSS 24; IBM, Armonk, NY, USA, 2016). Descriptive statistics were provided for the EOR and 6-month follow-up questionnaire data. Non-parametric tests of association were used due to the small sample size. Wilcoxon signed rank tests were used to compare total, video- and written-based exam scores as well as perceived knowledge between EOR and 6-month follow-up. Spearman correlations were conducted to test the relationship between number of scans completed and exam scores, perceived knowledge and exam scores, and number of scans and perceived knowledge.

Results

Eleven anesthesiology residents completed the FoCUS rotation and the 6-month follow-up assessment. The mean exam score (out of 50) was 44.1 at the EOR and 43 for the 6-month follow-up (Table 1). The mean video exam scores (out of 13) for the two time points were 12.3 and 11.5 respectively; the equivalent values for the written exam scores (out of 37) were 31.8 and 31.5. Fig. 2 provides the median values; none of the three comparisons attained statistical significance using the Wilcoxon signed rank test.

Table 1

Demographic and descriptive information of anesthesiology residents in FoCUS (n = 11).

Resident	Level of study (PGY level)	Prea FoCUS self-rated knowledge (of 10)	Post FoCUS self-rated knowledge at EOR (of 10)	Post FoCUS self-rated knowledge at 6-month follow-up (of 10)	Number of scans completed during FoCUS rotation	Exam score at EOR (of 50)	Number of scans completed after EOR (6-month follow-up)	Exam score at 6-month follow-up (of 50)
1	4	2	8	6	64	44	10	44
2	3	2	9	7	80	41	2	41
3	3	0	9	7	57	47	3	42
4	4	4	8	6	85	47	2	46
5	3	1	7	6	65	41	n/ab	45
6	3	2	7.5	5	50	39	3	42
7	3	1	7.5	5	50	47	2	46
8	5	1	7	5	n/ab	44	5	42
9	4	2	9	5	52	44	4	40
10	4	1	9	5	53	47	5	46
11	4	1	8	4	50	44	6	39
Mean	4	1.5	8.0	5.5	60.6	44.1	4.2	43.0
SD		1.0	0.8	0.9	12.8	2.8	2.5	2.5

PGY, post-graduate year; FoCUS, focused cardiac ultrasound; EOR, end of rotation.

The pre FoCUS self-rated knowledge data were collected at the EOR.

n/a, Not available. The total number of scans was not reported by the resident on the respective time points. Hence, the mean scores and standard deviations (SD) for the columns “Number of scans completed during FoCUS rotation” and “Number of scans completed after EOR (6-month follow-up)” were calculated based on the 10 residents whose data were available.

Figure 2

Wilcoxon signed rank test p = 0.12 (Video), p = 0.72 (Written), p = 0.50 (Total).

At the EOR, all trainees felt comfortable using FoCUS in their clinical practice, whereas at 6 months 10 out of 11 trainees still felt comfortable. Residents had significantly higher perceived knowledge (out of 10) at EOR (8.0) than at the 6-month follow-up (5.5), p = 0.003, but their knowledge perception was not statistically significantly correlated with their exam scores either at the EOR (rho = 0.32, p = 0.34), or at the 6-month follow-up (rho = 0.17, p = 0.62).

The average number of scans completed during the FoCUS rotation and the following 6 months were 61 and 4, respectively, and the most cited reason for not using FoCUS more frequently was the lack of perceived clinical need (Table 1). There was no correlation between number of scans completed during the FoCUS rotation and the post exam score (rho = 0.032, p = 0.93), but there was a strong and statistically significant (rho = 0.804, p = 0.005) correlation between number of scans completed during the FoCUS rotation and 6-month follow-up perceived knowledge.

Discussion

One goal of medical education is to promote long-term knowledge and skill retention. Following a 13-day US course, novice anesthesia interns’ scores in a knowledge test involving basic echocardiography and perioperative US were sustained and similar to senior residents 90 days later. Similarly, residents’ knowledge of ultrasound increased immediately after a 2.5-hour US workshop (which did not include cardiac imaging), but significantly decreased in a follow-up 12-month assessment. In the present study, residents experienced a negligible decrease in their exam scores 6 months after the completion of the FoCUS rotation (Fig. 2). Whereas infrequently used knowledge and skills generally decay over time, this was not the finding of this study at 6 months. In fact, despite the residents’ perception that their knowledge was worse at 6 months (perhaps because of the less frequent clinical use of FoCUS after the EOR), this was not corroborated by their exam scores between the time points that they were tested. The explanation for such finding is likely multifactorial. Firstly, the fact that all trainees have incorporated FoCUS into their clinical practice may be considered a contributing factor as it is accepted that constant reinforcement of new knowledge leads to longer-term retention. In fact, dispersed learning and reinforcement in this manner is associated with improved long-term knowledge retention. Secondly, proctored US training, a component of our FoCUS curriculum, has been shown to yield knowledge retention. Finally, and perhaps most importantly, over-learning in the initial phases of knowledge acquisition is associated with strong long-term memory, and this might have occurred in our cohort given the robustness (a dedicated 4 weeks of study and practice) of the initial training.

Anesthesiology societies have been repeatedly reminded of the importance of US education in contemporary anesthesia training.6, 26 It is, therefore, foreseeable that FoCUS will become an integral component of anesthesiology training across Canada. Indeed, a recent survey revealed that FoCUS training is currently offered by 92% of Canadian anesthesiology training programs. Overall, the goal would be to achieve Level I (basic) proficiency, for which guidelines have already been proposed.28, 29 However, unlike Emergency and Critical Care Medicine, a formal FoCUS curriculum in Anesthesiology is still lacking, and the Royal College of Physicians and Surgeons of Canada has not yet taken the steps to establish FoCUS as part of the anesthesiology core training requirements.

In our cohort, residents who completed more scans during their FoCUS rotation had higher perceived knowledge at the 6-month follow-up. We speculate that a higher number of scans results in deeper learning, which in turn promotes self-efficacy (i.e., self-confidence). Furthermore, a premise of our FoCUS curriculum is that more scans equals more feedback. Feedback provided by expert staff on multiple occasions may promote experiential learning through enhancement of reflective observation (i.e., reviewing and reflecting on the scan and staff feedback) and abstract conceptualization (i.e., using feedback to learn from the experience). Finally, completing more scans may indicate highly intrinsically motivated individuals (although knowing they were going to be retested cannot be neglected as potentially influencing residents’ behavior during their FoCUS rotation). Intrinsic motivation has been favored in the educational literature as it is associated with self-initiated learning and deep learning. Hence, feedback allows learners habituated in an intrinsically motivated learning approach to integrate new knowledge to already existing knowledge to solve clinical problems.

At present, different guidelines for FoCUS training have been proposed13, 28, 29, 35; however, they are limited by little evidence-based data. A subject of constant debate is the amount of training and clinical experience required to achieve competency.36, 37 While studies have demonstrated that as little as a few hours of training allow participants to reliably identify cardiac abnormalities,38, 39 formal recommendations for proper credentialing are lacking. What seems clear, however, is that adequacy of training should be determined by competency-based assessment. Establishing criteria for maintenance of certification is equally important as practitioners rapidly lose their skills without practice. Ideally, echocardiography societies in conjunction with specialty training committees would create a core set of curricular requirements and competencies pertaining to FoCUS training in anesthesiology postgraduate training programs.

Barriers to the implementation of a FoCUS rotation in anesthesiology training include time constraints, a lack of manpower and equipment, and limited expertise. As we faced all of these barriers at our institution, we endeavoured to implement from the outset a practical and sustainable training program with minimal resource utilization that is tailored to the reality of our institution. Key features to this program include the use of a commercially available online curriculum based on existing guidelines from the International Statement on Training Standards for Critical Care Ultrasonography, and the self-directed nature of the rotation requiring residents to take initiative and ownership of their learning. The success of our curriculum, therefore, depends on highly motivated learners, which was never an issue given the perceived value of FoCUS training. Indeed, 91% of anesthesia residents in the United States agreed that FoCUS should be part of anesthesiology training. Furthermore, our curriculum has been constantly refined over the years in order to meet our local needs and requirements. Of note, given the small nature of our program (20 residents in total) and the great emphasis on self-directed learning, no more than 2 staff supervisors are required to adequately run the FoCUS rotation. We therefore report on a self-directed FoCUS curriculum that has allowed many anesthesia trainees to achieve – and most importantly, to maintain – Level I competency over time.

This study has several limitations. First, residents identified their baseline self-perceived FoCUS knowledge following enrolment in the FoCUS curriculum rather than prior to enrolment. While pre-FoCUS self-perceived knowledge was lower than that at post-FoCUS rotation and at the 6-month follow-up, indicating that residents perceived that their participation in FoCUS training enhanced their knowledge, baseline data would have provided a comparison with the initial/baseline condition. Nevertheless, a previous study using similar assessments revealed that novices without FoCUS training perform poorly and rate their pre-FoCUS knowledge in the same range as residents in the current study (1.5 out of 10). Second, the small sample size may limit our ability to generalize our results to other populations. Additionally, the small nature of our program and the timeline of our study design precluded us from using a control group. Third, although our results showed similar exam scores at the EOR and 6 months later, the 6-month follow-up exam did not include a practical bedside FoCUS scan, and therefore, the trainees’ technical ability to perform FoCUS was not assessed. Fourth, the FoCUS rotation in our institution was optional during our data collection. Therefore, a selection bias (including highly motivated individuals with a specific interest in FoCUS) may have positively influenced our results. Fifth, we did not have a follow-up beyond 6 months due to residents finishing their training. Finally, the multiple-choice examination we used at the EOR and at the 6-month assessment has not been formally validated to assess for a standard level of proficiency in echocardiography. However, we have previously shown that the exam used in this study is able to discriminate between trainees and experts, and that any improvement observed in exam scores after FoCUS training is not simply a result of re-testing.

Summary

This pilot study suggests that 4 weeks of intensive FoCUS training consisting of a combination of didactic and practical sessions results in adequate knowledge acquisition and 6-month knowledge retention. Further studies are required to determine if long-term retention is in fact related to clinical proficiency with FoCUS.

Conflicts of interest

The authors declare no conflicts of interest.