Using Simulation to Assess Cardiology Fellow Performance of Transthoracic Echocardiography: Lessons for Training in the COVID-19 Pandemic.

To the Editor:

The coronavirus pandemic (COVID-19) has caused a widespread reduction in echocardiography volumes and direct educator-trainee interactions. Assessing competency in transthoracic echocardiography (TTE) scanning is a core component of the Core Cardiology Training Symposium certification. Current TTE scanning competency is based on self-reported procedural volumes and feedback during image interpretation, which may be limited by the pandemic. Moreover, prior studies evaluating TTE scanning using objective measures have found only modest correlations between scanning volume and skill. Simulation-based training and assessment in echocardiography can be a useful adjunct to traditional methods of education.3, 4, 5

Coincidental to COVID-19, we created a simulation-based scanning task using the 3D Systems U/S Mentor simulation program and mannequin (Simbionix, Airport City, Israel) at the University of Texas Southwestern Medical Center. Standardized tasks were created by the Advanced Echocardiography Training Director (A.B.): nine basic two-dimensional parasternal, apical, and subcostal views for task 1 (Figure 1 ) and focused images for evaluation of pericardial tamponade (task 2) and aortic stenosis (task 3). Fellows were given immediate feedback based on their performance. Two expert sonographers also completed the simulation tasks. A preset checklist for basic competency was created for all two-dimensional views and advanced techniques expected for each task by two level III Core Cardiology Training Symposium–certified echocardiographers (A.C. and A.B.) who independently scored the saved deidentified images at a later date in a blinded fashion on a four-point Likert Scale (0-3).

Figure 1

Images required for tasks 1-3 with scoring system. CW, Continuous wave; IVC, inferior vena cava; LV, left ventricle; LVOT, left ventricular outflow tract; PLAX, parasternal long axis; PW, pulse wave; RV, right ventricle.

General cardiology fellows (13 men, 10 women; 9 first year, 7 second year, and 7 third year) who participated had a median of 100 scans performed per fellow (interquartile range: 50, 200) and a median of 3 months of echocardiography training (interquartile range: 1, 4). No significant difference was observed between first-year fellows' and senior (second- and third-year) fellows' median imaging score (24.5 vs 22.5, P = .14; Table 1 ). No difference was observed between the first-year fellows' and senior fellows' median imaging score on task 2 (14.5 vs 13.3, P = .08) or task 3 (11 vs 12.5, P = .8). When stratified by quartile of TTE scans performed, no significant trend was noted in median imaging scores for task 1 (P trend = .34). Similar results were seen for tasks 2 and 3. Sonographer median imaging scores were higher than the median scores of the overall fellowship for all tasks (Table 1). The intraclass correlation coefficient between the reviewing cardiologists was 0.93 (95% confidence interval, 0.66-1.00) for all task scores.

Table 1

Median scores of fellows and sonographers on assigned tasks

Demographic	Task 1: Basic images	Task 2: Pericardial effusion	Task 3: Aortic stenosis
Overall fellowship	23.0 [21.5, 26.0]	13.5 [11.0, 15.5]	11.5 [10.0, 13.0]
Sonographers	26.3 [26, 26.5]	17.8 [17.5, 18.0]	14.5 [14.0, 15.0]
P value	.04	.06	.04
Training year
First year	24.5 [22, 25.5]	14.5 [13.5, 17.0]	12.5 [11.5, 13.0]
Senior	22.5 [21.5, 24]	13.3 [8.5, 14.0]	11.0 [10.0, 13.0]
P value	.14	.08	.80
Gender
Men	23.0 [21, 24]	14.5 [12.5, 17.0]	13.0 [10.0, 13.5]
Women	22.8 [21.5, 24.5]	13.3 [11.0, 13.5]	11.0 [10.0, 12.5]
P value	.88	.10	.26
Fellows scoring 0 on any item,∗ %	13	61	39
First year, n	2	4	4
Senior, n	1	10	5

Data are reported as median [interquartile range]. The maximum possible score for task 1 was 27, for task 2 was 18, and for task 3 was 15.

Fellows were assigned a score of 0 for any image on the task per the scoring system detailed in Figure 1. The percentage listed represents the percentage of overall fellowship with numbers of first and senior fellows detailed as above.

Training level and number of scans performed did not translate to significant differences in TTE scanning skill when assessed via simulation. Simulation-based training was sensitive to skill differences between expert sonographers and trainees and provided an opportunity for efficient, targeted, and direct feedback. Simulation-based tasks can be modified and repeated over the course of training to document longitudinal progress objectively. Given the challenges of the COVID-19 era that may extend into the foreseeable future, simulation may offer an alternative method of training and assessment without sacrificing valuable time in fellowship training and with minimal risk of exposure. Simulation for TTE education should be considered as a curricular complement for cardiology fellowships.