Old Dog, New Trick: Efficacy of Self-Directed Procedural Training for Attending Critical Care Physicians.

Background: In teaching hospitals, the majority of central venous lines (CVL) are placed by trainees, resulting in little opportunity for attending critical care physicians to maintain this procedural skill. Additionally, not all attending critical care physicians have been trained in the most up-to-date method of dynamic ultrasound (US) guided CVL placement. Furthermore, there is no standardized method to assess procedural competency of attending critical care physicians or to train them in the evolving practice of CVL placement. Despite these limitations, attending critical care physicians are ultimately responsible for supervision of CVL placement by trainees. Objective: To assess the utility of an instructional video to impact attending critical care physicians' competency and confidence in dynamic US guided CVL placement.
Methods: A pre-post intervention study was conducted at an academic medical center. Attending critical care physicians were first asked to obtain CVL access on a gelatin model using US guidance. They then participated in the intervention, which consisted of watching a short instructional video demonstrating a method of dynamic US guided CVL placement. They were then asked to obtain access again, this time using the described method. All CVL placements were video recorded to assess competency in dynamic US guided CVL placement as well as the time required to obtain CVL access. Two blinded and independent reviewers evaluated each video with discrepancies resolved by a third reviewer. Participants were also surveyed pre and post intervention to assess their confidence in performing and supervising CVL placement.
Results: A total of 21 attending critical care physicians were included. Pre-intervention, four used dynamic US guidance compared to 16 post-intervention (P < .001). Confidence in both CVL placement and supervision improved post-intervention (P = .03 each). Time required to obtain CVL access did not differ significantly pre and post intervention. The majority (20/21) believed there should be required competency testing for CVL placement. Conclusions: Pre-intervention dynamic US guided CVL competency was poor in this sample of attending critical care physicians but improved significantly with an instructional video intervention. This study suggests there is a role for procedural competency testing among attending critical care physicians, and that significant improvement is achievable with relatively minimal instruction.

Introduction

More than five million CVLs are placed in the United States every year. In academic medical centers, the majority of these are placed by trainees including interns, residents, and fellows, resulting in fewer opportunities for attending physicians to maintain this skill. Additionally, within the last several years, dynamic US guidance has become the standard of care for placement of both internal jugular vein (IJ) and femoral vein CVLs.[2,3] This is a skill that many attending physicians were not taught during their training. The attending physician, however, is still ultimately responsible for providing supervision of trainees for this procedure. This has raised concern as to the competency of attending physicians in placing CVLs. A recent study conducted across all Veterans Affairs hospitals validated this concern by showing large variations in competency among experienced attending physicians. Another study showed only 11% of attending nephrologists meeting a pre-specified passing score for simulated placement of IJ temporary dialysis catheters. Currently, there is no standardized method of assessing and training attending physicians in the skill of US guided central venous lines (CVL) placement.

The purpose of this study was to assess attending critical care physicians’ competency and confidence in dynamic US guided IJ CVL placement and determine the effectiveness of a short instructional video in improving both.

Methods

Participants

Twenty one attending critical care physicians at an academic medical center participated in the study. After an appropriate appraisal, this study was considered exempt from IRB review. The study was conducted from February 2019 to July 2019.

Procedure

After filling out a pre-intervention survey, participants were asked to obtain venous access using US guidance on a gelatin model representing the IJ and carotid artery (Figure 1). Participants were then shown a four-minute instructional video demonstrating the “walk down” method of dynamic US guidance (see Figure 2 for description of “walk down” method). After watching the video, participants were asked again to obtain venous access on the model, this time using the technique described in the video. A post-intervention survey was then completed. Both the pre- and post-intervention attempts at venous access were video recorded. No instruction or help was given to the participants other than the instructional video, and the video was only shown once per participant. The recordings were then analyzed by two independent and blinded reviewers to determine whether the participants used dynamic US guidance to actively track the needle tip during the procedure. Disagreements between the two reviewers were settled by a third blinded reviewer. Time from initial needle insertion to venous access was documented for all attempts.

Figure 1.

Cross-section representation of gelatin model.

Figure 2.

Stepwise description of the “Walk Down” method of dynamic ultrasound guidance.

The gelatin model

Low-fidelity gelatin models were chosen for this project as they have demonstrated comparable, if not superior, quality when compared with high-fidelity models.[6,7] Gelatin models are also low in cost, have high portability, and are easily modified to provide practice for anatomic variation. The gelatin model used in this project (Figure 1) was adapted from a previously described protocol.[8-13] Briefly, the vein was simulated by filling a long balloon with blue colored water and tying the end closed. The artery was simulated using rubber tubing filled with red colored water and tied closed at both ends. The simulated vessels were then suspended in the anatomically correct configuration (vein superficial and lateral to artery) in a gelatin mixture inside of a reusable food storage container. Once solidified, a silicone mat was then placed on top of the gelatin to simulate the resistance of skin.

Measurements

Primary outcome metric was attending physicians’ ability to track the needle tip using dynamic US guidance. Secondary outcome metrics included successful venous access and time from needle insertion to venous access.

The pre-survey asked participants the year they completed training, the number of CVLs they had both supervised and placed in the preceding year, as well as their confidence in both supervising and placing CVLs. The post-survey (which was completed after the participants watched the instructional video and obtained the venous access) assessed their confidence in placing and supervising CVLs, the overall helpfulness of the video, whether the video should be used for training purposes, and if they believed attending critical care physicians should be required to show competency in the procedure.

Statistical analysis

Pre- and post-intervention procedure time (in seconds) was analyzed with a paired t-test after adjusting for missing data (2 participants were unsuccessful in their pre-intervention CVL attempt). Needle tracking during the procedure was recorded as a binary response (1 = yes, 2 = no). Since pre- and post-survey data was measured on a scale and because both the survey data and the needle tracking data showed non-parametric distribution (confirmed by Kurtosis and Shapiro tests), the pre-intervention and post-intervention metrics were compared with a Wilcoxon signed-rank test.

Twenty-one attending critical care physicians participated in the study. The average time since completion of training was 11 years. Twelve of the 21 participants reported that they were taught dynamic US guidance during their training. Participants reported that in the preceding year they had independently placed an average 5.6 CVLs (with a range of 0 to 50) and supervised an average of 14.5 CVLs (with a range of 0 to 50). Baseline characteristics are listed in Table 1. Both confidence in placement of CVLs, as well as supervising CVL placement, when compared to the baseline, were significantly improved post-intervention (P = .03 for both the former and the latter).

Table 1.

Baseline characteristics.

NUMBER OF PARTICIPANTS	21
Average years since completion of training	11
Percentage of participants who were taught dynamic US guidance during their training	57% (12/21)
Average number of CVLs placed independently in the preceding year	5.6
Average number of CVLs supervised in the preceding year	14.5
Average pre-intervention confidence in placing CVLs (1-10)	8.2
Average pre-intervention confidence in supervising CVLs (1-10)	8.3

Pre-intervention, four out of 21 participants used dynamic US guidance, which improved to 16 post-intervention (P < .001). In the pre-intervention group, two participants were unsuccessful at obtaining venous access, while all were successful post-intervention.

Pre-intervention, the average time from needle insertion to venous access was 36.8 seconds (this average excludes the two participants who were unsuccessful). Average time to venous access increased to 71.1 seconds post-intervention, however this increase was not statistically significant (P = .14).

The instructional video was rated as very helpful with an average rating of 8.8 (on a scale of 1-10, with 10 being the most helpful). Additionally, 95% (20 out of 21) of the participants were of the opinion that trainees would benefit from watching the video, and that competency testing in CVL placement should be required for attending critical care physicians. To a question asking how often competency testing should occur, with options including every year, every five years, every ten years, or only once, 43% of the participants (9 out of 21) favored the option of every year testing and 33% (7 out of 21) favored the option of every 5 years testing. Results are shown in Table 2.

Table 2.

Results.

	PRE- INTERVENTION	POST- INTERVENTION	P-VALUE
Primary Outcome
Needle tracking using dynamic US guidance (Number of participants)	4	16	P < .001
Secondary Outcomes
Successful venous access (Number of participants)	19	21
Average time from needle insertion to venous access	36.8	71.1	P = .14
Confidence in placing CVLs (1-10)	8.2	8.7	P = .03
Confidence in supervising CVLs (1-10)	8.3	8.7	P = .03
Helpfulness of the instructional video (1-10)		8.8
Percent of participants who believed trainees would benefit from watching the instructional video		95% (20/21)
Percent of participants who believed competency testing should be required		95% (20/21)
- One time		14% (3/21)
- Every year		43% (9/21)
- Every two years		5% (1/21)
- Every five years		33% (7/21)
- Should not be required		5% (1/21)

Discussion

This study has several interesting findings. Despite the reported pre-intervention high confidence in placement of CVLs, two participants were unable to successfully obtain venous access, and only four used dynamic US guidance. This adds to the findings of Barsuk et al that a wide variance in competency exists with this procedure among experienced attending physicians. Possible explanations for this include not having formal training in dynamic US guidance, a relative lack of opportunities to practice this procedure, and unfamiliarity with the models used.

There was a trend towards increased time required to obtain flash post-intervention, however this was not statistically significant. This is likely attributed to participants being asked to use a different method than what they were familiar with. Additionally, the “walk down” method may intrinsically take more time than other methods since it requires successive, short advances of the needle rather than one continuous advancement.

This study highlights an important issue regarding the current model of procedural competency in academic medical centers. In general, procedural skills, including those required for CVL placement, are assumed to be adequate upon completion of medical training. For critical care physicians, this includes years of formal hands-on training, required maintenance of procedure logs, and often additional simulation. Once training is complete, however, these skills are assumed to be maintained throughout one's career, regardless of how frequently they are used. Procedures in academic medical centers are often performed by trainees, meaning there are few opportunities for attending physicians to maintain these skills. This can lead to skill decay, which is the loss of ability to perform a task due to lack of continued use or practice. Perez et al showed that military surgeons’ skill in laparoscopic surgery declines when they are performing combat care rather than laparoscopic surgery during deployments. Additionally, emergency medicine residents’ skill in central line placement has been shown to weaken three months after CVL training. The results of our study provide an opportunity to both assess and reduce skill decay among attending critical care physicians in the performance of CVLs.

In our sample, baseline competency in the skill of dynamic US guidance for CVL placement was poor, but significant improvements were observed with a short instructional video intervention. We believe that this represents an opportunity to improve CVL procedural competency among critical care physicians. Institutions could establish scheduled evaluations for physicians using this process, and for those deemed not competent, a self-paced learning program could be followed using video instruction as well as hands-on simulation practice. This would help ensure that procedural competency is maintained throughout careers, despite the otherwise limited opportunities.

To our knowledge, there is no standardized competency testing in CVLs for attending physicians at most academic medical centers. A concern that may arise in enacting competency testing for CVL placement is the burden that it would place on attending critical care physicians, causing resistance if instituted. However, among the 21 attending critical care physicians in our study, all but one were in favor of competency testing.

This study has several limitations. Our sample size was small at 21 participants, the majority of whom primarily work in medical intensive care units. Second, post-intervention attempts had the benefit of already practicing once on the model, bringing into question whether improvement was due to the instructional video or simply familiarity with the model. Third, this testing was performed on gelatin models rather than actual patients. Fourth, determination of dynamic US guidance is subjective. Lastly, dynamic US guidance is a single part of the CVL placement procedure, the remainder of which was not evaluated in this study. Additional studies are needed to confirm these findings, and to determine how frequently testing and training sessions are needed.

Conclusions

Pre-intervention dynamic US guided CVL placement was poor in this sample of attending critical care physicians but improved significantly with a self-directed instructional video intervention. This study suggests there is a role for procedural competency testing among attending critical care physicians, and that significant improvement is achievable with relatively minimal instruction.