Experience of a tutor centric model for sonography training of emergency department registrars in an Australian urban emergency department 2009-2012.

Purpose: To assess the impact of a regular sonographer proctored training program for emergency medicine trainees in the use of Emergency Department bedside ultrasound
Methods: Emergency Department (ED) Registrars in the Swan District Hospital ED were provided with proctored instruction in bedside ultrasound in performance of extended focused assessment sonography in trauma (eFAST) and abdominal aortic aneurysm (AAA) detection. Training was conducted by credentialed sonographers for individual trainees in a 1:1 or 1:2 setting for 1 hour on a weekly basis. Registrars who trained in the Department between Jan 2009 to Dec 2012 were invited to participate in a survey conducted between June-August 2013 designed to assess the impact of training on their confidence and use of bedside sonography.
Results: Registrars reported increased perception of their skill level in AAA and eFAST post-training. High levels of confidence in their ability to adjust machine settings for image optimisation, recognition of free fluid in the abdomen and ability to recognise an AAA were also reported. The participants who completed at least 10 hours of training and at least 20 scans showed significantly greater improvement in their perception of skill and confidence levels than those with less time. Registrars reported training was of significant benefit, improving their confidence in obtaining good quality images and their understanding of the equipment, which contributed to them obtaining accreditation. Benefits were ongoing and 50% of participants reported using ultrasound in clinical practice at least 3 times per week and a further 30% at least weekly after leaving ED.
Conclusion: Proctored training in the clinical context for ED registrars resulted in improvement in skills, confidence and willingness to maintain skills through practice in the clinical context over the long-term.

Introduction

Sonography has been shown to be a useful modality in emergency departments to assist in the management and diagnosis of a number of conditions. – The use of ultrasound by emergency department (ED) clinicians has increased as the benefits have become more widely accepted and it is now commonplace for ultrasound to be part of the work‐up in ED to improve both the accuracy and speed of diagnosis. –

Current medical graduates have limited or no exposure to the use of sonography in their undergraduate medical training. Therefore training the current emergency medicine workforce requires postgraduate training in the clinical context. In the context of a busy ED, this presents a number of challenges.

The availability of ultrasound machines in Australasian emergency departments has increased in recent years, with a recent survey reporting 94% of emergency departments in Australia have ultrasound available. Despite this widespread availability there is a recognised mismatch in training with the same survey reporting that only 60% of the Australian emergency departments had a credentialing process in place. Barriers to training included turnover of staff, insufficient access to trained supervisors and the perceived length of training required for skill proficiency. –

In a previous survey of emergency clinicians working in a trauma centre environment, the completion of an ultrasound course did not significantly increase doctors' perceived sonography skill. Emergency Doctors who had proctored training in sonography, however, were reported to have improved their skills and maintained this improvement for at least 6 months.

Interestingly, despite previous reports suggesting no prior training was required to be able to identify AAA, the same study reported that confidence in the interpretation of images was lacking in those without training. Therefore the benefits of training may not only be in the increased ability but also to maintain confidence and promote use of sonography in the longer term. , –

Despite the widespread availability of sonography, limited training and often limited confidence means it is under‐utilised. Therefore it is important to find ways to better engage clinicians in postgraduate training such that the benefits of the use of sonography can be more widely adopted.

This report includes emergency medicine registrars' post‐training perceptions regarding the impact of proctored bedside training on clinical and technical skills required to proficiently utilise emergency sonography. It was hypothesised that proctored training would significantly improve confidence and skills in the use of sonography for AAA and eFAST. It was also hypothesised that there would be a positive association between duration of contextualised training and improvement in confidence and perception of skill.

Methods

Ethics approval

This project was approved by the University of Notre Dame Australia Human Research Ethics Committee and was conducted in accordance with the relevant NHMRC statement and guidelines.

Study setting

The Swan District Hospital Emergency Department is an outer urban emergency department with a mixed adult and paediatric patient population and average annual census of 41005 during the period 2009–2012 (increasing from 35820 in 2009 to 46812 in 2012).

Sonography Training Program

All registrars in the ED were provided access to proctored instruction in bedside sonography in performance of eFAST and abdominal aortic aneurysm detection. Training was conducted by credentialed sonographers (Diploma in Medical Ultrasound) for individual trainees in 1:1 or 1:2 settings for 1 hour periods on a weekly basis.

The training was conducted on adult patients within the ED setting. The ultrasound tutor provided real‐time supervision of the trainee to include image acquisition and interpretation. Trainees were instructed in the imaging of the abdominal aorta from the level of the superior mesenteric artery to the bifurcation of the abdominal aorta in transverse and longitudinal planes. eFAST scanning instruction included standard four quadrant abdominal and pelvic views and lung windows.

Participant survey

Perceptions of skill and confidence were obtained via an online anonymous survey which was conducted from June 2013 to August 2013. It contained both closed questions requiring ordinal scale (1–10) responses and open questions requiring free text responses. Registrars who had undertaken training in the Department between Jan 2009 to Dec 2012 were invited to participate in the study. The survey was administered by an email invitation sent to all registrars who had undergone training in the relevant period. The email contained a link to the online survey (See Supporting Information). Consent was assumed via participation in the survey. Follow‐up emails requesting completion of the survey were sent to all participants two additional times.

Statistical Analysis

Responses were exported from Survey Monkey into SPSS IMB statistics v 22(2013) for further analysis. To determine the impact of training correlations between the ratings of perceived skill level and confidence before and after training, nonparametric correlations (Kendall's tau_b) for ordinal responses were performed. In addition differences between mean rating for before and after training for skills and confidence ratings were calculated. Independent two tailed t tests were used to determine if there was a difference in confidence ratings for those who had more practice. Low practise level was defined as completing 20 or less scans. High level of practice was defined as completing between 20 and 100 scans during the training period.

Free text responses were exported to a word file and the frequency of responses falling into thematically identified categories were tallied and proportioned.

Results

Participants

The response rate was 55% (n = 39/71). Approximately half of the respondents were female (51%). Fifty six percent of the cohort were between 31 and 35 years of age (range: 26–50 years). Forty four percent were between 3 and 5 years post‐graduation and a further 33% were between 7 and 9 years post‐graduation. Eighteen percent were employed as senior registrars and the remainder as registrars. There were 6 respondents who had graduated more than 10 years prior to completing the survey. Seventy five percent had never undertaken formal sonography training prior to the study period and 77% undertook the proctored training over a 6 month period. Nine respondents were involved in the training for over 12 months. Thirty eight percent were involved in more than 10 training sessions and the remainder between 4 and 9 sessions.

Perceived skill increased after training

Training increased the perceived skill for both recognition of AAA (× = 1.28, SD = 1.7 prior and × = 5.54, SD = 2.25 post training) and ability to perform eFAST (× = 1.26 SD = 1.79 prior and × = 5.59, SD = 2.26 post‐training, (Figure 1a).

Figure 1

Perceived level of skill recognising an abdominal aortic aneurysm (AAA) and eFAST before and after sonography training.

A: Perceived skill is increased in both AAA and EFAST scans after training.

B: Those students who performed < 20 scans had lower increases in perceived skill than those who performed > 20 scans for both AAA and EFAST.

Those who reported completing a low number of AAA scans (0–20) had a modest increase in perceived skill post‐training (x = 3.93, SD = 2.03) whereas those who had completed a high (21–100) number of scans reported a significantly larger improvement in perceived skill post‐training (x = 5.55, SD = 1.01, t (38) = 2.29, p = 0.03). The results were similar for the eFAST scans. Those who only completed a low number of eFAST scans (0–20) had a smaller increase in perceived skill (x = 3.71, SD = 2.20) compared to those who completed 21–100 scans (x = 5.70, SD = 1.1, t (38) = 3.04, p < 0.00 (Figure 1b).

Perceived Confidence after training

Those who reported completing a low number of AAA scans (0–20) had a modest increase in confidence post‐training (x = 5.03, SD = 2.16) whereas those who had completed a high (21 – 100) number of scans reported a significantly larger increase in confidence post‐training (x = 7.22, SD = 1.71 (t (38) = 2.78, p = 0.01)). The results were similar for the eFAST scans. A low number of scans completed (0–20) resulted in a smaller increase in confidence (x = 5, SD = 2.22) compared to those reporting a higher number of practice scans (21–100, (x = 6.92, SD = 1.66)) (t (38) = 2.75, p = 0.01 (Figure 2)).

Figure 2

Increase in perceived confidence is higher in those completing more scans for both AAA and eFAST.

The increase in perceived confidence was highest in those performing the greater number of scans for both AAA and eFAST (A).

Mean perceived confidence levels post‐training in improving the image through machine adjustment (x = 6.1, SD = 2.15), ability to recognise free fluid (x = 6.47, SD = 2.2,) and ability to recognise an AAA (x = 6.41, SD = 2.46,) were all high. After training the ability to manage patients utilising sonography was also rated highly (x = 5.46, SD = 2.83 (Figure 3))

Figure 3

Perceived confidence in all trainees post‐training is high. Perceived confidence to improve sonography images through machine adjustment, recognise free fluid and recognise an AAA were high in the cohort that underwent sonography training (B).

These results were also reflected in the number of training sessions attended, with those attending 10 or more training sessions reporting the largest increases in confidence for AAA (x = 5.54, SD = 1.1) and eFAST (x = 5.64, SD =1.0). It should be noted that those attending the greater number of sessions were predominantly those who also conducted the greater numbers of scans. Total duration of training was not correlated with confidence in recognition of AAA (Tb = 0.116, p = 0.40), eFAST (Tb = 0.116, p = 0.40) obtaining a better image (Tb = 0.06, p = 0.66) or recognising free fluid (Tb = 0.13, p = 0.36). The relationship between the number of scans performed and the specific impact of feedback on changes in skill acquisition and confidence in using the skill was not addressed in this study.

Analysis of the free responses showed that > 50% of participants reported using sonography at least three times per week and a further 20% at least weekly (Figure 4a). In clinical practice sonography was used predominantly for Focused Assessment with Sonography in Trauma (FAST), Vascular Access and identification of Abdominal Aortic Aneurysm (AAA, (Figure 4b)).

Figure 4

Use of Sonography post‐training. > 50% of participants reported using sonography at least 3 times per week (A). In clinical practice sonography was used predominantly for Focused Assessment with Sonography in Trauma (FAST), Vascular Access and identification of Abdominal Aortic Aneurysm (B).

The majority of registrars reported the training was of significant benefit in that it improved their confidence in obtaining good quality images, improved their technical skills and confidence in using the equipment and contributed to them obtaining accreditation.

Discussion

This study showed that effective training in the use of sonography in the workplace environment was achievable. The training resulted in an increase in the registrars' perceived technical proficiency and importantly improved confidence in the use of sonography for AAA, eFAST and assisted vascular access. These training benefits led to demonstrated use of sonography in the clinical employment context after training was completed. This study also demonstrated that the lasting benefits of training and the clinical use of sonography post‐training were both associated with attendance at more than 10 sessions and when more than 20 scans were performed.

It is known that practice is critical for the acquisition of any complex skill including the diagnostic use of sonography. For the novice, practice together with specific feedback is important. Acquired knowledge is used more quickly and efficiently and this enables faster performance with less demand on mental resources. With practice, low level tasks such as setting up the machine and using the probe become second nature, allowing better concentration and focus for higher level tasks such as interpretation of the image. , Skilled performance in a new task therefore depends on contextualised practice and feedback of the component processes. Whilst this may appear effortful at first sufficient contextualised practise enables a skilled user to generalise their skill to a range of contexts. In this way they can maintain and enhance their skill set via increasing practice in a number of different contexts. These points are clearly illustrated in this context, with the increased training leading to continued use of sonography not only for eFAST and AAA identification but also in a range of other contexts.

Also important was the relevance of the bedside training context and the impact of personalised feedback and support provided by the trainers. Immediate and specific feedback is critical for the acquisition of any skill, and other studies have also shown the importance of the use of trainers. – It is not surprising therefore that this personalised training approach had positive outcomes. The fact that the trainers were skilled sonographers and not part of the registrars' supervisory chain may have also facilitated their willingness to attempt new tasks and enhanced their learning experience.

The availability of skilled tutors in sonography is currently limited in emergency departments with small numbers of faculty. Sonographers who have experience in teaching ultrasound can readily be employed to train clinicians. This is a realistic option when the numbers of clinicians with suitable skills is lacking. It is the skill set of the trainer that is the critical feature. In a study by Costantino, the amount of training conducted by emergency physicians was not correlated to an improved test score in a written sonography examination, whereas, as in this study, other factors such as the number of scans completed and time spent using the machine were found to be important.

The use of a tutor sonographer means the teaching can be arranged independently and is not influenced by the activity of the department at the time of teaching. It may also provide a less pressured environment that permits registrars to experiment with new skills. In this environment specific feedback may be more effective.

Speelman and Kirsner highlight the importance of context and practice in moving through the stages of knowledge, mastery and fluency in acquiring complex skills. In addition they suggest that the process of skill acquisition modifies the user's mental representations. That is, the act of acquiring a new skill set can also change the way novel complex problems are perceived. Implicit in this progression is the observation that a novice differs significantly from an expert in the way they undertake a task. This underpins the need to identify effective training programs for those at different stages of their progression in medicine. Here, an effective training program in sonography for Registrars has been described, with the potential to enhance use of sonography in a wider context.

Limitations

This study demonstrated perceived improvements after delivery of sonography training. The key limitations are the indirect measures of skill change. Implicit in the observed increase in the use of the skill was an assumption of improvement in that skill. In addition changes in confidence ratings may or may not reflect an actual change in skill. It is also possible that the respondent (55% of those contacted) population was skewed towards those who felt they benefited from the training. Now that the feasibility of bedside teaching has been established future studies are needed to assess the comparative efficacy of workshop and bedside teaching strategies.

Conclusions

The proctored training program was effective in promoting confidence, skill and future use of sonography in practice. Those participants that completed more than 10 hours of training and over 20 scans showed much greater improvement in their confidence and perceived skill level. Therefore future programs adopting this model should aim to ensure participants achieve this level of training. Employment of sonographers as instructors provides an accessible training solution. Increasing the amount of practice was important in leading to greater changes in confidence and skill. Future programs using proctored training with substantial practice will be helpful in bridging the gap between sonography availability and its use.

Author contributions

GS was involved in the training program, ethics submission, development of the survey, participant recruitment and manuscript review. MF was involved in survey development, analysis and drafting of the manuscript. KH led survey development and analysis and interpretation of data, in addition to manuscript preparation and review.

Competing interests

GS is involved in point of care ultrasound education both at his workplace and also for Ultrasound Village, a provider of courses in clinician performed ultrasound. KH and MF have no competing interests.

Supporting Information

Ultrasound Teaching Survey

Age □ 21–25 □ 26–30 □ 31–35 □ 36–40 □ 41–45

Sex □ M □ F

Number of years post graduate at time of Swan term? (circle one)

2 3 4 5 6 7 8 9 >10

Have you undertaken formal training in ultrasound before exposure at Swan Districts Hospital

YES NO

During which period did you undertake the US training whilst at Swan Districts Hosptial

2009 □ Jan – June □ July – Dec

2010 □ Jan – June □ July – Dec

2011 □ Jan – June □ July – Dec

2012 □ Jan – June □ July – Dec

For how many months were you exposed to the training program?

□ 6m □ 12 m □ 18 m □ >18 month

What was your level of employment when undertaking the training ?

□ registrar □ senior registrar

List two (2) outcomes that you expected from the training program.

Did you miss any training sessions? □ YES □ NO

If yes, what was the main reason for your absence?

What is your current status in emergency medicine?:

□ Still working in emergency medicine □ Now working in another field

Which one of more of the following training programs have you participated in?

□ accredited course(s)

□ attained CCPU

□ working toward DDU

□ Undertaken a Fellowship

Current employment level at time of this survey

□ Registrar senior registrar consultant □ Left ED training

During your attachment how many eFAST did you perform?

□ 0–20 □ 21–50 □ 50–100 □ >100

During your attachment how many abdominal aortic scans do you perform

0–20 21–50 50–100 >100

What was your perceived level of skill: (0 = none, 10 = complete)

AAA PRIOR to training 0 1 2 3 4 5 6 7 8 9 10

AAA AFTER training 0 1 2 3 4 5 6 7 8 9 10

EFAST PRIOR to training 0 1 2 3 4 5 6 7 8 9 10

EFAST AFTER training 0 1 2 3 4 5 6 7 8 9 10

What is your perceived level of confidence: (0 = none, 10 = complete)

to obtain a better image through machine adjustment 0 1 2 3 4 5 6 7 8 9 10

in your ability to recognise free fluid 0 1 2 3 4 5 6 7 8 9 10

to recognise an abdominal aortic aneurysm 0 1 2 3 4 5 6 7 8 9 10

Has the US training impacted on your ability to manage patients: (0 = No effect 10 = significant benefit) 0 1 2 3 4 5 6 7 8 9 10

What was the most important skill you acquired from the training program?

Identify one other benefit of participating in the training program?

How could your training experience have been improved ?

What are you currently using ultrasound for in your clinical practice

How often are you using ultrasound in your clinical practice?