Practically prepared? Pre-intern student views following an education package.

BACKGROUND: Graduating medical students enter their internship with varied levels of practical experience in procedural skills. To address this problem, many medical schools have introduced intensive skill training courses immediately prior to graduation. This study examines the impact of a pre-intern (PrInt) education package, consisting of a short intensive course, followed by a one-month clinical attachment.
METHODS: In September 2014, all PrInt students (n = 53) at the Central Clinical School (Sydney, NSW, Australia) attended three days of intensive training. This included a didactic introduction, case-based scenarios, and interactive workshops. This was followed by four weeks of targeted, experiential learning during a clinical attachment (PrInt term). Immediately prior to training and following PrInt, all students were invited to complete a six-domain questionnaire containing 40 subscale closed questions to assess their knowledge, experience, and confidence in key practical skills essential for a successful internship.
RESULTS: A total of 41/53 (77%) students completed an identical questionnaire prior to PrInt, and 37/53 (70%) immediately following PrInt. Respondents reported statistically significant increases in their experience, ability, knowledge, and confidence in a number of domains. The key changes were the following: knowledge of pharmacy skills (mean improvement = 26.48, confidence interval 95% [CI 95%] = 17.29-35.66, p ≤ 0.0001) and management of procedural skills (mean = 24.46, CI 95% = 16.58-32.34, p ≤ 0.0001). Despite the positive overall increase in most domains, some subscale results remained low following the educational package; only 44% students had inserted a nasogastric tube; only 44% reported confidence in commencing patients on warfarin; and only 42% in managing a hospital emergency. Surprisingly, there was a slight decline both in confidence in communicating with members of the hospital team (10%) and in awareness of the causes of hypoglycemia (7%).
CONCLUSION: Final year students perceived substantial benefit from an educational package specifically aimed at improving their practical skills immediately prior to internship.

Background

Medical graduates’ readiness for internship entails an awareness of their abilities (and limitations) in basic practical skills in the clinical environment, as well as their confidence to safely commence internship.1 It is not surprising that this “readiness” is strongly linked to their previous “hands on experience” as students,1,2 which is unfortunately opportunistic and, therefore, highly varied.3 Consequently, new graduates commence their internship with vastly differing levels of both knowledge and experience in practical skills, required to safely undertake their new role in patient care.3 Intern knowledge gaps have been particularly identified in the management of the acutely ill patients, prescribing,4–6 and non-technical skills, including communication skills,7 team work, and task management.

To address this variability in readiness for internship, preparatory courses such as “Coda” accelerated learning8 “Boot Camp”,9 Student assistantships,10 and “Shadowing” are offered in many medical schools immediately prior to graduation. To ease the transition into internship, Sydney Medical School implemented a pre-internship term in 2000. This comprised activities such as allowing students to “shadow” an intern, perform procedural skills under supervision, gain experiential knowledge in non-technical skills such as team work, and to gain an awareness of hospital protocols crucial to patient safety.

While the PrInt term was seen as useful for internship, we felt there was room for further improvement in core procedural skills and designed a new educational package to cover obvious gaps. The aim of our study was to assess the impact of this new educational package on students’ self-reported ability, confidence, knowledge, and awareness of the key practical skills required for internship.

Methods

The Sydney Medical Program is a four-year graduate entry course with problem-based learning as the key teaching strategy. This study took place at the CCS Royal Prince Alfred Hospital, a teaching hospital for The University of Sydney, Sydney Medical School. In the last four weeks of the medical program, students enter their pre-internship term.

In September 2014, all PrInt students at CCS (n = 53) who had completed their final exams attended a three-day PrInt induction course. The course had been designed and piloted previously as a five-day course in 2011.11 The course described here is a three-day modification of the pilot study.

Education package

Didactic teaching

This three-day intensive teaching program included key situations that new interns were very likely to encounter,12 including core procedural skills, such as “managing” procedural skills, identifying N/G tube placement on X-ray, recognition of the acutely unwell patient, complex communication skills – presented as ward-based scenarios, interpreting common X-rays, suturing, and effective handover utilizing ISBAR (a mnemonic created to improve safety in the transfer of critical information).13

Small group teaching sessions

These were designed to provide students with individualized practice, with real-time feedback from experienced clinicians. Students were also given the opportunity to clarify concerns in performing practical skills and were provided with strategies for identifying and dealing with common clinical challenges. This style of learning encouraged reflection and integration of skills and supported their transition to real life as a junior doctor.14

Case-based teaching

We used case-based scenarios, known to be associated with errors/adverse events, such as test ordering, prescribing medications, recognition of the deteriorating patient, and specimen collection/labeling. Specific education was delivered by pharmacists in “pharmacy/prescribing skills”, and by emergency physicians in “emergency skills”. In addition, while on their subsequent clinical attachments, they had weekly bedside tutorials conducted by hospital clinicians.

Clinical attachment

This component of the package involved four weeks when the students were assigned to a medical or surgical team, where they “shadowed” the intern and became active members of the team. PrInt students are required to perform various mandatory practical skills under supervision, and their overall performance as part of the team must be deemed “satisfactory” by their PrInt supervisor before they can graduate.15

Instrument

To develop our instrument, focus groups were conducted at CCS with hospital clinicians. Following data analysis, we generated a novel (unvalidated) six-domain questionnaire. We were mindful of including key procedural skills that interns would be expected to perform. The scores ranged from 1 to 100, and we arbitrarily considered 5 points (i.e., 5%) to be the minimal relevant change. Each of the six domains consisted of additional subscales, giving a total of 40 questions answered by either “Yes” or “No”.

The domains are as follows:

Performance of core procedural skills

Knowledge and confidence in managing procedural skills

Clinical recognition scores (recognition of the acutely ill patient)

Pharmacy skills/prescribing skills

Communication skills

Organizational skills

Students anonymously completed two identical questionnaires – the first, immediately prior to the three-day induction course (in late September 2014), and the second, immediately following four weeks of PrInt (in October 2014). Questionnaires were distributed and collected by a third party not involved with the study.

Statistical analysis

The two questionnaires were compared using multiple regression analysis. We were not able to compare individual student results due to the anonymous nature of the research.

Ethical considerations and participant consent details

Ethics approval was granted by The University of Sydney Ethics committee. Permission to undertake this research has been granted by Human Ethics approval number 2013/211 at The University of Sydney, NSW, Australia. Participants received an accompanying statement informing them that lodgment of forms was considered to be their consent for future use of data.

Results

A total of 41/53 (77%) PrInt students, attached to CCS, completed identical questionnaires prior to their PrInt induction session and 37/53 (70%) at the completion of four weeks’ PrInt. Of the 41 preinduction respondents, 24/41 (59%) were male, 15/41 (37%) were female, and 2/41 (5%) did not report their gender. Respondents’ ages ranged from 23 to 34 years.

The mean scores and mean differences in each domain of the six closed questions concerning both experience and knowledge in key skills are presented in Table 1. Respondents reported significant improvements in knowledge of pharmacy skills and managing procedural skills. However, because of preexisting high baseline scores, there was little improvement with respect to other domains, including communication, organizational, and procedural skills.

Table 1

Report of the practical skills ability of PrInt students in order of magnitude of improvement following pre-internship

Practical skill	LS mean (95% CI)	LS mean difference (95% CI)	p-Value
Knowledge of pharmacy skills	80.17 (72.8, 87.54)	26.48 (17.29, 35.66)	<0.0001
Managing procedural skills	85.88 (79.44, 92.32)	24.46 (16.58, 32.34)	<0.0001
Clinical recognition skills	88.93 (82.36, 95.49)	11.92 (3.73, 20.1)	<0.0043
Communication skills	97.81 (90.15, 105.48)	11.59 (2.04, 21.15)	<0.0173
Organizational preparedness	91.07 (81.51, 100.64)	12.36 (0.44, 24.28)	<0.0420
Procedural skills performance	81.87 (75.71, 88.04)	6.55 (−1.13, 14.22)	<0.0947

Abbreviations: LS, least square; CI, confidence interval; PrInt, pre-intern.

Subscale results

The subscale results in the order of magnitude of improvement are demonstrated in Figures 1–6.

Figure 1

Subscale results of PrInt students’ self-reported knowledge and awareness in safe prescribing (pharmacy skills) in 2014 before the educational package and immediately after pre-internship. Note that students’ confidence in commencing patients on warfarin remains low.

Abbreviation: PrInt, pre-intern.

Figure 2

Subscale results of PrInt students’ self-reported confidence and awareness in their experience of management of procedural skills in 2014 before their educational package and following PrInt.

Abbreviations: N/G, nasogastric; PrInt, pre-intern.

Figure 3

Subscale results demonstrating PrInt students’ self-reported knowledge in clinical recognition skills prior to and following PrInt in 2014. There was reported decline in knowledge in the precipitating causes of hypoglycemia; recognition of diabetic ketoacidosis also declined by 5%, whereas knowledge of the management of risk factors leading to hyponatremia improved.

Abbreviation: PrInt, pre-intern.

Figure 4

Subscale results demonstrate the decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) and a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events.

Abbreviation: PrInt, pre-intern.

Figure 5

Subscale results demonstrating PrInt students’ self-reported awareness and confidence in organizational preparedness before the education intervention and following PrInt in 2014. There was a decreased awareness of 11% (from 99% to 88%) reported for knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) as did performing an effective verbal consult to a senior colleague by 10% (from 68% to 78%).

Abbreviation: PrInt, pre-intern.

Figure 6

Subscale results demonstrating PrInt students’ self-reported performance in a range of procedural skills in 2014 before the educational package and following PrInt. Low pre- and post-scores were recorded for their practical experience of the insertion of a nasogastric tube.

Abbreviations: ECG, electrocardiogram; PrInt, pre-intern; BSL, blood sugar level.

Pharmacy skills

The results of students’ perceived knowledge and awareness and knowledge of pharmacy skills are shown in Figure 1. There were statistically significant improvements in five areas: safety awareness in pharmacy skills in writing the prescriptions increased by 58% (from 34% to 92%), knowledge in the use of the medication management plan increased by 51% (from 10% to 61%), confidence in prescribing low molecular weight and standard weight heparin increased by 42% (from 22% to 64%), confidence in commencing patients on warfarin therapy increased by 22% (from 22% to 44%), and recognition and treatment of opiate-induced respiratory depression increased by 16% (from 78% to 94%)

Managing procedural skills

Respondents’ perceptions of their ability to manage situations related to procedural skills are shown in Figure 2. The most significant improvements in students’ confidence were as follows: managing common problems associated with indwelling catheters increased by 39% (from 39% to 78%), intravenous cannulas increased by 37% (from 49% to 86%), identification of correct placement of peripherally inserted central catheter-lines on X-ray increased by 32% (from 36% to 78%), and confidence in managing clinical emergencies on the ward increased by 32% (from 10% to 42%).

Clinical recognition skills (recognition of the acutely unwell patient)

Ten questions were asked to participants regarding their ability in the recognition, awareness, and management of the acutely ill patient. The results are shown in Figure 3. There were significant improvements in the following: students’ perceived confidence in recognizing respiratory acidosis increased by 17% (from 51% to 68%), confidence in managing the fluid status of a patient with hyponatremia increased by 12% (from 49% to 61%), confidence in the assessment and management of patients with oliguria increased by 13% (from 63% to 76%). There was a small decline in knowledge regarding the precipitating causes of hypoglycemia, which decreased by 7% (from 95% to 88%); awareness of the risks of ceasing oxygen for a patient with hypoxemia and hypercapnea also declined by 7% from (from 90 to 83%).

Communication skills

As seen in Figure 4, there was a decline in respondents’ reported confidence in working with the hospital team on ward rounds by 25% (from 98% to 73%) as well as a small decline by 5% (from 88% to 83%) in prioritizing patients to minimize the possibility of adverse events both of which had relatively high preexisting baseline scores.

Organizational preparedness

Three closed questions asked respondents to self-report their organizational preparedness. Figure 5 demonstrates a decreased awareness of 11% in respondents’ reported knowledge of writing in the patients’ notes, and students’ ability to perform an effective clinical handover increased by 12% (from 61% to 73%) and their ability to perform an effective verbal consultation with a senior colleague increased by 10% (from 68% to 78%).

Procedural skills

The results of seven closed questions related to students’ reported performance of procedural skills are shown in Figure 6. Respondents indicated very high baseline levels of performance in cannulation (98%) and venipuncture (98%), and there was no room for further improvement. However, there were significant increases to IDC insertion by 15% (from 78% to 83%), electrocardiogram performance by 13% (from 59% to 72%), and N/G tube insertion by 15% (from 29% to 44%).

Discussion

The results of this study demonstrate that PrInt students perceived a substantial benefit from a combined educational package and a month of experiential learning in their hospital attachment as PrInt students, in particular, improvements in pharmacy skills and management of procedural skills. However, despite the positive overall increase in most domains, two subscales remained low with only 44% of students reporting confidence in commencing patients on warfarin therapy, and while most respondents had attended an emergency on the ward, only 42% felt confident about handling emergencies. This is despite having had extensive experience in simulated cardiopulmonary resuscitation throughout their medical course and advanced life support skills training in the simulation laboratory during year three, and PrInt. This clearly indicates that respondents felt that they had not had enough practical experience.

While students reported increased performance of N/G tube insertion by 15% (from 39% to 44%), the score remained low, suggesting that the students had low exposure to the procedure in both year three and PrInt. The 15% increase (from 66% to 81%) in taking BSLs was attributed to “hands on” use of the BSL equipment in the PrInt education course.

Students reported a moderate level of confidence in their ability to recognize the acutely ill patient (clinical recognition skills). Interestingly, there was a slight decline in students’ reported ability to manage and treat diabetic patients, as well as in communicating with other members of the medical staff following PrInt (Figure 4). The likely explanation for this is that students had overestimated their knowledge in the aforementioned skills prior to engaging in further education and learning on the wards. This finding is in accordance with Kruger and Dunning who wrote that “those who are unskilled may over estimate their abilities as they are unaware of their own incompetence”.16

Our findings are similar to those of other recent researchers. Coombes et al in their questionnaire study on pharmacy skills found that medical graduates when commencing internship had little confidence in commencing patients’ warfarin, believing initiation of anticoagulants as being “too great a responsibility”.17 They found that students also felt less confident in the management of patients with diabetes.17 A study by Jones et al also supports our findings in that medical graduates rated themselves well prepared in venipuncture and working in a team but least prepared with N/G tube insertion.18 Wall et al also found that medical graduates are well prepared for working in a team, but in contrast to our findings, students reported to be less prepared in practical skills and pharmacy skills.19

Our students reported lack of confidence in managing an emergency is a well-known, worldwide concern. 20,3,21 Clearly, while simulation can shorten the learning curve, these programs do not replace real “hands on experience”.3

While it is necessary for new interns to be equipped with sufficient knowledge and ability to perform practical skills, safe internship is also dependent on non-technical skills, such as working environment and ward protocols. Previous cohort studies have found that graduate entry PBL medical courses along with “shadowing“ were associated with improvements in preparedness for internship.1 This was described by Wenger as “legitimate peripheral participation in a community of practice”,22 and this occurs when students work under supervision and integrate their clinical knowledge into practice while learning how the hospital community operates.23 Conversely, it has been reported that “shadowing” can be limiting as an observational role, leaving the student feeling demotivated and frustrated.24 Ideally, “shadowing” should be in the same hospital that they will be allocated to as interns (i.e., their new work environment).23 While PrInt placements are currently held at the “parent” hospital where they were students, unfortunately their intern placement is usually not in their “parent” hospital. To address this issue, all new interns in NSW (Australia) now attend a short orientation, which includes “shadowing” the outgoing intern for one week, prior to commencing their new independent clinical role as interns.

Limitations

This study has several limitations. The use of a control group would have defined if specific gains were due to the education package immediately before PrInt or experiential learning during the four weeks of PrInt.

We acknowledge that this is a relatively small, singlecenter cohort study. Moreover, the questions in this study pertain to self-reported knowledge and confidence, and in the absence of observational feedback, there are obvious concerns of its reliability and validity.25 Instructional courses may improve the learner’s perception of confidence, including their competence in performing the skill. However, while students reported that their knowledge had increased, it may not be an accurate indicator of their actual skills, knowledge, or performance ability.26 A further limitation to this study was its failure to find out how many times a procedure had been performed and whether any feedback had been given.

Further investigation

Results from our findings are important to patient safety, and we are currently conducting further research to determine whether our educational intervention has helped interns to safely care for patients. Medication safety is of particular interest as interns initiate medications, including those which are of high risk, such as warfarin,27 without the counter-signing by an experienced clinician. Students’ lack of confidence in the management of an emergency on the ward is of real concern, which suggests that extensive simulation training throughout their course is not sufficient to overcome this problem. The low frequency of N/G tube insertion is a further concern, which highlights the need for collaboration with nursing staff, who are more likely to perform this skill.

Recommendations

We recommend that a short, intensive transition course be implemented in the curriculum prior to students commencing their PrInt clinical attachment. In Sydney Medical School, the week before PrInt (so called “conference week”) is an ideal time to present the learning package, as students are not involved in other teaching, nor exams, and can shift their focus to their procedural skills, and patient safety issues, prior to transiting their first clinically independent role as interns.

Conclusion

Our results indicate that students do perceive positive benefits from an educational package specifically aimed at improving their practical skills immediately prior to internship. The need for intensive education prior to internship is also highlighted, to address the variation in students’ experiences during clinical rotations to enable students’ awareness of their abilities and limitations prior to internship.