Ready or not, here they come: acting interns' experience and perceived competency performing basic medical procedures.

OBJECTIVE: To assess acting interns' (AI's) experience with and perceived level of competency performing 6 basic medical procedures.
DESIGN: Fourth-year medical students at the University of Cincinnati (UCCOM) are required to complete 2 AI rotations in Internal Medicine. All AIs in 2003-2004 (n = 150) and 2004-2005 (n = 151) were asked to complete a survey about whether during each of their rotations they had performed and felt competent performing the following procedures: phlebotomy, intravenous (IV) catheter insertion, arterial blood gas (ABG), nasogastric (NG) tube insertion, lumbar puncture (LP), and Foley catheter insertion.
RESULTS: Four hundred sixty-seven of 601 possible surveys (across both years and both rotations) were completed (78% response rate). During both rotations, relatively few students performed the procedures, ranging from 9% for Foley catheter insertion (24/208) to 50% for both ABG and NG tube insertion (130/259). The two procedures most often performed were ABG (range 46-50%) and NG tube insertion (range 42-50%). Feelings of competency varied from 12% (LP) to 82% (Foley catheter). Except for LP, if students performed a procedure at least once, they reported feeling more competent (range 85% for ABG to 96% for Foley catheter insertion). Among the students who performed LP during a rotation, many still did not feel competent performing LPs: 23 (74%) in rotation 1 and 20 (40%) in rotation 2.
CONCLUSION: Many fourth-year students at UCCOM do not perform basic procedures during their acting internship rotations. Procedural performance correlates with feelings of competency. Lumbar puncture competency may be too ambitious a goal for medical students.

BACKGROUND

The role of medical students in caring for hospitalized patients has changed considerably in the last two decades. The trend in the inpatient setting has been to minimize scut work, including many of the common bedside procedures (e.g., phlebotomy and intravenous (IV) catheter insertion) previously felt to be crucial to a medical trainee’s education. Medical students have come to expect that being able to perform common bedside procedures is no longer expected or required for their residency training because many of these procedures are now performed by ancillary hospital personnel. At the same time, students are spending a greater portion of their training in the outpatient setting where they conduct and perform fewer bedside procedures.1

As recently as the last decade, residents and medical students were the final step in the process of obtaining IV access in difficult-to-stick patients. Today, at our institution, trainees rarely attempt placement of peripheral IVs on patients with poor venous access, often subjecting patients to painful invasive procedures with higher complication rates. Indeed, residents rarely perform many bedside procedures and thus lack competency in both performing them and teaching medical students how to perform them. Thus, students may not become proficient performing these procedures before beginning their own residency programs. At our institution, the natural training ground for some basic procedures has been lost and the apprenticeship system of “see one, do one, teach one” has largely atrophied.

In 1996, the American Association of Medical Colleges embarked upon the Medical Schools Objectives Project (MSOP). The goal for the first phase of the project was to develop a consensus within the medical education community on the attributes that medical students should possess at the time of graduation and to set forth learning objectives for the medical school curriculum that would allow students to learn these attributes.2 Among other recommendations, the MSOP working group concluded that students must be able “to skillfully perform those diagnostic procedures warranted by their patients’ conditions and for which they have been trained,” for example, routine technical procedures including phlebotomy, insertion of an IV catheter, arterial puncture, thoracentesis, lumbar puncture (LP), insertion of a nasogastric (NG) tube, insertion of a Foley catheter, and suturing lacerations.

Results from three survey studies showed general agreement between Internal Medicine clerkship directors and associate deans in U.S. and Canadian medical schools that graduating medical students should be proficient performing basic technical skills.3–5 In a recent survey of associate deans, most agreed that graduating medical students should be proficient to perform basic technical skills and a majority responded that their students were proficient in venipuncture, IV placement, suturing lacerations, Foley catheter placement, and arterial puncture. However, most schools in the survey do not rigorously teach or evaluate the procedures outlined in the MSOP report and only 55% of schools required students to keep logs of the procedures they performed.4

In 2001, the Subinternship Task Force of the Clerkship Directors in Internal Medicine (CDIM) recommended a curriculum for the Internal Medicine subinternship that included 17 inpatient clinical scenarios and 4 integrative skills.6 One of the integrative skills was the ability to perform 6 basic medical procedures including phlebotomy, IV catheter insertion, arterial blood gas (ABG), NG tube insertion, LP, and Foley catheter insertion.

OBJECTIVES

Using the CDIM task force’s recommendations for the acting internship, we sought to answer the following questions regarding our acting intern (AI) experience: (1) To what degree do our fourth-year students report performing the 6 CDIM recommended basic medical procedures during their acting internship rotations? (2) Is there a difference in perceived competency between those who do and those who do not perform the procedures during the AI?

DESIGN

The acting internship in Internal Medicine at the University of Cincinnati (UCCOM) is uniquely positioned to study students’ experiences because all fourth-year medical students must complete 2 month-long AI Internal Medicine rotations during their fourth year (months are not usually contiguous). Students have supervised patient care responsibility for 3–5 patients at a time and are on call an average of every fourth night. The students rotate at 5 different hospitals, all of which have 24-hour ancillary services for all of the procedures studied except LP and placement of a NG tube. The AIs write all orders (with cosignature) on their own patients, including orders directing ancillary personnel to perform many procedures. Therefore, students have firsthand knowledge of the procedures being performed on their patients and an opportunity to perform the procedures themselves. However, the students do not receive specific encouragement to perform any procedures during the acting internship.

In the academic years 2003–2004 and 2004–2005, at the end of each of their 2 separate month-long Internal Medicine AI rotations, all students were asked to voluntarily complete a survey regarding their performance of and perceived degree of competence performing the 6 basic procedures described above. Specifically, the students were asked 2 questions about each procedure:

Did you perform this procedure during this AI rotation?

Do you feel competent performing this procedure without supervision?

Anonymous data were collected after each rotation using the online evaluation system that is used for all course, clerkship, and faculty evaluations. Data were downloaded into SPSS statistical software (SPSS, Inc., Headquarters, 233 S. Wacker Drive, 11th floor, Chicago, Illinois 60606) where descriptive analyses were conducted. Given the similarity of the students’ experiences during the acting internship in Internal Medicine from year to year, we combined the data from academic years 2003–2004 and 2004–2005. Also, because the anonymous student responses could not be linked from rotation 1 to rotation 2, we analyzed data for each rotation separately. Specifically, for each rotation we looked at the frequency with which students said they performed the procedures and their perceived level of competency performing them without supervision. We also examined and tested (using Fisher’s exact test) the relationship between these 2 variables.

RESULTS

Although students were not required to complete the course evaluation as part of the AI rotation, an excellent response rate was obtained. The anonymous survey was completed by 208 out of 300 students in rotation 1 (69%) and by 259 out of 301 students in rotation 2 (86%), for a combined overall response rate of 78%.

Frequencies Across All Responses

Performance

Of the 6 procedures recommended by the CDIM, the vast majority of students reported not performing 4 of them during their AI rotation (Table 1): phlebotomy, IV insertion, LP, and Foley catheter insertion. The 2 exceptions were ABG and NG tube insertion. However, whereas a statistically significantly higher percentage of students reported performing an ABG and NG tube insertion when compared to the other 4 procedures (P < .01), the percentage still was not a majority: 95 (46%) students performed an ABG in rotation 1 and 130 (50%) in rotation 2; 87 (42%) students performed an NG tube insertion in rotation 1 and 130 (50%) in rotation 2. The distribution of responses across all procedures was consistent for both rotations 1 and 2.

Table 1

Distribution of Performance and Perceived Level of Competency with Basic Medical Procedures by Rotation

	Did perform		Do feel competent performing without supervision
	Rotation 1 (n = 208; range 204–208*)	Rotation 2 (n = 259; range 256–259*)	Rotation 1 (n = 208; range 204–208)	Rotation 2 (n = 259; range 256–259)
	% (n)	% (n)	% (n)	% (n)
Phlebotomy	26 (54)	21 (55)	76 (158)	76 (198)
IV insertion	19 (40)	15 (38)	53 (109)	56 (142)
ABG	46 (95)	50 (130)	63 (130)	75 (194)
NG tube insertion	42 (87)	50 (130)	71 (146)	81 (208)
Lumbar puncture	15 (31)	19 (50)	12 (25)	25 (64)
Foley catheter insertion	13 (26)	9 (24)	82 (169)	82 (211)

*Rotations 1 and 2 involve the same learners. All students complete 2 months of AI in Internal Medicine. Small discrepancies in the “n” are because of missing data. In some cases, students did not fill out all questions on every survey.

Competence

In general, students’ assessment of their level of competence performing the 6 procedures without supervision was less variable than their self-reported performance (Table 1). Across both rotations, for 5 of the 6 procedures (phlebotomy, IV insertion, ABG, NG tube insertion, and Foley catheter insertion) a majority said that they felt competent performing them without supervision by the end of both months (ranging between 53% for IV insertion to 82% for Foley catheter insertion). The exception was LP. Compared to the other procedures, statistically significantly fewer students, 25 out of 208 (12%) in rotation 1 and 64 out of 259 (25%) in rotation 2, reported feeling competent performing a LP without supervision (P < .01).

Cross Tabulations

The data were then analyzed using Fisher’s exact test to examine the association between performance of a procedure and perceived level of competence performing the procedure (as shown in each 2×2 cell of Table 2). For phlebotomy, IV insertion, ABG, NG tube insertion, and Foley catheter insertion (rotation 1 only for Foley), students who performed the procedure were statistically significantly more likely to perceive themselves as competent to do so without supervision (P < .01 for all except Foley catheter in rotation 1 where P < .05 and is nonsignificant for rotation 2). The LP data, however, show a different pattern, particularly for rotation 1. Of the 31 students who performed a LP during the first rotation, 23 (74%) still did not feel competent performing it without supervision (P < .01). The data for LP in rotation 2 is similar to the other procedures. That is, the students who performed the procedure compared to those who did not were statistically significantly more likely to perceive themselves as competent to do so without supervision (P < .05). However, among those who did perform LP in rotation 2, the difference between those who did feel competent (n = 30, 60%) compared to those who did not feel competent (n = 20, 40%) is much smaller than for the other procedures.

Table 2

Relationship between Whether Students Performed the Procedure during Their AI Rotation and Whether They Did or Did Not Perceive Themselves to Be Competent Performing It without Supervision

Rotation	Performance	Perceived feelings of competency
		Phlebotomy		IV insertion		ABG		NG tube		Lumbar puncture		Foley catheter†
		Do	Do not	Do	Do not	Do	Do not	Do	Do not	Do	Do not	Do	Do not
		% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)	% (n)
1	Did	89 (48)*	11 (6)*	90 (36)*	10 (4)*	85 (81)*	15 (14)*	91 (79)*	9 (8)*	26 (8)*	74 (23)*	96 (25)†	4 (1)†
	Did not	71 (110)*	29 (44)*	44 (73)*	56 (93)*	43 (49)*	57 (64)*	56 (67)*	44 (52)*	10 (17)*	90 (156)*	80 (144)†	20 (36)†
2	Did	94 (52)*	6 (3)*	89 (34)*	11 (4)*	88 (115)*	12 (15)*	91 (119)*	9 (11)*	60 (30)†	40 (20)†	92 (22)	8 (2)
	Did not	72 (146)*	28 (58)*	49 (108)*	51 (110)*	62 (79)*	38 (49)*	69 (89)*	31 (39)*	16 (34)†	84 (175)†	81 (189)	19 (45)

*P < .01 (using Fischer’s exact test)

†P < .05

CONCLUSIONS

The AI in Internal Medicine should provide fourth-year students a valuable opportunity to hone procedural skills. However, 50% or fewer students performed any given procedure on the patients under their care. Fincher and Lewis7 made a similar discovery in 1994 when they surveyed students at the end of their third year. Students’ self-reports indicated that they performed procedures on inpatients infrequently during the third year, although a precourse on the procedures was provided at the beginning of the year. Ten years later, our study of fourth-year students suggests that the same challenges exist in educating students on basic procedures. At our institution, the AI in Internal Medicine is the primary rotation where senior students have the opportunity to perform bedside procedures because the majority of the fourth year is comprised of electives where students have much less direct patient care responsibility. Therefore, it is likely that many of our students enter internship lacking many of these basic procedural skills.

Interestingly, despite the fact that phlebotomy and IV insertion are the most common of these 6 procedures performed on patients in the inpatient setting, far more students performed an ABG and placed an NG tube. Medical residents at our institutions more regularly perform ABG and NG tube insertions, compared to phlebotomy and IV insertion, which are most often performed by ancillary staff. Therefore, it is likely that the procedural experience of our students is influenced at least in part by the experience of the residents with whom they work.

Except for LP, students who performed a procedure at least once were significantly more likely to report feeling competent to perform the procedure. Studies of third-year students and residents have also revealed that self-assessments of competency correlate with frequency of procedure performance.7,8 Consistent with our findings, a study by Hicks et al.8 revealed that Internal Medicine residents reported needing 6–10 LP experiences to reach a “comfort threshold,” defined as the number of procedures at which two thirds of the house staff reported being comfortable or very comfortable performing the procedure. Therefore, LP competency is likely too ambitious a goal for medical students and its inclusion by the CDIM task force should be reconsidered.

Our study had several important limitations. The study cohort consisted exclusively of fourth-year medical students from one institution. To our knowledge, the UCCOM is the only institution that requires 2 months of acting internship in Internal Medicine; thus, some of the results may not be generalizable. In addition, our survey depends upon students’ recall of whether they had performed a given procedure rather than procedure logs. Our study would have been strengthened by data on the number of procedures performed by the students. Finally, students’ self-assessment of competence is different than actual competence. An objective measure of competence would be ideal but was not addressed in this study.