Mock Paging and Consult Curriculum to Prepare Fourth-Year Medical Students for Medical Internship.

Introduction: Internship preparation should include curricula to hone key skills such as acute medical management and communication with consulting and interprofessional providers.
Methods: To enhance these skills, we developed an interprofessional mock paging and consult curriculum incorporating direct observation and peer, faculty, and nursing feedback for fourth-year medical students entering medical internships. Our brief mock paging and consult curriculum was designed as part of a larger 2-week internship preparation course. Our curriculum was delivered in two 2-hour sessions by physician and nurse educators. Sessions were conducted in small groups, offering the opportunity for direct observation and feedback from faculty, nurse educators, and peers. Our curriculum was expanded from a pilot for 10-15 students to 60 students after 2 years of a successful pilot.
Results: Mock paging and consult sessions were highly rated by medical students and resulted in significantly enhanced self-assessment of preparedness in key intern skills such as returning pages, interprofessional communication, calling a consult, and managing acute issues for cross-cover patients. Discussion: We have demonstrated the effectiveness of a brief, interprofessional mock paging and consult curriculum incorporating faculty, nurse educator, and peer feedback. The tenets of our curriculum can be widely adopted for other learner groups.

Educational Objectives

By the end of this session, learners will be able to:

Demonstrate an approach to responding to registered nurse pages that includes asking appropriate questions to determine the situation and level of urgency, giving emergent orders as needed, relaying a plan of care, and responding in a professional manner.

Demonstrate a clear consult question and succinct, pertinent summary of a patient case to a consultant on the phone.

Identify (a) if a cross-cover issue is routine, urgent, or emergent after a 1- to 3-minute phone conversation with the nurse and (b) if the patient needs to be seen or not.

Provide and receive constructive feedback to and from peers and facilitators regarding communication skills and medical knowledge on common paging topics.

Introduction

Residency program directors have expressed concern that medical students are not prepared for internship.[1,2] Commonly cited concerns by residency program directors are lack of organizational skills, underdeveloped professionalism, and subpar communication skills.[2] To address residency readiness, internship preparation courses (IPCs) have proliferated.

On the first day of internship, many physicians-in-training are entrusted with the responsibility of answering pages from nursing staff and other health professionals. Effective paging behavior unites each area of program director concern: organization, professionalism, and strong communication skills. To address this well-recognized need, some medical schools have incorporated mock paging curricula into IPCs. These curricula have demonstrated high levels of learner satisfaction,[3,4] improved confidence in handling pages,[3] and improved clinical decision-making[5] and communication skills.[6] In addition to handling pages, requesting specialist consultation is a key skill to develop early in clinical training. Previous efforts to teach how to request a consult from the emergency department effectively have been successful, incorporating both didactic and simulated components along with direct observation and feedback by faculty.[7,8]

The importance of developing effective paging and consult etiquette among students and interns is clear from these previously published data. At our institution, a needs assessment was conducted among internal medicine faculty and residents. Interacting with consultants and handling pages were ranked among the most important skills for an intern. To address these needs, we developed and implemented a brief, interdisciplinary mock paging and consult curriculum for fourth-year medical students entering medical internships embedded within a 2-week IPC at our institution, the University of Wisconsin School of Medicine and Public Health (UWSMPH). The content of the curriculum was informed by medical residents, faculty, and nurse educators.

While several successful mock paging curricula have been published, including those intended for surgical,[9] obstetrics and gynecology,[3,10] and pediatric[11] interns, we are not aware of a robust mock paging curriculum for students entering medical internships. Although peer observation and feedback in a mock paging curriculum have been described,[11] we also incorporated feedback from nurse facilitators to offer the perspective of the discipline often interacting with interns in the context of pages. We have designed the curriculum to be conducted from start to finish within two 2-hour sessions, which is different than some conducted across the duration of a course, requiring faculty and staff resources over an extended period of time, often at night.[3,5,6,9] We integrated mock paging and mock consult scenarios into each session, which is novel.

Success of this interdisciplinary and peer educator curriculum demonstrates the feasibility of a brief, efficient approach to building skills necessary for effective paging and consultative behaviors.

Methods

Curriculum Development

Our mock paging and consult curriculum was developed as a component of a 2-week IPC for fourth-year medical students entering medical internships at the UWSMPH. The content of our mock paging and consult sessions was designed to address essential intern skills identified in an internal needs assessment. This needs assessment was completed in the form of a survey sent to internal medicine residents and faculty at UWSMPH, with a 60% response rate (61 out of 102). We asked respondents to rank intern skills for inclusion in the IPC by importance (1 = unimportant, 5 = very important). Respondents ranked the following highly: interacting with consultants and interprofessional team members (4.06), approach to an unstable patient (4.50), and returning pages to/communicating with an RN (4.42). The present curriculum aims to prepare students for these essential intern tasks.

We created the precise content of the mock paging and consult sessions in an interdisciplinary manner, with contributions from senior medical residents, hospital medicine nurse practitioners, and hospital medicine faculty from UWSMPH. We applied the pedagogical theories of deliberate practice[12] and peer feedback[13] to construction of these sessions. Cases reflected the broader themes taught in the IPC and served as an opportunity to practice and apply the key knowledge and skills emphasized in other aspects of the IPC. The content themes for our mock paging curriculum are reflected in Table 1. Our mock consult curriculum emphasized communicating the urgency of consults, defining expectations for consultants, and receiving recommendations professionally. Please see Appendix A, the facilitator guide, for the content of the mock pages and mock consults.

Table 1.

Content Themes of Mock Paging and Consult Curriculum

Theme	Special Knowledge or Skill
Acute symptoms or change in vital signs	New delirium
	Uncontrolled pain
	Hypotension
Lab abnormalities	Hyponatremia overcorrection
	Hyperkalemia
	New, acute anemia
Routine intern tasks	Verify placement of tubes and lines
	Electrolyte repletion
	Laxative order
Communication and professionalism	Request to update nurse
	Request to update family
	Handling pages to the incorrect provider
	Clarifying nursing questions and needs

The IPC at UWSMPH underwent expansion from an elective serving 10–15 students in 2015 and 2016 to a graduation requirement for all fourth-year medical students in 2017. While the initial 2 years of the pilot curriculum divided mock paging and mock consults into separate sessions, the 2017 curriculum integrated these simulated scenarios, although two sessions were still conducted. This integration was based on student feedback indicating a desire for greater similarity to an intern's workflow.

Target Learners

Our mock paging and consult curriculum was conducted within an elective fourth-year IPC for students entering medicine internships from UWSMPH. In 2015 and 2016, this was an elective opportunity serving 10–15 students. In 2017, our curriculum expanded to serve the nearly 60 students entering medical internships, including, but not limited to, categorical internal medicine and preliminary internships in neurology and radiology. The content of our curriculum required basic knowledge in assessment and treatment of common medical situations on an inpatient general medical ward and was most appropriate for students with some clinical experience. While our curriculum was part of an IPC, it could easily be delivered in another educational venue for learners of similar clinical aptitude.

Phone with speakerphone capabilities.

Projector for slides with pages.

Remote room with phone for floor nurse/consultant.

Mock sign-out sheet.

Facilitator manual for faculty facilitator and nurse educators.

Whiteboard (optional, for debriefing).

Pager (optional, for realism of receiving pages if they can be sent easily).

Personnel

All physician facilitators and nurse educators were provided with instructions on goals for student learning in the session, options for challenging students in real time, and appropriate prompts or cues to direct student behavior, among other key considerations in teaching students paging and consult etiquette. Physician facilitator and nurse educator instruction manuals are provided in Appendix A.

Physician facilitator: Facilitators should have relevant clinical experience in the field from which content is being derived. In our curriculum, this was inpatient internal medicine.

Nurse: In our curriculum, nurse practitioners from the Division of Hospital Medicine role-played floor nurses.

Consultant: In our curriculum, nurse practitioners from the Division of Hospital Medicine role-played consultants.

Students: We conducted these sessions with eight to 12 students during each allotted 2-hour session.

Session Flow

The sessions were each 2 hours in duration and took place in a conference room with eight to 12 students and a faculty facilitator. A brief orientation, including learning objectives and intended session structure, was presented. Key components of effective paging, communication, and consult etiquette were reviewed; this content that can be seen in Appendix C. Students were provided with a mock sign-out sheet to reference (Appendix B).

The first half of the session focused on returning pages from nursing colleagues. Scenarios began with one student receiving and returning a page from a floor nurse, a role played via phone by hospital medicine nurse practitioners stationed remotely. One learner at a time responded to a single paging scenario as the cross-cover intern, while the remaining students observed the ensuing phone conversation. The students rotated the role of the cross-cover intern, with each completing one or two paging scenarios, depending on group size, over the 2-hour session. For ease of group learning, the content of these pages was projected for all learners to see. Students were directed to return the page no matter the apparent acuity; in debriefing, faculty facilitators invited dialogue regarding the appropriate prioritization of the particular paging scenario in workflow. The conversation was observed by the faculty facilitator and colearners, with instructions to student observers to note positive and constructive feedback for their peers. Paging scenarios were immediately debriefed with the cross-cover intern; then, the observing students were invited to name what their peer did well in the exchange. This was followed by conversation about what was challenging in the scenario and considerations for similar future cases. Examples of content from pages used in our session can be viewed in Appendices C and D. Students returned pages via telephone equipped with speakerphone capabilities located in the conference room.

The second half of the session integrated calling consults effectively with mock paging cases. At the midway point, there was a brief reminder of the tenets of calling consults effectively, including how to construct a clinical question and how to convey the urgency of a requested consult. Students were prompted regarding a consult scenario when the faculty facilitator displayed a slide with details of the case and instructions about which team was to be consulted. Then, students used the conference telephone to call a specialty provider, a role played by the same group of hospital medicine nurse practitioners stationed remotely as in the first half of the session. Nurse practitioner educators were provided with instruction on how to include additional challenges in order to help students practice professionalism and patient advocacy. Facilitator guides are located in Appendix A.

After each scenario, the faculty facilitator, nurse, and colearners had the opportunity to give feedback. First, the student in the intern role was asked for self-reflection on the patient scenario and on his or her own skill in communicating with the other health care provider. Second, observing students were asked to provide their peer with specific positive and constructive feedback. Next, the faculty facilitator and nurse practitioner role-playing the floor nurse and consultant provided positive feedback and specific next steps for the learner. Then, the entire group of students, with guidance from faculty educators, discussed key learning points from each scenario. Depending on the level of discourse, facilitators challenged students with more complex scenarios or questions, which are included in the materials for each mock paging and consult case.

Curriculum Evaluation

In 2015 and 2016, students completed postsession evaluation forms immediately following the mock paging sessions. In response to “This session enhanced my knowledge and/or skills to better prepare me for internship,” students rated agreement on a 5-point scale, with 5 indicating the greatest degree of agreement. In response to “Overall, how would you rate this session?” students used a 5-point scale where 5 was the most favorable. This evaluation form (Appendix E) was a standard template used for all sessions in the 2-week IPC curriculum.

After 2 years of a successful pilot, the IPC and our mock paging and consult curriculum expanded from an elective to a required experience for fourth-year medical students at UWSMPH. With the expansion, our assessment practices changed significantly. A new course-wide postsession evaluation was implemented in 2017. Only one statement directly addressing the mock paging and consult curriculum was included, with students asked to rank on a 7-point scale their agreement with “Mock paging/consults was effective,” with a score of 1 indicating strong agreement.

In all years, we asked students for comments on the most effective aspects of the curriculum and suggestions for improving it in the future.

Learner Evaluation

In the 2 years of the pilot curriculum, students completed a self-assessment of preparedness to address topics or skills taught in the IPC preceding and upon completion of the course. The evaluative statements relevant to the mock paging and consult curriculum, an excerpt from the course-wide self-preparedness survey, are included in Appendix F. Students were asked to rate their preparedness on a 5-point scale, with 5 reflecting the greatest degree of preparedness. For the purposes of the mock paging curriculum, the following five skills were examined:

“Manage common cross-cover issues?”

“Identify urgent or emergent patient issues?”

“Address over phone if patient needs to be seen for acute issue?”

“Develop and relay plan of care to RN when returning a page?”

“Communicate effectively with interprofessional team?”

With the new assessment implemented in 2017, all fourth-year UWSMPH students completed a common self-preparedness survey at the conclusion of the IPC. One question was related to our mock paging and consult curriculum: “Overall, how prepared do you feel to communicate with your medical team?” One example provided in this question was calling consults.

To determine whether our curriculum led to statistically significant improvement in learner self-assessment of preparedness, we conducted two-tailed t tests comparing means of pre- and postcourse scores using Microsoft Excel.

Results

Sessions were highly rated by students, with an average rating of 5.00 on a 5-point scale for the first session (n = 23) and 4.83 for the second (n = 24). Students felt sessions enhanced their preparation for internship, with an average rating of 5.00 for the first session (n = 23) and 4.88 for the second (n = 24) in response to “This session enhanced my knowledge and/or skills to better prepare me for internship.” In 2017, in response to “Mock paging/consults was effective,” average ratings were 1.76 on a 7-point scale, with 1 indicating they strongly agreed with the statement. Table 2 details these results.

Table 2.

Student Evaluation of Mock Paging and Consult Sessions

	Average Response
Response Item	2015/2016 Session 1a,b	2015/2016 Session 2a,c	2017d
This session enhanced my knowledge and/or skills to better prepare me for internship.	5.00	4.88
Overall, how would you rate this session?	5.00	4.83
Mock paging/consults was effective.			1.76

Five-point scale (5 = most favorable, 1 = least favorable).

n = 23.

n = 24.

Seven-point scale (1 = strongly agree, 7 = strongly disagree), n = 52.

When asked what was effective about the session, students praised the authenticity of the page and consult content and the ability to be put in the hot seat while receiving multisource feedback in a safe environment. Selected student comments are included here:

“It was helpful to practice replying to pages individually. Being put in the hot seat prior to residency relieves some of the anxiety about getting the first page as a resident.”

“Good one-on-one practice with instant feedback.”

“Being in the hot seat and doing the pages and consults is very valuable. Using the phone is also more realistic. I also liked the extra time spent on discussing the cases and what if scenarios for management.”

“Examples [sic] scenarios were very realistic and a great way to practice and prepare for intern year.”

“Experienced nurse on the phone was very realistic and helpful in simulating future experience.”

“Practice with small and safe group. Provided space for error.”

When asked what should be changed about the mock paging curriculum in the future, students expressed a desire for increased volume and difficulty of paging and consult scenarios, particularly during the second session. Students also identified the potential role that additional resources, such as having a history and physical or a virtual chart to review in real time, can play in approximating the intern experience more closely. Additionally, some students felt the content of the second mock paging and consult curriculum was redundant and recommended more complex cases.

Students' pre- and postcourse self-assessments of preparedness demonstrated statistically significant improvement in domains addressed by the mock paging curriculum, as shown in Table 3. The skills in which students felt more prepared after the mock paging sessions were the ability to “manage common cross-cover issues on other providers' patients,” “identify urgent or emergent patient issues,” “assess over the phone if a patient needs to be seen for an acute issue,” “develop and relay a plan of care when returning a page to an RN,” and “communicate effectively with interprofessional team.”

Table 3.

Student Pre- and Postcourse Self-Assessment of Preparedness to Perform Skills Taught in the Mock Paging and Consult Curriculum

Response Item	2015 and 2016 Self-Assessments Combineda		2017 Self-Assessmentsb
	Precourse(n = 24)	Postcourse(n = 24)	Precourse(n = 56)	Postcourse(n = 50)
Communicate effectively with interprofessional team members.	3.54	4.33c
Develop and relay a plan of care when returning a page to an RN.	3.00	4.50c
Assess over the phone if a patient needs to be seen for an acute issue.	2.27	4.33c
Identify urgent or emergent patient care issues.	2.67	4.42c
Manage common cross-cover issues on other providers' patients.	2.21	4.15c
Call a consult.	3.38	4.33c
Overall, how prepared do you feel to communicate with your medical team?			2.14	2.02

Five-point scale (1 = not prepared, 5 = very prepared).

Five-point scale (5 = not at all prepared, 1 = very prepared).

p ≤ .0001.

Discussion

Our brief, interprofessional mock paging and consult curriculum was well received by students; it also enhanced self-reports of preparedness in key intern skills associated with paging and consultative practices. Strengths of our curriculum include the authentic experience of receiving a page with little prior knowledge of the patient it concerns, the interprofessional aspect of involving nurse educators, the relatively brief and low-resource nature of the intervention, and the incorporation of direct observation and feedback by peers, faculty, and nurse educators.

Several successful mock paging curricula have been published.[3-6] All address key intern knowledge and skills and show a positive impact on learners. Many of these curricula are designed to be delivered over several weeks, requiring physician or nurse educators to page students in a random manner, often overnight.[3,5,6,10] This approach is certainly high fidelity, but it requires significant educator resources and may limit the opportunity for robust, well-timed, immediate feedback. By condensing a mock paging curriculum into 4 hours and incorporating peer observation and feedback, the educator resources required are limited, and students are able to provide additional valuable feedback. The success of our mock consult curriculum highlights again the efficacy of a low-resource intervention compared to other published curricula,[7,8] with the added benefit of peer feedback.

Based on 2 years of a successful pilot curriculum, UWSMPH expanded the IPC as a graduation requirement for all fourth-year students. Hence, in 2017, the learner group expanded from a previous maximum of 16 learners to almost 60 learners pursuing medical internships. This increase in student size presented new challenges. To maintain fidelity to the successful pilot, facilitators ran seven simultaneous sessions with eight to 10 students each. This significantly increased the number of physician and nurse educators required to execute the mock paging and mock consult curriculum, necessitating more explicit orientation materials. An additional change was to better integrate the mock pages from nurses and mock consult requests to more closely simulate the workflow of an intern. Additional advanced discussion questions were added to each scenario for small groups requesting additional challenge or nuance.

Our curriculum, while innovative, does have limitations. First, our method of curriculum evaluation relies largely on satisfaction and does not incorporate a means of assessing persistent learning, behavioral change, or outcomes. Furthermore, the adoption of our curriculum into the required fourth-year IPC in 2017 necessitated transition to a common course-wide evaluation, limiting our ability to delineate outcomes related singularly to our mock paging and consult curriculum. We believe the brief nature of our intervention is a strength, but it does not simulate the workflow of an intern or the important triage decisions an intern will be faced with when receiving multiple requests for contact in a short period of time. Although we developed the curriculum in an interdisciplinary manner based on a needs assessment conducted with internal medicine residents and faculty, expectations of interns may differ widely among institutions, and our content may not be applicable to all programs. Finally, to facilitate concurrent small-group sessions, many physician and nurse educators were required; recruitment of this number of educators may be challenging in some medical schools.

A next step is to use the 1-year postgraduate program director survey to determine if students who underwent the mock paging and consult curricula in the 2 pilot years have had a measurable advantage in their professionalism and communication skills. We are also exploring inclusion of other interprofessional learners and colleagues, including nursing students.

We have demonstrated the effectiveness of a brief, interprofessional mock paging and consult curriculum incorporating faculty, nurse, and peer feedback for fourth-year medical students entering medical internships. While our learner group is well defined, there is potential for broad adaptation. The structure of the curriculum could be used to deliver non–internal medicine content or content designed to challenge students at other stages of learning.

A. Facilitator Guide.pdf

B. Student Orientation Materials.pdf

C. Presentation Session 1.pptx

D. Presentation Session 2.pptx

E. Student Evaluation Form.pdf

F. Student Self-Assessment.pdf

All appendices are peer reviewed as integral parts of the Original Publication.