An elective entrustable professional activity-based thematic final medical school year: an appreciative inquiry study among students, graduates, and supervisors.

PURPOSE: The transition from undergraduate to postgraduate training can be facilitated by offering electives that focus on increased patient care responsibilities. This transitional year model in the Netherlands has recently been expanded to offer packages of electives geared to specific residencies and was called "dedicated transitional year" (DTY). At University Medical Center Utrecht, an entrustable professional activity (EPA)-based multidisciplinary DTY in acute care (DTY-AC), rooted in self-determination theory (SDT), has been implemented. The current study aimed to understand strengths and challenges regarding the implementation of this specific DTY.
METHODS: An explorative qualitative study among students, graduates, and faculty was conducted using an appreciative inquiry methodological approach. We gathered first-hand accounts of experiences with the DTY-AC in focus groups and interviews with students and interviews with graduates, supervisors, and mentors. Transcripts were analyzed with a directed content analysis approach.
RESULTS: Participants found the DTY-AC to focus learning, offering coherence by clear learning objectives, aligned assessment, and teaching sessions and offering a congenial learning community. However, EPAs were not the focal point of workplace assessment and evaluation. Providing sufficient hands-on student engagement in actual acute care situations was another challenge.
CONCLUSION: The concept of the thematic DTY is embraced, and it seems to help in meeting the SDT needs. Enhancing delivery in the workplace by improving formal implementation with information and faculty development, expanding EPA-focused workplace assessment, and extending hands-on experience of students could further unlock the potential of this final medical school year design. Our lessons learned may help in the development and implementation of similar programs, other models of DTYs, and final-year redesigns.

Plain language summary

In the Netherlands, medical schools use packages of elective rotations in the final year to facilitate the transition to specialty training. These elective tracks are called “dedicated transitional years” (DTYs) and should improve graduates’ readiness to start working as a doctor. How these years produce impact on learning is unknown. This study explored experiences of students and faculty with the Utrecht DTY in acute care (DTY-AC). Both students and faculty found the DTY to focus learning by offering coherence and a congenial learning community. Full potential of the DTY-AC could be reached by enhancing delivery in the workplace by increasing visibility and expanding workplace assessment and hands-on experience. This study provides insight in strengths and challenges regarding the DTY that may help in the conception and implementation of final-year redesigns.

Introduction

Graduation from medical school is a significant step in the educational continuum from student to being a life-long learning professional.1,2 This transition is a well-known challenge. Students, graduates, and program directors have regularly identified gaps in medical graduates’ readiness to assume clinical responsibility in postgraduate training,3–8 and educators have advocated to optimize this readiness in better designs of the final year of medical school.4,9–11

In the Netherlands, undergraduate medical education consists of a 3-year mainly theoretical bachelor phase and a 3-year master phase, largely devoted to clinical rotations.12 Usually, master years 1 and 2 comprise compulsory rotations in several disciplines, whereas year 3 comprises elective rotations. To facilitate the transition to postgraduate training, schools have started to redesign the final year to create the so-called dedicated transitional year (DTY). The DTY is a final-year elective track meant to enable the development of early residency-level competence in order to give graduates a head start in postgraduate training.13 In a DTY, students grow toward the responsibilities of a resident, acquire early specialty-specific expertise, explore career options, and try to improve their chances on the job market.14 Dutch medical schools piloted and introduced local interpretations of this concept, but little is known about how and to what extent a DTY produces impact on learning.

In 2014, University Medical Center (UMC) Utrecht and the Antonius Hospital introduced a multidisciplinary DTY in acute care (DTY-AC) in a collaboration of the following five specialties: anesthesiology, cardiology, emergency medicine, intensive care medicine, and respiratory medicine. The theme of acute care was chosen as young doctors in many specialties have to face acute care challenges and because participating disciplines recognized the need for focused preparatory training in this area. The DTY-AC distinguishes itself from regular transitional years by being a coherent thematic year that aims for demonstrable competence. Coherence is provided by the selection of disciplines, monthly teaching sessions, personal mentoring, learning objectives framed as entrustable professional activities (EPAs), which transcend disciplines, and multimodal assessments of course objectives prior, during, and after the DTY (Table 1).15

Table 1

Key features of the Utrecht DTY-AC

Key features of the Utrecht DTY-AC
Thematic rather than narrowly specialty focused
Three clinical electives and one research elective, limited to anesthesiology, cardiology, emergency medicine, intensive care medicine, and respiratory medicine
Three transdisciplinary EPAs related to acute care EPA 1: Recognition and initial treatment of vitally threatened patients EPA 2: Evaluation of respiratory compromised patients EPA 3: Evaluation of circulatory compromised patients Performance target: (near) independent supervision
Workplace assessments linked to EPAsTailored observation forms in conjunction with existing general observation forms
Pretest and post-test of knowledge, know-how, skills, and (simulated) clinical performanceWritten examination, case-based discussions, objective structured clinical examination, and simulated management of critically ill patients
Monthly teaching sessionsInteractive small group teaching, peer supervision, integration, and deepening of knowledge
Senior residents acting as mentorsLongitudinal developmental support and buddy

Abbreviations: DTY-AC, dedicated transitional year in acute care; EPAs, entrustable professional activities.

Self-determination theory (SDT)16,17 is the conceptual framework that guided the design of DTY-AC. The educational features mentioned earlier, which are not offered in standard transitional years, aim to secure the following three basic necessities for motivation and growth described in SDT: relatedness, competence, and autonomy. SDT states that people’s natural tendency to develop is stimulated when the need for confidence and effectance (competence), the need for volition, ie, choosing what a person desires or finds useful (autonomy), and the need to belong to a group or community (relatedness) are satisfied.16,17

Teaching sessions intend to support students’ need to belong to a group (relatedness). Thematic focus aligned learning objectives and assessment intend to foster confidence and effectance (competence). Clinical responsibility in patient care, formative assessment, and mentoring are meant to support autonomy.

DTY-AC has been running since September 2015 and has been offered as a pilot program within the elective constraints of the new Utrecht undergraduate curriculum,18,19 preceding the full development and implementation of this curriculum reform.

This study aims to explore the experiences of students and faculty with DTY-AC and to investigate whether the intended curriculum matches these experiences. Understanding strengths of and challenges with the implementation of current design will help in the development and implementation of final-year redesigns.

Methods

Study design

To investigate the experiences with the DTY-AC, an explorative qualitative study of perceptions from students, graduates (ie, junior doctors), and faculty was conducted. The study was conducted within the constructivist research paradigm that considers social reality to be formed by subjective human experiences and social contexts.20,21

We took an appreciative inquiry (AI) methodological approach. AI identifies strengths and possibilities for change to unlock future potential, instead of diagnosing deficits in the process of interest.22

The Ethical Review Board of the Dutch Association for Medical Education (NVMO) granted ethical approval for this study (NERB file 823).

Setting

Every 6 weeks, two to six students enroll in this year-long program on a rotational basis. Students take three clinical rotations and a research elective in UMC Utrecht and affiliated regional teaching hospitals. At any time, 15–25 students are taking part in the DTY-AC simultaneously. Departments also host other final-year medical students not taking the DTY-AC.

Participants and sampling

At the time of data collection, 42 students were participating in or had graduated from the DTY-AC. We announced the study to students during a teaching session. Next, all students and graduates were approached individually by email providing information about the study. All graduates were asked permission to analyze their previously conducted and transcribed exit interviews.

Supervisors are clinical faculty responsible for rotations and evaluation of students. We aimed to study supervisors (n=41) from all participating specialties and from UMC Utrecht and regional hospitals anticipating heterogeneity in knowledge and involvement. Initial purposeful sampling included supervisors responsible for student rotations and evaluations. Further snowball sampling was used to contact other supervisors that had worked intensively with DTY students.

Mentors are residents who provide developmental support to DTY students. All 20 resident-mentors in our database were approached. All supervisors and mentors were approached individually by e-mail providing information about the study.

Data collection

Data collection took place between February and April 2017. Two focus groups and individual interview sessions were organized to explore experiences from current DTY-AC students. Experiences of graduates were explored in individual face-to-face or telephone semistructured interviews and by the analysis of previously conducted exit interviews. Supervisors and mentors were interviewed individually in a semistructured face-to-face or telephone conversation.

Two researchers (WRO and CMEV) developed and revised the initial interview guides based on DTY-AC documentation, conversations with the developers and coordinators of DTY-AC, and discussion with the other authors. AI guided the formulation of questions starting with what works well and going to the ideation of potential future improvements.22 The interview guides were organized topically addressing SDT needs, curricular components, and implementation. The interview guides were piloted for clarity and information yield, and no subsequent changes were made. Suggestion and prompting were prevented by formulating broad and open questions. Table 2 shows the topics and examples of questions. Interviews and focus groups were conducted in the UMC Utrecht by WRO and CMEV, who had prior experience in conducting interviews for research purposes.

Table 2

Interview topics with examples of questions for St and Su

Topic	Sampled question
Relatedness to DTY-AC	St: Do you identify yourself as a DTY-AC student?
	Su: To what degree do you feel connected to DTY-AC?
Competence	St: Are you able to build competence on junior doctor level in the domain of acute care?
	Su: What would you say regarding DTY students’ knowledge, skills, and attitudes within the domain of acute care?
Curriculum	St: What were your main reasons to choose DTY-AC?
	Su: Can you elaborate on the DTY-AC philosophy?
Assessment	St: How are you being evaluated in your rotations?
	Su: Which sources of information and tools do you use in evaluating your DTY students?
DTY-AC in the workplace	St: Do you feel engaged with the department or department’s work?
	Su: How do you facilitate DTY students in reaching their learning objectives?
EPAs	St: Can you explain the role of the EPAs in your workplace learning?
	Su: What is your opinion on the DTY EPAs? (Prompt: regarding utility, feasibility, and clarity)
Organization and logistics	St: What are the strengths of DTY-AC?
	Su: What would you like to see improved in DTY-AC?
Professional identity and future	St: What is occurring with regard to your orientation on the job market and choice for a specialty?
	Su: How do you view DTY students, as compared to non-DTY final-year students?

Abbreviations: DTY, dedicated transitional year; DTY-AC, DTY in acute care; EPAs, entrustable professional activities; St, students; Su, supervisors.

Data management and analysis

Audio recordings, transcripts, and analyses were stored digitally password protected. Interviews were audiotaped, subsequently analyzed, and summarized by two authors (WRO and CMEV). The summary was sent to the participants for member checking. The focus group sessions were audiotaped and conjunctly analyzed by two authors (WRO and CMEV). A directed content analysis approach23 was applied, using preidentified codes, stemming from both the SDT framework and AI terminology, and codes around themes that appeared to be important during analysis. This approach was chosen as this study focused on perceived strengths and views of future potential in relation to the design of the DTY-AC and assuming contextual influences on these perceptions.22,24 Content analysis was performed using the qualitative software package NVivo (QSR International, Daresbury, UK), using the topics of Table 2 as initial code tree.

Results

After written informed consent, first-hand accounts were gathered from 17 students, 17 graduates, and 21 faculty (Figure 1).

Figure 1

Overview of students, graduates, and faculty participating in the study.

Note: Supervisors were from all five specialties.

Abbreviation: UMC, University Medical Center.

Findings are presented by the following four broad themes found: the three SDT necessities for growth and motivation (relatedness, competence, and autonomy) and delivery. Perceptions from students and graduates are presented as students’.

Relatedness: genesis of a learning community

Students highly appreciated the evolving sense of belonging to a DTY-AC peer group. Teaching sessions were pivotal in the origin of a community.

Virtually all students felt connected to the DTY-AC and their peer group. Students described their DTY-AC peers as motivated, ambitious, hardworking, and like-minded students.

Specifically because we’re all keenly interested we go that extra mile and delve that bit deeper. [St9]

Thematic focus, mentors, and the pre- and post-tests reinforced cohesion, but above all, the teaching half-days fostered the origin of a learning community:

It’s especially the teaching half-days on which you reunite with your group, all focused on the same domain, and discuss things you’ve experienced; that’s of surplus value as compared to the normal transitional year. [St15]

The teaching sessions induced learning especially by discussing real-life experienced clinical cases. Students felt it broadened their view on acute care, enhanced knowledge retention, and promoted clinical reasoning. The exclusive, confidential setting of these sessions was deemed important for interactivity and safety to express oneself.

Specifically the small scale added value. We learn more in a small group and there’s more opportunity to share experiences. In larger settings it’s more like taking turns to contribute to the discussion. [St39]

Further value could be added by including simulation training and input from teachers from a wider array of backgrounds.

Competence: EPAs and assessment focus learning

Apart from the exclusive monthly teaching sessions, the main reasons for choosing DTY-AC were thematic focus on acute care, an opportunity to stand out among graduates and preparation for residency.

Students felt that the DTY-AC contributed to their competence in delivering acute care and preparedness to work as a junior doctor. They stated to have obtained integrated and deepened knowledge in a medical domain they had only been minimally exposed to prior. DTY-AC students considered themselves, in comparison to peers, more interested and more knowledgeable in acute care.

Supervisors suggested that DTY-AC students are proactive, more motivated, and more focused, ask more specific questions, and may have slightly more content knowledge than regular final-year students in the same rotation.

Most students used the three DTY-AC EPAs in clarifying the learning objectives for their rotations and made them a topic in the conversation with their supervisor at the start of their rotations. Students said that the EPAs focused their learning process during the rotations.

By the end of the year, I want to master those EPAs, and that’s what I’m working on. And, you know, without those objectives I would have just done the rotation and it would have been okay to semi-master them, but now I’m keen on fully mastering them. [St22]

Many supervisors put effort in enabling students to reach DTY-AC learning objectives by advising which learning activities to pursue or altering their placement:

Basically, the rotation looks the same, but based on the goals of the student, I try to facilitate the student to reach those objectives; for example by placing them in the Emergency Room, or not. [Su30]

Many supervisors found the EPAs to provide a good overview of the expected level of competence at graduation, and they appreciated the transdisciplinary nature of the EPAs:

Because of this specific setting of rotations that are connected and aligned within a theme, students are better prepared to work in all these specialties. [Su15]

The DTY-AC-specific preassessment of knowledge, know-how, skills, and performance in acute care was thought to provide valuable insight in their current level and in areas for improvement.

The added value of the entry test is to get to see where the focus is put on and in which areas your level is reasonable and in which you need to improve. [St6]

The pretest was a strong motivator to work on their acute care proficiency. Students would like to enhance opportunities to develop this ability, both in clinical and simulated settings. The post-test was seen similarly: a measure of current knowledge and skill and direction for personal growth.

In workplace assessment and rotation evaluations, however, EPAs did not play a central role, students and supervisors claimed. In workplace assessment, some students chose to use the DTY-AC-specific short observation forms which link to the EPAs, whereas most used the generic forms. Supervisors deemed the DTY-AC-specific observation forms to be clear and valuable and the generic ones too nonspecific.

We often complete such a standard form for a very specific encounter or aspect and most of those items are not applicable. [Su5]

Students suggested strengthening the role of EPAs in the workplace by requiring DTY-AC-specific observation forms. It would focus feedback on the performance regarding the learning objectives of the DTY-AC.

Supervisors evaluated students using the regular rating forms for elective clinical rotations. In addition to the observation forms, the global impressions supplied by colleagues were taken into account. Supervisors struggled to envision an improved workplace assessment practice within the DTY-AC. Many indicated the desire to incorporate more informal assessments and a developmentally oriented e-portfolio.

Students felt that, instead of global rotation evaluations, explicitly making the entrustment decision, the focus of assessment would reinforce the role of EPAs in the workplace.

I think you’d be more focused if you know that goal in advance. You have those EPAs and you want to get a pass mark on them, so you pay more attention to them. [St33]

Some students doubted whether entrustment decisions would lead to decrease in supervision, given that residents usually perform these acute care activities.

Autonomy: clinical responsibility, albeit limited in acute situations

Students had a focused clinical experience in the acute care domain and felt supported. Frontline experience, however, was difficult to obtain.

Most students indicated that the DTY-AC confirmed their preference for a specialty. Half of the students appreciated the transdisciplinary set-up of the DTY-AC allowing for a shift in preference within the spectrum of acute care.

Students were overall satisfied with the clinical responsibilities in their rotations. However, students faced limited possibilities to get hands-on clinical experience in the EPAs, although students deemed the expected performance level of these acute care professional activities realistic and appropriate. Some supervisors viewed the EPAs as ambitious and perhaps unfeasible for students in practice. Students advised making the EPAs better known to staff to improve supervision and involvement, which would extend possibilities to gain hands-on experience.

As described earlier, students indicated the formative nature of the pre- and post-tests to be motivating and to direct choices in their learning.

Students thought the mentor to be of added value, with examples of mentors providing advice about rotation choices and helping with applications for training posts.

You have someone to direct questions to, discuss things, and obtain advice when you’re in doubt; this is all very handy. [St4]

Students and mentors appreciated the room left for individual approaches to mentoring meetings. Most mentors explained to have chosen a casual, companion approach. A few students stated that the purpose and role of mentoring were unclear and wished more guidance on mentoring. Mentors felt that they did not need more instruction. Supervisors emphasized the merit of mentoring and mentors indicated that merit varies per student:

I think it’s very dependent on how things are going, which problems you encounter and how you’re supported on the ward. If that goes arduous, it’s great to have someone to unload with. I think one student has more needs there than another. [M4]

Delivery

Students took their rotations in a range of departments in many regional hospitals. The uptake or delivery of DTY-AC differed from location to location.

Familiarity of the training site with the curriculum of the DTY-AC varied per hospital and department, students said. Rotation supervisors tended to be well informed, but consultants and trainees less so. All supervisors were aware of the DTY-AC principle that students differentiate within acute care to gear for a residency in this domain of medicine. Most supervisors lacked information on the background, objectives, and curriculum of the DTY-AC. The timing of the information often did not match the need for information, long preceding the arrival of a first DTY-AC student.

Supervisors indicated not to feel a close link to DTY-AC but did feel connected to individual DTY-AC students. Supervisors tutored 3–20 students, but for some, it had not always been clear who was a DTY-AC student and who was not, given that other students could simultaneously do final-year rotations in these disciplines.

A supervisor suggested that regional hospital faculty could get more involved by organizing DTY-AC network events, for example. Being kept up to date with developments was felt to be motivating:

I envision that if, for example, data on student outcomes were known, these would be shared among supervisors. This would make the DTY-AC come more alive. [Su21]

Discussion

This study investigated successes and challenges associated with the current design and delivery of a dedicated final year of medical school and aimed to formulate options to unlock further potential. The main findings (Table 3) and steps for improvement are discussed below.

Table 3

Successes and challenges with design and delivery of DTY-AC

Strengths
DTY-AC focuses learning in a coherent program with clear learning objectives
Educational sessions contribute to coherence and focused learning and create a congenial learning community
The DTY-AC program attracts highly motivated and proactive students
Possibilities for change and improvement
Possibilities	Improvement strategies
Heterogeneity in involvement of supervisors	Just in time information Faculty development Supervisor network events
Position of EPAs	Expand workplace assessments Entrustment decision
Hands-on experience in acute care	Bed-side teaching Legitimate workplace roles Entrustment-based discussion Simulation

Abbreviations: DTY-AC, dedicated transitional year in acute care; EPAs, entrustable professional activities.

SDT16,17 underpinning the design of DTY-AC is reflected in the delivered curriculum.

Foremost, the DTY-AC program focuses students’ learning, as intended. It offers a coherent curriculum with learning objectives (EPAs) that transcend rotations and disciplines. Starting the year with a multimodal pretest makes expectations explicit and reinforces the focus of personal learning. These aspects reflect the SDT need for competence.

Second, the DTY-AC-exclusive teaching sessions contribute greatly to the impact of the program. They are perceived as a valuable and appealing asset of the DTY-AC. As intended, they provide coherence and focus for learning. Above all, students value to be part of a learning community, reflecting the SDT need for relatedness. The DTY-AC class may be seen as a community of practice,25 a group that shares common interests, tasks, and activities in which participatory learning occurs. Students feel part of a distinct and exclusive group of peers who are on the same wavelength. From a Social Identity Theory perspective, this “in-group” perception provides a highly accessible and fitting group identity.26 The close class creates a congenial climate for peer supervision, ie, learning from an exchange of experiences and reflections.

The third DTY-AC strength is that it succeeds in attracting highly motivated proactive final-year students. We believe that the DTY-AC offers a motivating and supportive environment for student growth, reflecting the SDT need for autonomy.

Despite the strengths of the DTY-AC, the quality of program delivery in the workplace could be improved. Supervisors appeared only marginally involved with and informed about the program. The fact that DTY-AC students do rotations in parallel to non-DTY-AC students decreases the visibility of DTY-AC. Provision of more and timely information could increase the familiarity of supervisors and departments with the intentions of the DTY-AC and convey a clear message on what is expected from supervisors and mentors. Support should be offered to increase rotation sites’ readiness to supervise DTY-AC students.27 Faculty development sessions providing tailored training on how supervisors can contribute to the learning of students by exploiting learning opportunities potentially improve the quality of delivery and may improve the involvement of supervisors. Such sessions would provide networking opportunities as well. We hypothesize that the hampering effect of limited workplace implementation has been partly offset by the proactive role of the motivated students. Students became informal change agents by drawing attention to the DTY-AC. Increasing uptake of DTY-AC curricular ideas into daily clinical work of the students will be the focal point of improvement strategy. With the advent of the new EPA-based Utrecht undergraduate curriculum,18,19 working with EPAs will be everyday practice in departments hosting clinical students and this will ease the full implementation of advanced EPAs in DTY-AC.

Another avenue for unlocking potential regarding the quality of delivery is reinforcing workplace assessment with DTY-AC-specific observation forms. Moreover, the EPAs could play a more central role in the evaluation of students. Mapping a developmental trajectory in an e-portfolio would raise the awareness of and growth toward the learning objectives and would be an essential condition for entrustment decisions. Also, the repertoire of workplace assessments could be expanded by adding multisource feedback28 and entrustment-based discussions.29

The theme of acute care with learning objectives described as professional activities for (junior) doctors in acute care settings is perceived as relevant. Students expressed the desire to have more hands-on experience in the DTY-AC EPAs. However, in practice, the care of instable or acutely deteriorating patients is in many settings a situation in which residents or more experienced doctors are at the helm. Possibly, restricted workplace affordances, because of norms of practice, seniority, and unfamiliarity of team members with the learner,30 preclude students from engagement in acute care. This has impact on the feasibility of the program’s goals and intentions. Supervisors could be encouraged to exploit acute care situations as learning opportunities. First of all, supervisors could debrief students observing the management of acute care,31 provide bedside teaching,32,33 and have case-based discussion to teach and evaluate the clinical reasoning and management of unobserved situations.29 Second, supervisors should provide legitimate workplace roles34 by allowing students to perform observable and assessable acute care tasks (eg, take a focused history of a dyspneic patient). Furthermore, simulation training and assessment could be emphasized in the DTY-AC curriculum to guarantee a certain level of exposure to critically ill patients.

This study on the delivery and adoption of a DTY has strengths and limitations. A strength is that we took several measures to enhance dependability, credibility, and confirmability.35 The data collection methods were developed by two authors in close collaboration with the other authors and were piloted. The two researchers collecting the data managed the data confidentially, analyzed the data in conjunction, and checked extracts of the interviews with participants. A limitation of this study is that it was an AI into the delivery and uptake of this program, rather than a curriculum evaluation or a study on the efficacy of the program (ie, demonstrable competence in acute care). The latter is part of a separate study. As such, data from this study only provide insight in the experienced delivery of the program. In hindsight, more focus could have been put on looking at implementation in the different workplaces, for example, from an implementation research perspective.24 Nevertheless, our lessons learned provide some important factors associated with success and challenges of a design for the final year of medical school and these may well be transferable to other settings (Table 4).

Table 4

Lessons learned regarding medical school’s final-year designs

Lessons learned
A thematic final year may selectively attract motivated students
Teaching sessions foster a learning community and the need of relatedness
EPAs as learning objectives and aligned assessment help in meeting the need of competence
Invest in a community of teachers as well to stimulate their motivation and relatedness
Invest in faculty development and timely information to stimulate teachers’ competence

Abbreviation: EPAs, entrustable professional activities.

Conclusion

We found the DTY-AC to provide a focus on an interdisciplinary domain of medicine, supported by learning objectives, aligned assessment, and tailored teaching sessions and by offering a solid learning community. Enhancing delivery of the program in the workplace by increasing the visibility of the program, elaborating on workplace assessment, and extending hands-on experience of students can further unlock the potential of this final medical school year design.