Design of an entrustable professional activity for adult extracorporeal membrane oxygenation.

BACKGROUND: Extracorporeal membrane oxygenation supports severe cardiac or pulmonary failure. There are currently no competency-based standards for extracorporeal membrane oxygenation training.
METHODS: Extracorporeal membrane oxygenation experts were interviewed using a structured interview. Responses were audio recorded, transcribed, and validated by respondents. Interviews were coded using grounded theory with a constant comparison method. Themes were developed and used to construct the entrustable professional activity, which was reviewed by the extracorporeal membrane oxygenation experts.
RESULTS: Nine experts were interviewed; all had experience with trainees. Interview themes identified include patient selection, circuit and medical management, multidisciplinary communication, problem-based learning and simulation, and entrustment decisions. Essential functions of the entrustable professional activity were patient selection, circuit management, cannula selection, responding to circuit emergencies/complications, anticoagulation management, weaning, and family/team communication.
CONCLUSIONS: Essential functions of an extracorporeal membrane oxygenation entrustable professional activity were defined using data from structured interviews. The resultant entrustable professional activity could be implemented by critical-care programs as a scaffolding for competency-based fellow training.

Introduction

Extracorporeal membrane oxygenation (ECMO) is a form of Extracorporeal Life Support providing cardiac or pulmonary support for patients with severe heart or lung failure. The patient can be on venovenous or venoarterial ECMO depending on the need for lung or heart rest, respectively. The patient is cannulated with a large-bore catheter in a central vein, and the blood passes through a pump with an artificial membrane lung and then returned to the patient. Since the landmark series in 1977 by Robert Bartlett [1] describing the use of ECMO, the indications for adult ECMO have dramatically expanded [[2], [3], [4], [5], [6]], and the number of ECMO cases has steadily increased, with over 10,000 cases performed internationally in 2018 [7]. Despite the expanding use of ECMO, there are no standards for ECMO credentialing or certification, with each institution determining its own unique approach [8]. Training at the critical-care fellow level in ECMO is even more nebulous, with no overarching schema to guide training programs. A recent survey [9] of critical care program directors found that, of responding directors, 50% agreed/strongly agreed that fellows should be competent in ECMO and 70% felt that ECMO will be important in training in the next 10 years. However, only a small minority of program directors felt that their graduating fellows were competent to manage ECMO patients.

Competency-based medical education has been slowly redefining best-practice graduate medical education over the past 40 years in the Canadian CanMeds project [10] and by the Accreditation for Graduate Medical Education competency milestones [11] project. Olle ten Cate in 2005 introduced the concept of entrustrable professional activity (EPA) [12], which is an observable measure of what a physician actually does, that is, a descriptor of work, on a daily basis [13,14]. It is a means to concretely assess how a trainee is performing through the lens of supervision, and the trainee is granted a level of entrustment using both formative feedback in addition to more structured summative evaluations [14]. As the learner progresses and demonstrates greater competency, he or she is granted more autonomy.

The aim of our study, therefore, was to create an ECMO EPA to guide critical-care fellow training. We elected to use an EPA to fill the ECMO education gap because of the significance of competency-based medical education and the overall shift of medical education to EPAs. Because of the complexity of ECMO and with the goal of making the EPA broadly applicable to multiple training programs, expert consensus through a structured interview process was deemed to be the most appropriate method for generating the data to create the EPA.

Materials and Methods

This study was approved by the Oregon Health & Science University IRB (#18831), and our methods were guided by the best-practice consolidated criteria for reporting qualitative research framework [15]. Interview questions were devised by author WCC and then pilot tested with 3 ECMO faculty at our institution not involved in study design. The interview questions were revised following each interview based on qualitative feedback from the faculty. Eight interview questions were initially created and subsequently revised to include a final of 7 interview questions, approved by authors KB, BZ, and DZ and listed in Table 1.

Table 1

Structured interview questions

Who is the target population for ECMO training?

What are the essential functions of an experienced physician who consults and cares for patients on ECMO, specifically: a. Consultation & initiation b. Cannulation c. Maintenance and ventilator management d. Circuit troubleshooting e. Weaning

How would you assess a trainee's ability to consult and care for patients on ECMO?

How would you determine the degree of supervision the trainee requires, and when would you feel confident the trainee could manage ECMO independently?

What are the difficulties or struggles that you faced while learning ECMO that you hope future trainees will not encounter?

How do you believe the ECMO skill set is best taught? (didactics, simulation, patient care)

What other suggestions or thoughts do you have on ECMO instruction and education?

Interview questions used to conduct the structured interview.

Author WCC, a male general surgery resident, conducted in-person interviews at the 29th annual Extracorporeal Life Support Organization (ELSO) Conference (Scottsdale, AZ, September 13–16, 2018). One phone interview was conducted following the meeting. The interviewees were selected by BZ and DZ based on their expertise in the field and their demonstrated experience with trainees. The interviewees were contacted by e-mail requesting their voluntary participation; WCC had no prior relationship with the interviewees. The interviews were conducted privately and audio recorded; no notes were taken. Each participant was asked all questions, but follow-up questions varied depending on participant responses. The audio responses were then transcribed by authors HH and WCC. The transcripts were returned to the interviewees for validation. Using a constant comparison method [16], the transcripts were coded by author WCC into themes generated de novo using NVivo 12 (QSR International, Melbourne, Australia) until thematic saturation. Similar codes were grouped together under larger umbrella themes (for example, within medical management, this broad theme encompasses minor themes including anticoagulation, pulmonary support, hemodynamic support, and sedation). From these themes, the EPA was written and revised by author WCC from the themes, and refined by authors KB, BZ, and DZ. The EPA was then sent to all interviewees for comment, and using a modified Delphi approach, the EPA was iteratively revised and subsequently sent to all interviewees until no further changes or comments were suggested. All 9 interviewees responded during the revision process, with a total of 4 rounds of revisions.

Results

A total of 9 individuals approached for the project agreed and completed the interview; 8 of the interviews were in-person at the ESLO meeting; 1 phone interview was conducted in September 2018 following the meeting. The interviewees were all physicians, whose specialties included the following: 3 surgery critical care, 3 pediatric critical care, 1 cardiothoracic surgeon, 1 pediatric cardiothoracic surgeon, and 1 pulmonary critical care. The interviewees had a median of 14 (IQR 7–20) years of ECMO experience, with 3 interviewees having more than 20 years of experience. All the interviewees had experience training residents and/or fellows in the use of ECMO. The interview durations ranged from 13 to 55 minutes, with a median interview duration of 21 minutes.

The interview themes included circuit management, medical management, entrustment decisions, problem-based learning and simulation, multidisciplinary communication, and patient selection. Exemplary quotes for each theme are highlighted in Table 2. Most respondents emphasized the importance of understanding the circuit and the unique ECMO physiology. Furthermore, respondents expressed that ECMO is truly an extension of critical care and that managing ECMO is synonymous with practicing critical care. Even among proceduralists, respondents did not feel that cannulation was a required competency. Interviewees expressed that there are different practice models of cannulation at different institutions. However, all respondents expressed that the trainee needs to understand cannulation configuration options, be able to discuss cannulation with those performing the procedure, and be prepared to handle cannula complications. Most respondents also expressed that when they are working with trainees, they are performing numerous ad hoc assessments and granting autonomy on a daily basis. In addition, respondents emphasized that problem-based learning, with “what if?” style questions, can be used to evaluate trainees' understanding and preparedness during these ad hoc assessments. In regard to multidisciplinary interactions and communication, interviewees expressed that this skill is essential for the ECMO patient. Interviewees also reflected that the outcome of ECMO patients hinges in part on appropriately selecting patients for the modality. In response to the question “What other suggestions or thoughts do you have on ECMO instruction and education?” respondents overwhelmingly highlighted the role of simulation. Most respondents reflected that, during their training, they learned ECMO through direct clinical experience with little formal ECMO education. Respondents expressed however that high-fidelity simulation can be time consuming and expensive with limited learner throughput and may not be available at all institutions.

Table 2

Interview themes and exemplary quotes

Theme	Number of references	Exemplary quotes
Circuit management	52	I think all physicians that care for somebody on ECMO should understand the circuit, know how the circuit works, how to troubleshoot the circuit, maybe even know how to put the circuit together, just as an experience....The circuit is part of the patient and vice versa.—Interviewee 3...understanding all of the complications how to avoid and rescue from those complications. So that would be things like you know recognizing cannulation position, mal-position and any kind of misadventures related to cannulation.—Interviewee 7
Medical management	36	...ongoing daily management. And that includes understanding anticoagulation management, understanding ventilator management, understanding hemodynamics understanding sedation, nutrition just like any other ICU patient.—Interviewee 2You know as well as rehabilitation and physical therapy which hopefully are beginning during the ECMO process.—Interviewee 7
Entrustment decisions	35	I think it sort of depends on circumstances…if it's a respiratory failure case where it's not as time essential um I would absolutely let the resident do it, see what their skill set is and determine maybe next time you know how much uh how much do I need to participate actively. Um and maybe I wouldn't even need to put gloves on but just kind of stand there and watch them do it.—Interviewee 1You actually have to be there and to uh observe them and I also think that uh I think that graduated independence is important uh but in the past I think we have been bad at structuring that so people tend to operate in two modes right I'm here and telling you what to do because you do it wrong all the time or I'm not here and therefore you have independence. Whereas I think as educators it's much harder now to really teach yourself to be present but really allow somebody independence within confines of safety.—Interviewee 8
Problem-based learning and simulation	28	...you're gonna have to rely on simulation because that's where you can do things to the circuit and make bells and whistles and all this stuff go off and see how people react.—Interviewee 3I don't really believe that the high-fidelity simulation is obligatory. It's I think for, for simply emergencies, for simply education, you can start with simply really simplified circuit, and then um just to explain how it works, and so on and so on. Then, for more advanced training, or for team training, the high-fidelity simulation is perfect.—Interviewee 4
Multidisciplinary communication	24	...the number one problem that uh you face is not specific to ECMO. It's multidisciplinary interactions, right? Surgeons wanna be in control, intensivists wanna be in control, cardiologists wanna be in control, nobody's in control there's a lot of you know and especially as a trainee right as a fellow they have to learn to you know they have to learn to navigate all those paths.—Interviewee 3...you're able to have real conversations with families and uh other members of the multidisciplinary team.—Interviewee 7
Patient selection	16	I would start with candidacy so you would need to have a little background on the literature um keep abreast with the literature in order to say yes this is a candidate or not a candidate.…—Interviewee 2...the first thing is you've got to know is um how to select cases appropriately. And that changes with the environment you practice also as we generate more knowledge.—Interviewee 5

Themes derived from the interviews, with number of total references for each theme, along with exemplary quotes.

The essential functions generated from the interview themes for ECMO care are summarized in Table 3, and the full EPA is provided as supplementary material.

Table 3

ECMO EPA essential functions

Understand the indications for venovenous (VV) and venoarterial (VA) and venovenoarterial ECMO

Select appropriate cannula size and cannulation configuration for both VV and VA ECMO

Manage common circuit emergencies and anticipate complications

Determine the adequacy of support and make appropriate circuit adjustments

Manage anticoagulation strategies for VV and VA ECMO

Direct a weaning trial and removal of support, understanding the candidacy, and timing for transition to a durable cardiac or respiratory replacement

Communicate with family and team members in a multidisciplinary manner

The 7 essential functions for ECMO care.

Discussion

Graduate medical education is undergoing a gradual and accelerating transformation to competency-based assessment, with the recognition that trainees need to exercise increasing levels of autonomy through their education trajectory to be prepared to appropriately care for patients at the completion of their programs. Entrustable professional activities are increasingly prevalent; they are being pilot-tested for undergraduate medical education [17,18] and are being used in a variety of graduate medical education programs [[19], [20], [21], [22], [23]].

As an educator, one must balance granting an appropriate level of trainee autonomy with patient safety concerns, ideally granting the learner just enough autonomy to permit a safe struggle in the learner's zone of proximal development [24]. Entrustable professional activities are ideally suited to satisfy the competing goals of patient safety and trainee education, as the EPA framework permits educators to make meaningful assessments on how the physician trainee is performing in concrete terms. The educator can then assuredly grant the appropriate level of autonomy for the trainee to care for future patients under those circumstances, knowing that patients will receive appropriate and safe care. Lastly, learners are able to internalize the expected behaviors of them and develop a mental model that serves as a roadmap to achieve the next level of entrustment.

There are unfortunately no specific best practices for obtaining information used in the design of an EPA, and different approaches exist including working groups (69%), literature review (26%), and interviews (15%) [25]. Following the EPA draft, the refinement process can take the form of a Delphi process, surveys, or an unstructured focus group. We elected to use the structured interview process due to the convenience of international ECMO experts gathering at the annual ELSO meeting while minimizing participant time involvement as compared to a working group.

Not every aspect of patient care is amenable to being assessed under the lens of an EPA. An EPA is best suited to a task that is a specific unit of physician work encompassing multiple competencies. The unit of work should be observable, measurable, and one in which an entrustment decision can be made [13,14]. Managing a patient on ECMO is an ideal example. The essential functions of our EPA include discrete elements of care along the ECMO patient timeline, ranging from appropriate patient selection to weaning and possibly transitioning to a durable heart or lung replacement. The essential functions are intentionally broad without significant granular detail while also encompassing several core competencies of ECMO care. Under the direction of the EPA, the trainee should be evaluated for each essential function with multiple forms of assessment, including direct observation, multisource feedback from nursing and other staff, written trainee reflection, completion of simulation exercises, problem-based learning discussions, standardized written assessment, and case review presentations. All of this material should be synthesized into a summative evaluation granting the trainee the appropriate level of autonomy.

Recognizing that ECMO is a highly specialized modality and that ECMO will not suit every trainee's personal interest, we hope that the ECMO EPA will serve as a noncore, “elective EPA” [14] to supplement standard critical care training. A multisociety working group [21] representing key stakeholders in pulmonary/critical care medicine outlined overarching program EPAs and curricular milestones, and our EPA should be viewed as complementary. Prior to widespread implementation, however, the EPA requires validation within a clinical context, a step that we intend to pursue as a future direction, ideally at multiple centers.

Strengths of our study include following best practices for qualitative research while designing the study. We attempted to increase transparency by following best practices for interview design [26,27] and qualitative research [15] according to the Enhancing Quality and Transparency of Health Research network [28]. The refinement of the EPA was performed using an e-mail–based modified Delphi, with each interviewee blinded to the revisions of others to minimize the potential risk of one individual's opinion holding undue influence over the group. To ensure that our EPA was valid and could be used by a variety of training programs, we adhered to the best practices and included all essential elements for an EPA, as outlined by ten Cate [[12], [13], [14]]. We also recruited highly experienced ECMO experts, all of whom had experience in instructing trainees in ECMO, to participate in the interview process. We feel that the ECMO EPA is broadly applicable to most adult critical care training programs due to the fact that interviewees represented both medical and surgical subspecialties. The focus of the project was limited to adult ECMO, but the competencies could easily be extended to pediatric critical care training programs; indeed, there were pediatric intensivists and a pediatric cardiothoracic surgeon that participated in the EPA creation.

There are limitations to our study. There is inherent bias to performing an interview, and there may have been an unexplored topic that was not broached because of the nature of the interview questions. We attempted to minimize this limitation by pilot testing and refining our questions with ECMO faculty not involved in the research project prior to the formal interviews. Furthermore, through the EPA revision process, interviewees had the opportunity to improve upon and add other information to the drafts; edits did occur that prompted additional rounds of review. Lastly, the interviews were conducted and themes were coded by 1 researcher, possibly introducing bias. However, the EPA revision process with the ECMO experts helps to minimize this bias.

Conclusions

In summary, we have successfully used a structured interview process for EPA design, and an e-mail–based modified Delphi process for EPA revision, to create an elective ECMO EPA for adult critical care fellowship programs. We have defined the core essential functions of a physician performing ECMO and the multiple forms of assessment to guide summative evaluations to provide a framework that can be used to guide competency-based ECMO education. Future directions include validating the assessment methods, clinically implementing the EPA in a variety of critical care settings, and soliciting further feedback and refinement as the EPA is put into practice.

Author Contributions

WCC designed the study, conducted the interviews, performed the thematic analysis, drafted the EPA, and authored the manuscript. BZ facilitated interviews and refined the EPA internally. HH transcribed the interviews. KB helped in the study design and refined the EPA internally. DZ helped in the study design, facilitated interviews, and refined the EPA internally. All authors reviewed and approved of the manuscript. WCC and DZ take responsibility for the integrity of the data.

Conflict of Interest

BZ is chair of the ELSO Education committee.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.