Implementation of Crew Resource Management: A Qualitative Study in 3 Intensive Care Units.

OBJECTIVES: Classroom-based crew resource management (CRM) training has been increasingly applied in health care to improve safe patient care. Crew resource management aims to increase participants' understanding of how certain threats can develop as well as provides tools and skills to respond to such threats. Existing literature shows promising but inconclusive results that might be explained by the quality of the implementation. The present research systematically describes the implementation from the perspective of 3 trained intensive care units (ICUs).
METHODS: The design of the study was built around 3 stages of implementation: (1) the preparation, (2) the actions after the CRM training, and (3) the plans for the future. To assess all stages in 3 Dutch ICUs, 12 semistructured interviews with implementation leaders were conducted, the End-of-Course Critique questionnaire was administered, and objective measurements consisting of the number and types of plans of action were reported.
RESULTS: The results categorize initiatives that all 3 ICUs successfully launched, including the development of checklists, each using a different implementation strategy. All ICUs have taken several steps to sustain their approach for the foreseeable future. Three similarities between the units were seen at the start of the implementation: (1) acknowledgment of a performance gap in communication, (2) structural time allocated for CRM, and (3) a clear vision on how to implement CRM.
CONCLUSIONS: This study shows that CRM requires preparation and implementation, both of which require time and dedication. It is promising to note that all 3 ICUs have developed multiple quality improvement initiatives and aim to continue doing so.

Crew resource management (CRM) has been increasingly applied in health care to improve safe patient care.[1] It consists of a team training that was developed in the aviation field and aims to increase participants' understanding of how certain threats, such as miscommunication, can develop[2] as well as provides tools and skills to respond to such threats. The focal point of the training is nontechnical skills, such as communication, teamwork, leadership, situational awareness, decision making, and problem solving.[3] These nontechnical skills complement the technical abilities and contribute to safe and efficient task performance.[4] At long last, CRM should be embedded in the organizational culture as a way of doing things.[2]

The findings of evaluations of CRM as a classroom-based training are promising but inconclusive with regard to behavioral change.[5] For instance, McCulloch and colleagues[6] found an increase in the use of nontechnical skills for nurses but not for anesthetists or surgeons. Rabol et al[5] recommend more qualitative research to get a deeper insight into why these mixed results occur. The effects of CRM—and interventions in general—are determined by the persuasiveness of its program, as well as the quality of the implementation.[7,8] Therefore, the quality of the implementation might explain the mixed results of CRM.

To date, implementation has never been the main focus of CRM evaluation research. It has mainly been described alongside the quantitative results in 2 ways. It has been described, first, as a predefined part of the training, expressed in the Methods section. For instance, Stead et al[9] state that the implementation of CRM comprises 3 phases: site assessment, training, and sustainment. Second, it has been described by discussing the main barriers and facilitators perceived by the researchers while conducting their study. Morey et al,[10] for example, stress that support from management was a prime facilitator in the implementation of CRM.

Although both ways of describing the implementation yield valuable information, they do not depict the whole process of implementation and overlook the underlying vision, structure and follow-up. An exception is the study by Marshall and Manus,[11] in which they described the important characteristics, goals, changes, barriers, and facilitators for each participating department. They did not, however, distinguish different phases of implementation.

The present research systematically describes the implementation from the perspective of 3 trained intensive care units (ICUs) based on interviews with implementation leaders. The choices, rationales, and consequences that played a role in the implementation process will be assessed, whereas existing system change frameworks present in each unit will be used to characterize the implementation in each ICU. In this way, we aim to gain insight into contextual factors influencing the effects of CRM and to present practical examples to readers interested in the implementation of CRM. In addition, increased understanding of the implementation process of CRM in 3 ICUs might help explain the effects of CRM. In short, the present study takes a first glance at the implementation of CRM.

METHODS

Design and Setting

The design of the study was built around 3 stages of implementation: (1) the preparation, (2) the actions after the CRM training, and (3) the plans for the future. To assess all 3 stages, semistructured interviews with implementation leaders were conducted. In addition, after the training, a questionnaire was administered to measure the reaction of the participants about CRM training. Finally, we collected all plans of action that had been formulated during the training sessions.

Three Dutch ICUs participated in the present study and received CRM training. The ICUs were part of nonacademic teaching hospitals in The Netherlands, with a mean of 872 beds, all located in an urban environment. The ICUs had 12 to 14 beds and 65 to 79 unique employees. All units delivered level 2 care, which implies a 24-hours-per-day availability of an IC physician for the care of patients, and structurally around 0.40 full time-equivalent IC physicians per bed.[12] The medical staff of the ICUs included IC physicians, residents, nurses, nurse trainees, and members of the management. The training was not free of charge; therefore, the ICUs had to be able to make the necessary financial and organizational arrangements. The efforts to make these arrangements ensured that the ICUs were willing to receive CRM training. The selection process of the participating ICUs is described elsewhere.[13]

All ICUs started discussing CRM seriously within their unit after being approached to participate in the larger effectiveness study[13] of which the present research is a part. This provided the opportunity to monitor these ICUs from the point at which the first person—the pioneer—tried to convince staff about the benefits of CRM until 15 months after receiving the CRM training. The larger study compared 3 ICUs that received CRM training with 3 comparable ICUs in a controlled trial with 1 premeasurement and 2 follow-up measurements. The trial aimed to assess all levels of the Kirkpatrick evaluation framework for training programs (reaction, learning, behavioral change, and organizational impact). The study was approved by the Ethical Committee of the VU Medical Center. Participation was confidential, and all gathered data were stored entirely anonymously.

Intervention: CRM Training

The CRM training was classroom based and consisted of 2 consecutive days from 9 am until 5 pm. Because a maximum of 15 participants per session had been set, several trainings were organized to include all members of the IC staff. Two ICUs received 4 sessions, and 1 ICU received 6. In total, 14 CRM sessions were organized, and in such a way that each discipline was represented during the sessions, guaranteeing a multidisciplinary audience. Each ICU was trained separately.

The training aimed to improve patient safety on all fronts of the ICU by creating awareness regarding the threats of unsafe behavior on the individual, team, and organizational level. Subsequently, the participants were stimulated to develop ways to recognize these threats and to prevent negative consequences on each of these levels. These solutions were continuously recorded during the 2 days of training by means of writing down concrete plans of action. An example of an action on the team level is to explicitly appoint a coordinator of the day to improve and clarify leadership, teamwork, and communication.

The ICUs were free to choose how they wished to organize their implementation of CRM and the plans of action that were formulated during the training. The 2 CRM instructors were available as consultants for a period of 1 day after the ICU had been trained. It was up to the ICU to decide on how to use this help (e.g., get organized, implement changes, reiterate theory). Furthermore, ICUs were encouraged to form a CRM change team. A detailed description of the training can be found in Kemper et al.[13]

Measurement and Participants

Interviews

Twelve semistructured interviews were conducted to assess the progression of the implementation process. These interviews were held at 3 moments in accordance with the 3 stages of implementation: (1) just before the CRM training, (2) 4 to 6 months after the training, and (3) approximately 15 months after the training. The content of the interviews was based on the phases of Grol and Wensing's[14] implementation model. The first interview focused on the “orientation” phase, the first phase of the model, which included a raised awareness and interest for the intervention. The second interview focused on “change”—phase 4 of Grol and Wensing—in which CRM is tried or used. Objective data on the types and number of changes that were implemented were recorded and categorized for each ICU. The third interview focused on maintenance—the fifth phase of Grol and Wensing—in which the integration into the daily routine and sustainment of CRM are the central themes. The phases of insight and acceptance—phases 2 and 3 of Grol and Wensing—were not used for the interviews because these are part of the CRM training, rather than part of its implementation by the ICUs. All interviews were conducted and transcribed by the first author.

The participants were all implementation leaders in each phase of the implementation, being identified through the various contact moments related to the starting up of the study. For the preparation phase, the pioneers were interviewed. These were the persons who first brought up the topic of CRM and convinced staff and management to train the whole ICU. When the pioneers were not responsible or involved with the further implementation of CRM, a second interview was conducted with the person who had in fact prepared the implementation. For the stage after CRM training and plans, the chair of the CRM change team or the person responsible for the implementation of CRM projects was interviewed. Some persons fulfilled several of these roles (e.g., pioneer and chair of the change team) and were therefore interviewed several times. In total, 12 interviews were conducted with 8 persons (Table 1). Besides implementation leaders, regular participants were initially interviewed as well. The forthcoming information was, however, too detailed and intertwined with specific initiatives that it lost its value to describe the general implementation. Therefore, these interviews were not continued and not included in the present study.

TABLE 1

Overview of the Topic List Divided Over the Phases of Grol and Wensing[14] for the Interviews, With Key Words and According to Main Questions for Each Stage in the Implementation Process

The interviews were digitally recorded and were worked out as chronological narratives, which were then presented to the interviewed persons, so-called respondent validation,[15] to minimize a biased interpretation of the interviewer. The narratives were then split up and categorized into the relevant stage of the implementation process, followed by a further subdivision across the topics within each stage (Table 1). The topics were based on aspects of the implementation model of Grol and Wensing and related implementation literature[16,17] as well as previous CRM evaluations.[11,18,19]

End-of-Course Critique

The reaction of the participants was assessed using the mean score on the End-of-Course Critique (ECC). All participants were asked to fill out the ECC at the end of the training. The ECC was originally developed by Grogan and colleagues[20] and adapted for use in the ICU. It measures the reaction immediately after the training, expressed as the extent of the perceived relevance and utility of the specific topics covered in the CRM training (e.g. “The session about ‘Human Factors’ was relevant and useful”). The ECC consists of 14 statements that are to be rated on a 5-point scale, varying from “strongly disagree” to “strongly agree.”

Plans of Action

Throughout the CRM training (i.e., for each of the topics covered), the participants were asked to write down concrete issues that they wanted to address in their ICU. These plans of action provide an insight into what kind of quality improvement initiatives CRM evokes.

RESULTS

Before the Training: Orientation

Table 2 shows that the 3 ICUs in the present study approached CRM in a similar way. All report a professional interest in quality and, in particular, CRM. The pioneers of all 3 ICUs indicated that they thought CRM was a new and promising opportunity for quality improvement. Their interest in quality improvement influenced the decision to place CRM on the agenda. In addition, all ICUs indicated that communication, whether it is multidisciplinary, between departments, or as part of teamwork, was something that could be improved. These 2 aspects provided the necessary momentum to initiate CRM training. The process of getting the green light for CRM in all 3 ICUs comprised several stages of convincing relevant stakeholders. All ICUs mention the costs of the training and staff hours as a barrier during this stage.

TABLE 2

Responses of the Participants of All ICUs to the Aspects of the Preparation Stage (1) as Described in Table 1

After the Training: Reaction (ECC) and Plans of Action

The participants rated the CRM training very positively in the ECC (mean, 4.47; SD, 0.45). In total, 94 different initiatives were formulated in the plans of action during the 14 training sessions of the 3 ICUs. Because reporting all initiatives would be too comprehensive, we focused on the ones that were mentioned by all 3 ICUs. These initiatives were most common at the unit level. The organizational initiatives concerned structural changes, such as assigning a coordinator of the day. The team level comprises initiatives that require teamwork, for instance, creating situational awareness by using a time-out procedure. The smallest number of common initiatives was found at the individual level. The participants indicated that they wanted to further develop nontechnical skills and to give feedback to each other. Table 3 summarizes those plans of action that were mentioned by all ICUs.

TABLE 3

Summary of the Plans of Action That Were Mentioned by All ICUs

After the Training: Change

All 3 ICUs report that several initiatives were implemented. The number of implemented initiatives designed to promote quality was recorded and categorized. In addition to these planned changes, they also report alterations that were not part of any plan. A common theme is openness in communication because patient safety is discussed more in ICU A, there is a receptive environment for a debriefing in ICU B, and ICU C reports enhanced mutual communication.

The results of the interviews at this stage (Table 4) reveal that all 3 ICUs had a clear, yet different, vision on what they wanted to accomplish with CRM. This vision determined their follow-up process and resulting changes. The strategy of ICU A showed a strong resemblance to the system change framework of Kotter.[22] In line with Kotter's framework, the change team thoroughly prepared CRM, by first formulating a mission statement and formally involving the management. They communicated their vision to the personnel regarding the envisioned change by means of a weekly newsletter and used CRM training to empower the staff to act on this vision. After the training, they started implementing small adjustments, so-called quick wins, such as an extra whiteboard and the mounting of photos, to pave the way for larger projects.

TABLE 4

Responses of the Participants of All ICUs to the Aspects After the Training as Described in Table 1

Intensive care unit B started with the implementation of some of the easier initiatives that were mentioned in the plans of action. Their further implementation was characterized by a focus on the development of checklists for high-risk situations that were identified during the training. To develop these checklists, ICU B explicitly chose to integrate CRM for a large part into the Comprehensive Unit-based Safety Program (CUSP).[23] The CUSP is a strategic framework comprising 5 steps specifically designed to reduce preventable harm through the development of tools, such as checklists. The CUSP has been successfully applied in the ICU.[24] Intensive care unit B used the CRM training to educate the staff on safety science, which was the first step of their CUSP program. With the use of a prospective risk analysis method, ICU B developed several checklists for standard operating procedures and handovers.

The implementation by ICU C can be characterized with the social movement approach.[25] Within this approach, implementation is an unstructured, self-organizing, and auto-catalyzing process. The commitment of the staff to the intervention is essential. In the eyes of the implementation leaders of ICU C, the CRM training elicited this kind of commitment. Especially, the development of a mutual CRM vocabulary helped the staff to express and address issues of quality and safety themselves. Besides allocating structural time to elaborate ideas, there was no structured guidance regarding which issues, derived from the plans of action or newly developed, should be chosen. Furthermore, no change team was put together to aid the implementation.

Although the ICUs differed in how they organized their changes, some resemblances were found. All ICUs used the plans of action as a starting point for change after the CRM training. After prioritizing and categorizing, these plans formed the input for the first initiatives. An overlap in themes was seen with regard to the implemented changes. All ICUs revised the role of the coordinating nurse and developed checklists, for instance, for the transport of a patient.

An important barrier, mentioned by all ICUs, was the lack of implementation knowledge and skills. For instance, how do you get and keep the staff involved, especially the less “CRM-enthused” part? This might explain why some initiatives were not well received, despite the bottom-up approach of the training, or perhaps the development of “implementation fatigue” played a role.

Plans: Maintenance

Intensive care units A and B indicated that they continued working with their change teams as a structural part of their unit. The goal of ICU A was to change the safety culture, whereas ICU B wanted to continue the development of checklists for high-risk situations. Intensive care unit C wanted to integrate the CRM mechanism of recognizing risks and addressing them, as a “normal” way of doing things, rather than as a “special” project. There were no concrete plans to structurally sustain CRM in ICU C. The results are summarized in Table 5.

TABLE 5

Responses of the Participants of All ICUs to the Aspects of Plans for the Future as Described in Table 1

DISCUSSION

The results of the present study demonstrate that all 3 ICUs successfully launched several initiatives, each using a different implementation strategy. Furthermore, all ICUs have taken several steps to sustain their approach for the foreseeable future. Despite the variety in strategies, 3 similarities can be seen between all 3 implementation processes that were crucial at the start of the implementation. First, all units reported problems with communication during the orientation phase. This acknowledgment of a performance gap is an indication that the participating ICUs, or at least the pioneers, possess a sense of urgency to change. This is an important first step for further implementation.[16] Second, all ICUs allocated structural time for quality improvement before CRM and for change after CRM. Third, despite having different strategies, all units had a clear vision regarding their goals and strategies concerning CRM.

All ICUs indicated that they would use the plans of actions that were formulated during the CRM training as a starting point for their follow-up initiatives. The role of the coordinating nurse and the development of checklists are themes that recur in each ICU and are in line with CRM topics such as leadership and standardized communication.[26] All ICUs mention the costs of CRM and a lack of implementation expertise as important barriers during the orientation and the change phase, respectively. The fact that CRM was perceived by the implementation leaders as a new and promising way to improve patient safety, as well as educating the whole staff, were regarded by all 3 ICUs as facilitating factors ro receive CRM training. Finally, all ICUs reacted very positively in the ECC.

The flexibility of the CRM follow-up initiatives provides opportunities but also creates pitfalls. An advantage of the flexibility is that the initiatives can be tailored to the specific situation and can be integrated in existing programs. For instance, Tapson et al[27] used CRM training to successfully enhance the appropriate use of venous thromboembolism prophylaxis in surgery. A pitfall of the flexibility is that a large number of CRM initiatives can be overwhelming and may lead to implementation exhaustion. Box 1 presents suggestions proposed by the implementation leaders to get the best out of the initiatives.

Text Box 1

Suggestions of the participating ICUs

Be ready to solve problems, not just to identify them

Get advice from implementation experts

Formulate end goals and evaluate them

Start directly after the CRM

Schedule enough time in advance

Be aware that CRM can easily lose momentum

The diversity in follow-up initiatives could explain the mixed results of classroom-based CRM training on a behavioral level as reported by Rabol et al.[5] First of all, as a result of the diversity, it is possible that an outcome that is used in an evaluation is not applicable for each site that is being studied. For instance, the use of a checklist is an often-used endpoint in CRM evaluation.[5] When applied to the present study, we probably would have found a large effect in ICU B, a small effect in ICU A, and no effect in ICU C. This exemplifies the difficulty in defining an outcome that is applicable in all units, especially when the evaluation is designed even before the training, in accordance with scientific discourse.

Second, it can be questioned whether behavior in general is likely to change as a direct result of diverse, smaller interventions within the evaluation period of 1 year. It can be argued that these initiatives influence behavior by changing the safety culture. When the implementation of CRM initiatives is perpetuated over time, it will change the way people think about issues regarding safety and quality. This resonates in the social norms, which partly determine behavior.[28] Behavioral change through culture takes much longer than 1 year; for instance, in aviation, it took approximately 40 years to gradually, but steadily, establish the safe culture that exists today.[29] However, once established, the change will be very robust.

On the basis of the present study, we are unable to recommend one implementation strategy over the other. Although the ICUs share some similar initiatives (e.g., role of the coordinating nurse), the execution was always a bit different. Therefore, we could not compare the ICUs on the same endpoints and make valid assumptions on which strategy led to the best results. In addition, it can be questioned whether 1 implementation strategy would have led to similar results in all ICUs. The ICUs chose their strategy on the basis of a clear vision on what they wanted to achieve with CRM. Therefore, the strategies were highly dependent on the context to which they were applied. Existing literature also shows that context is an important aspect when determining which implementation strategy should be favored.[30,31] However, more research on this topic would be an interesting venue for future studies.

Limitations

The present article is an exploration to qualitatively study the implementation process of CRM. The number of ICUs is limited; therefore, the results should be considered preliminary. In addition, the interviewees were all actively involved with CRM; therefore, their perception might be slightly positively biased. To counteract this bias, the interviews were focused on objective results, such as which projects were actually implemented.

The external validity is limited by the number of ICUs that participated in the present study. Because this study was conducted as part of a larger effectiveness study,[13] it was bound to the number of ICUs that participated in that study. Furthermore, participating in the effectiveness study required organizational and financial commitment, reflecting a willingness to receive CRM. This willingness should be taken into account when considering the external validity.

CONCLUSIONS

This research shows that CRM requires preparation and implementation, both of which require time and dedication. Consequently, it involves more than 2 days of training. The study illustrates that, despite the differences in vision concerning how to approach CRM, all 3 ICUs in the current research developed and implemented their own locally owned initiatives. The multitude and diversity in initiatives reflect the catalyzing effect of CRM on new and existing quality initiatives. Furthermore, it indicates that CRM helps participants to recognize, address, and handle safety issues. Finally, the diversity in initiatives may help explain the mixed results in outcomes in the present CRM evaluation research.

The results of the present study suggest that units that are considering CRM should base their strategy on a clear vision. The implementation strategy should probably be close to their own previous experience with the implementation of other projects. Structural time should be made available for preparation and implementation. The implementation should be tailored to the specific situation, depending on what goals are to be achieved using CRM.

All in all, it is promising to note that all 3 ICUs in the current research, despite their own barriers, visions, and strategies, developed multiple quality improvement initiatives and aim to continue doing so.