Using simulation to help healthcare professionals relaying patient information during telephone conversations.

BACKGROUND: The purpose of this paper is to describe a development project in which simulation was used to improve the telephone-based conversations between nurses in an emergency department (ED) and physicians from different specialties taking care of acutely ill patients.
METHODS: A needs analysis consisting of observations and interviews was conducted and a one-day simulation-based interprofessional team training course was developed. Observations of phone conversations pre-course, three and six months after the course were conducted in the clinical setting with 20 participants in each point of time. A 14-item evaluation tool was used to record how many information pieces were communicated.
RESULTS: Five courses were conducted for 66 nurses/nurse assistants and 17 physicians. 9 out of the 14 items were reported significantly more after the course. Item that were not reported in the pre-measurement, increased to around 20% reporting three months after the course but then fell to close to 0% again after six months.
CONCLUSIONS: The patterns of retention and decrease of the effect could be an indicator for norms, values and beliefs held by professions about what constitutes their task.

Introduction

Modern diagnostics and treatment of patients require frequent coordination and communication between healthcare professionals from various backgrounds. Challenges in interprofessional cooperation can compromise the care for patients [1,2,3,4,5,6,7,8]. The challenges are rooted in different aspects, including communication problems [9,10,11,12], or hierarchical power structures [13,14,15]. While growing into the field, roles, norms, values and beliefs are formed [16,17,18] and thus different aspects of care might be prioritized differently depending on professions and specialties [19,20,21,22,23,24,24]. There are also legal issues that make cooperation difficult, when considering breeches of standard procedures and responsibilities to keep up with work demands [25].

Some of the coordination of the patients' care and treatment is done over the telephone. The conversations might have different goals, including calling a colleague to help with a patient, requesting more information, medical prescriptions, coordinating care or arranging the handing over of the responsibility of the care for the patient to a colleague [26]. Several studies have found such coordination to be error prone in many regards. Relevant information is passed on incomplete, containing errors, or containing “made up”, additional elements [27,28]. The phone-based conversations pose additional challenges to the interprofessional cooperation, such as establishing a shared situation awareness, since more needs to be made explicit verbally, that could have remained implicit or on the level of body language in face-to-face conversations (e.g. transporting a sense of urgency). The purpose of this paper is to describe a development project in which simulation was used to improve the telephone-based conversations between nurses in an emergency department and physicians from different specialties taking care of acutely ill patients. We describe the needs analysis steps and the design, implementation and evaluation of the intervention.

Methods

First, we describe the setting and the needs analysis on which we based the development of the intervention (the course). Then we describe the methods used to develop and implement the course and finally we describe how we evaluated the effects of the course. The basic timeline of the project can be seen in Table 1. The description of the project follows the chronology of its steps. The description is based on a research diary that was kept throughout the project.

Table 1

Timeline of the project.

Activity	Time
Contact from the leadership in the ED	April 2010
Development of course concept, meeting with the head of ED, approval of observations and agreement for course concept	May–September 2010
Pre-observations	October 2010
Course conduct	November 2010–January 2011
3 months post observations	End of March 2011
6 months post observations	End of June 2011

The setting

The study was conducted in the emergency department (ED) of a tertiary hospital in the capital region of Denmark. The ED deals with all types of patients. The ED receives an average of 90 patients in 24 h. The leadership of the ED department comprises a leading physician and a leading nurse. They are both located within the ED but have organizational functions and thus are typically not involved in the treatment of patients. The nurses are employed directly in the ED, while the physicians are not employed in the ED, but in other departments within the hospital and attend patients in the ED if needed. This tends to lead to a different “attachment” to the ED, when comparing nurses and physicians. In informal conversations, physicians often would say that they consider themselves to be “visitors” in the ED.

When a patient arrives in the ED, an initial triage is done by the triage nurse who assigns the patient into a category [29,30,31,32,33,34]. This category determines the patient's maximum waiting time before being seen by a physician. The patient is then moved to another room in the ED and handed over to a treatment nurse, who decides which physician to contact, based on the symptoms, he or she noted in the patient. The treatment nurse conducts the telephone call to the physician. Together the nurse and the physician then arrange for the next steps over the telephone. Often this next step is that the patient is seen by the physician. Frequently, there will not be any face-to-face meeting between the calling nurse and the physician arriving, as the nurse is responsible for several patients and might be in a different room upon the arrival of the physician.

The information about vital signs and symptoms to be collected by the nurse has been standardized along the Airway-Breathing-Circulation-Disability-Environment (ABCDE) [35] structure and the ADAPT (Adaptive Process Triage for Emergency Care) tool has been introduced for the recording of this information [36,37]. However, when the project began, ADAPT was not used to report the recorded data during the telephone conversations under investigation.

Previous attempts to improve the phone-based conversations between treatment nurses and physicians have not been successful and the Copenhagen Academy for Medical Education and Simulation (CAMES) was contacted by the head of the ED to help with the implementation of the tool. The request was to design and implement a training course to improve the use of the ADAPT tool during telephone conversations. CAMES staff is well known and respected in the ED, based on prior collaborative work on patient safety [26].

Table 2 provides a summary of the development of the course in relation to Kern's six step approach [38].

Table 2

Summary of the course development according to Kern's six steps (see text for details).

Problem identification and general needs assessment	• Phone conversations are a necessary part of work coordination, but pose a risk for information loss. • Danish emergency departments (ED) depend on effective communication to ensure the timely consultation of patients by physicians, who are not directly employed in the ED. • Previous attempts to optimize phone-based conversations did not succeed to the intended level.
Targeted needs assessment	• Observations and interviews showed specific problematic communication patterns, including incomplete patient descriptions, lack of planning of the next steps in the care for the patient, unclear timing of the next steps, and undefined responsibility for these next steps.
Goals and objectives	• The aim of the course was to improve phone-based conversations between nurses and physicians (see text for detailed objectives)
Educational strategies	• Mixed methods, including lectures, workshops, simulations, and debriefings within the experiential learning paradigm (see Table 3 or details)
Implementation	• Five courses with 83 participants (66 nurses and 17 physicians).
Evaluation and feedback	• Questionnaire-based feedback on attitude towards the course. • Observations of behaviour changes in clinical practice.

Table 3

Condensed version of the course and evaluations results.

Title	Learning strategy	Time	Evaluation score∗
			Median (Range)
WelcomeIntroduction to simulation	• Presentation of the program and learning goal • Introduction to simulation	40 min.	2 (1–4)
Team thoughts/handover	• Plenum session • How to communicate and handover patient information? • What needs to be in the handover information? • How is information received and understood? • What does the recipient ‘want’ to hear? • Who has responsibility?	45 min.	2 (1–3)
ABCDEWorkshop	• Division into Simulation teams • Brief introduction to ABCDE workshop • Group walkthrough with the participants of ABCED understanding and meaning • ABCED exercise on mannequin	40 min.	1 (1–3)
Simulation 1,2,3 with debriefing	• Full scale simulation • Division into Simulation teams • 3 interdisciplinary simulations with debriefing during the day	60 min x3	1 (1–4)
Case prioritization	• Division into Simulation teams • Group exercise based on ‘Patient cards’ (displays, age, symptoms, vital values, triage color and more) to prioritize and distribute resources to maintain an overview in a full ECU • What kind of information would patient handover contain? • What information is vital in the communication about the patient? • Inspiration from interviews of personnel and/or observations from Study in ward A	30 min.	1 (1–3)
What should the participants focus on in the future?	• Plenum session • What should the participants focus on in the future? • In collaboration with the instructor, individual plans are compiled • The plans should contain a plan for a follow up between the participant and leader no more than 30 days later	45 min.
Overall impression of the course			1 (1–2)

Scale: “1 - very good”, “2 - good”, “3 - bad”, “4 - very bad”.

Needs analysis methods

A needs analysis focusing on the phone-based conversations was conducted based on observations and interviews. The observations were conducted by LFP with a convenience sample of 10 cases in the ED, involving day (7 cases), evening (1 case) and night shifts (2 cases). During these observations, the nurses as well as two of the attending physicians were involved. No demographics were collected on these participants. Data was collected in notes which were discussed in the research team. The results were also discussed with the heads of the ED, who then approved the development of an educational intervention (the course).

Course development and implementation methods

The course aimed to fulfill the goals of the The Joint Commission National Patient Safety: “Improve the effectiveness of communication among caregivers, requirement. 2c: Measure, assess and, if appropriate, take action to improve the timeliness of reporting of critical test results and values. 2e: Implement a standardized approach to “hand-off” communication, including opportunity to ask and respond to questions” [39]. Further literature on (phone-based) nurse-physician conversations was taken into account [40]. The importance of shared mental models for interventions regarding the patient treatment has been emphasized in other work [41,42,43,44,45].

The result of the development work was the course concept, which is described in the results section. The experiential learning paradigm was the guiding educational principle in the course [46]. As the course was aimed at interprofessional, post-licensure participants, we matched this characteristic of the participant group within the interprofessional instructor team (nurses and physicians) with experience in acute medicine and in training multidisciplinary teams in simulation settings [47]. Simulation in interprofessional training has a considerable history [48] and has shown to be effective for interprofessional teamwork training [49,50,51,52,53,54,55]. We found helpful recommendations for the design of simulation-based interprofessional education [56,57,58,59,60,61].

The course was mandatory for the nurses working in the ED. The invited participating physicians worked in one of two departments which are affiliated to the ED: a mixed medical/surgical gastroenterology department and a large general medical unit comprising endocrinology, respiratory medicine, and geriatrics. These two departments are home to the physicians who treat most patients in the ED (combined about 30% of all admissions). The chief executive officer of the hospital approved to enroll six physicians from these departments in each course. It was expected that at least three physicians should participate in each course, so that there was a physician for each scenario. Participants were paid their regular salary during course participation and were freed from clinical work during their participation. The need for such a strong integration into the organization was emphasized in the discussion with all stakeholders.

Evaluation methods

The evaluation of the course was based on two aspects: 1) observations in the clinic to check for changes in behavior and 2) attitudes towards the course. The observations address the Kirkpatrick level 3 (application of what was learned in clinical practice), while the attitude addresses level 1 (reactions) [62].

To assess the behavioral changes in the clinic, an evaluation tool containing 14 items (Figure 1) was developed by the project group. We based the items on the data in the ADAPT tool and the additional requirements of the Joint Commission International accreditation standards for hospitals for any handover situation [39]. The items addressed patient identification, triage scoring, and reaching a clear agreement on the plan for further treatment. The tool contained dichotomous categories to record which items on the ADAPT tool were actually reported during the telephone conversation. The tool also provided space for taking notes from the conversations. The tool (Figure 1) was pilot tested on 5 cases and some minor adjustments to the wording of the items were made after discussion in the research group. The heads of the ED department reviewed the tool for content and found it covering the points important for a hand-over conversation. There was no further investigation of the psychometric properties of the tool. The tool provided space to record both, information given by the caller and the receiver of the call. Typically, only the information could be recorded that the caller provided, as this was the only side of the conversation that LFP could see and hear. In some cases, however, it was possible to deduce what the other person in the conversation said or asked. As this happened in few cases only and as this deduction process seemed too unreliable, we only report those part of the data that describe the callers' delivering behaviors systematically. We use the deduced information for illustrative examples in Table 5.

Figure 1

Evaluation tool for the pieces of information delivered during the phone conversations.

Table 5

Illustrative examples of telephone conversations. The questions by the physicians were inferred by the answers of the observed persons.

Measurement point	Observation	Interpretation	Items reported
Pre course	Nurse: “This is Andrea∗from the emergency department. Elli was admitted here with diarrhea after she got antibiotics in relation to her pneumonia”Physician asks for the patient's social security numberNurse: “xxxxxxxx-4455”∗∗∗∗Physician: “OK, thanks. Bye”Nurse: “Bye”	Note that much information is not said and stays implicit, for example any plans on what to do with the patient. The nurse doesn't know if and when the physician will come to the ED. The physician does not get any information about the triage category in which the patient was placed.	□ Name□ Location from were call is made□ Referral (self admission, general practitioner, ambulance, other hospital department)□ Patient's symtoms□ Patient's diagnosis□ Social security number of the patient□ Age of the patient∗∗∗□ Name of the patient
3 month after course	Nurse: “This is Becky∗from the emergency department. I have Martin∗here, having a low blood pressure with 74/44. I think it is too low.”∗∗Physician: “Ok – I will come immediately”Nurse: “Thanks, bye”∗∗Physician. “Bye”	Note that although no triage category is mentioned the description of the low blood pressure together with the expression of concern which is sufficient for the physician to promise to come to the ED immediately.	□ Name□ Location from were call is made□ Patient's symtoms□ Name of the patient□ Triage category of the patient□ Relevant vital parameters of the patient
6 months after course	Nurse:” This is Maria∗from the emergency department. I have Thomas∗with observations of urinary infection. His triage category is green. And his temperature is as high as 38,5°C. When do you expect to come to the ED?”∗∗ Physician: “in about ½ hour”	Here some relevant vital signs (temperature), background information, and the triage category are provided.	□ Name□ Location from were call is made□ Patient's symtoms□ Patient's diagnosis□ Reason for the call□ Name of the patient□ Triage category of the patient□ Relevant vital parameters of the patient□ Use of ADAPT during call

Names changed by authors.

The physician statements were not heard by the observer, the information was deduced from the answer and in some case in short clarifications after the end of the call.

The age of the patient is implicitly given with the social security number, which contains the birthday as the first 6 digits.

The evaluation tool was used in the observations conducted by LFP before the course, as well as 3 and 6 months after the course. The observations were convenience sampled and done during daytime over a two-week period. Based on the scope of the project, it was not possible to collect data also during other times of the day or with more observers. These type of phone-based conversations happen rather frequently, but an observer still needs to spend considerable amounts of time collecting them during a day.

Behavioral changes between the points of measurement were operationalized as changes in the score in the evaluation tool [62]. Chi Square tests for three independent samples were used to test the differences between the measurement points. We chose the independent procedure, as we could not ensure to observe the same person on all occasions. A two-sided p-value of .05 or lower was considered as significant. The physicians were excluded from the analysis for several reasons: Out of the 83 participants in the course, 17 were physicians, mostly in the beginning of their careers. Therefore, many of the physicians were rotating to different departments, partly different hospitals during the project time. This rotation is part of the Danish educational system. Finally, as the physicians are working in different departments in the hospital, scheduling observations with them and then waiting for them to actually receive phone-based conversations about patients would have meant a workload that was beyond the scope of the project.

For investigating participants' attitudes towards the course, we developed an evaluation form based on a four point scale (“1 - very good”, “2 - good”, “3 - bad”, “4 - very bad”) and free space for comments. All participants in the courses filled in this form on a anonymous basis. It was not subjected to testing its psychometric qualities.

Ethics

Danish law exempts projects in which no intervention on patients take place from a formal review by the Ethics Committee. Each observed person was informed about the study, its voluntary character, and that they could withdraw any time.

Results

To preserve the anonymity of participants, to minimize the burden for study participants, and because of the explorative nature of the study, we did not collect any descriptive data of the participants. The participants can be assumed to have a range of experience in their positions. The majority of the nurses employed in the ED and of the physicians in the different departments in the hospital work full-time. LFP performed the observations and was also involved in the courses and coded, whether the nurse performing the call did participate in the course or not from memory.

Results from the needs analysis

During the needs analysis it became evident from the interviews and observations that treatment nurses did fill in the ADAPT scheme but rarely used it to a) structure their approach to patients and b) to communicate with the physicians in the telephone conversations. The nurses and physicians did not find the ADAPT tool useful for patient care and felt it was just “one more piece of paper” that prevented them from actually caring for the patient. Further, it was considered to be a tool used by nurses only.

There was little observable structure for the telephone conversations. A common sentence was “you have a patient” as the only relevant content of the conversations. It was not always clear that a) the sender of this message actually intended to defer the responsibility for the patient to the receiver, which is a defining element of handing over a patient to a colleague [26] and b) that the receiver actually took over the responsibility for the patient. Even when basic vital signs were out of the “normal range”, they were not always mentioned nor asked for during the conversations. At the end of the conversation there was often no agreed plan for further steps. This resulted in delays, missed steps and different mental models of the treatment nurse and the physician about the patient's challenges. Department internal guidelines stated: “the patient must be seen by a nurse, who has the responsibility for placing the patient in a team within 15 min”. The responsibility for further next steps after notifying a physician about a patient was not defined.

Results from course development and implementation

The aim of the course was to improve phone-based conversations between the nurse and the physician. The learning objectives were developed: a) to be able to identify the reason for calling as the aim to hand over the patient to the physician, b) to use the ADAPT tool to classify patients and to use the recorded information to structure the telephone conversation, c) to clarify and to agree upon the plan for the next steps in the care for the patient and d) to clearly acknowledge the next tasks and responsibilities for both nurse and physician. The basic assumption for the course was that the telephone conversation would improve if an increased number of the relevant points of information (distilled in the evaluation tool in Figure 1) would be covered during the conversation.

The course consisted of three workshops and three manikin-based simulations followed by debriefings. The debriefing ran according to the three phases (description, analysis and application) described by Steinwachs [63] and took about 30 min. Within the analysis phase the facilitators centered the discussion on how to improve the phone conversations.

Five courses were conducted during a three-month period. A total of 66 of the 87 nurses (75%) employed in the ED and 17 of the 58 physicians (29%) employed in the participating departments took part in the course.

In the first workshop, each participant got hands-on experiences by examining a manikin based on ABCDE assessment in different patient's cases. The participants were asked to react upon the manikins vital parameters and report this information to the other members in their team. In the second workshop, participants contrasted their current practice of phone-based conversations with a more structured approach based on the ABCDE sequence and the available information in the ADAPT. The third workshop focused on the prioritization of different patients. The objective of this workshop was to help participants see, that the information in ADAPT could give them an overview based on the ABCDE assessment. The simulations were based on patient cases with different symptoms and triage codes: septic shock, acute bleeding and acute brain injury. The objective of the simulations was a) to train the participants in using the ABCDE assessment, b) to improve their non-technical skills during the telephone conversations (e.g. double checking a mutual shared understanding of the plan for the next steps) and c) to accurately describe the patients' condition based on the information in the ADAPT.

At the end of the course each participant wrote an individual learning plan answering the following questions:

What is my goal?

What do I gain from that goal?

How do I achieve my goal?

By what time should I have achieved my goal?

A condensed version of the course elements is provided in Table 3 together with evaluation ratings for each course element.

Results from the behavioral changes evaluation

Three months after the course, all 20 observed nurses had participated in the simulation-course. Six months after the course, 7 out of 20 telephone conversations were carried out by nurses who had not participated in the course, due to personnel fluctuation. Figure 2 provides the results of the 60 observations conducted, 20 before the course, 3 months after, and 6 months after the course, respectively. All observations were performed during day shifts.

Figure 2

Count of the phone conversations in which a point was made by the calling nurse.

The duration of the observed telephone calls ranged between a few seconds to about 1 min across all measurement points. The number of the 14 items reported varied over time: Before the course, on average of 4.5 items (range 2–10) were reported, 3 months after the course 9.1 items (range 6–12), and 6 months after the course 8.6 items (range 4–13). The use of the ADAPT increased from 20% pre-course to 90% 3 and 6 months post-course (Figure 2). In total, 9 items were significantly improved after the course compared to the pre-course observations (Table 4).

Table 4

Results from the Chi Square Tests across all three measurement points (pre. 3 months post and 6 months post) per tool item (n = 20 for each measurement point).

Item	Count mentioned before (n = 20)	Count mentioned after (n = 20)	Chi Square Value	df	P (two–sided)
∗Name of the caller	16	20	8.571	2	.014
∗Profession of the caller	0	4	8.237	2	.016
Location from were is call made	18	20	4.138	2	.126
∗Referral (self admission, general practitioner, ambulance, other hospital department)	2	14	17.753	2	.000
∗Patient's symtoms	7	16	13.650	2	.001
Patient's diagnosis	3	7	4.313	2	.116
∗Reason for the call	13	19	12.404	2	.002
∗Social security number of the patient	0	1	10.027	2	.007
Age of the patient	2	2	4.444	2	.108
Name of the patient	13	18	4.375	2	.112
∗Triage category of the patient	1	0	18.572	4	.001
Relevant vital parameters of the patient	2	7	4.261	2	.119
∗Use of ADAPT during call	4	18	29.400	2	.000
∗Agreement on plan	8	18	19.733	2	.000

Significant on the 0.05 level.

There was a similar pattern of increase and decrease of reporting of the three items that were not reported at all in the pre-measurement. The social security number of the patient was not mentioned at all in the pre-measurements, reported in 25% of the cases after 3 months and in 5% of the cases after 6 months. A similar pattern holds for mentioning the profession of the caller (pre: 0%, 3 months post: 37%, 6 months post: 20%) and age of patient (pre: 0%, 3 months post: 20%, 6 months post: 10%). In summary we were able to demonstrate changes in clinical behavior in the workplace, even six months after the course.

Table 5 provides illustrative examples of telephone conversations.

Results from the attitude questionnaire evaluation

The participants indicated that they were very satisfied with the course, with 81 out of 83 participants rating the course as good or very good. All participants had a “good” or “very good” overall impression of the course. In the open text section of the evaluation questionnaire, participants appreciated the initiative to establish the course, which gave them the possibility to discuss across professions. As negative points, participants mentioned that some parts of the course were too “pedagogical” and that the information provided before the course could have been improved. One participant felt pressured to take part in the course.

Discussion

We developed and conducted an interprofessional team training course to improve phone-based conversations between nurses and physicians. The evaluation of the intervention is based on analyzing the behavior in clinical practice of nurses calling physicians before and after the intervention and surveying participants' attitudes towards the course. The intervention comprised comprising full-scale manikin-based simulations, workshops, mini-lectures, and discussions.

Within the small sample investigated, our results indicate transfer of learning from an interprofessional educational intervention to clinical practice. After the intervention, the nurses provided more information about the patient to the physician. The results show that even 6 months after training and with some of the originally trained people not present in the department any more there were still a more complete telephone conversations than there were during baseline. We assume that this positive effect is connected to at least two features of our project. Firstly, it was possible to train a substantial percentage of the nursing staff working in the ED and close to one third of the involved physician staff in a rather short period of time. Secondly, we could keep up an intense dialogue with the leadership of the ED throughout the project and we assume that this helped anchoring the learning points of the course in clinical practice.

The primary intention of the intervention was to train the nurses and physicians to jointly optimize the use of the available and relevant patient information during phone-based conversations. After the course more items were reported during such conversations, especially: history of the patient, where the patient comes from, reason for the call, triage category of the patient, use of ADPAT, and agreement on a plan for further treatment. This increase could indicate that the course actually did play a role in this change of practice. We did, however, not have the resources to investigate the link between these changes of behavior and the clinical outcome in terms of time to treatment, quality of treatment or effect of treatment. We believe that the intense work with the ADAPT tool and the process in which it was used during the phases of the project helped positively changing its perception by the clinicians. The close dialogue with stakeholders on the different organizational levels and our frequent presence in the actual setting allowed us to draw multiple connections between the educational intervention and the actual workplace. An example of anecdotal feedback from a former course participant illustrates a key issue of the project: “We had really good communication because I called and gave the important information. There I really felt like a team for the patient across departments”.

In three items of the tool we saw an increase from 0% use before the course to a medium percentage of use after 3 months and a substantial decrease after 6 months (social security number and age of patient as well as the profession of the caller). In these three items, there was a substantial drop between the 3 and 6 months post measurements, while the other items stayed much more stable over time. We speculate whether these items were pieces of information that were not seen as relevant by those involved (we do not assess here, whether that is a good thing or not). For such items, one might see an increased use directly after a course, but they might not become an accepted part of one's professional tasks and thus vanish quickly after a course [64]. Training might be able to “force” people to report information or use tools, but when working in their own rhythm and setting again, certain items and tools blend into the background and might be neglected on the action level. Analyzing the patterns of omissions and errors in the telephone conversations can help to provide a clearer picture about the perceived relevance of information items mentioned or not. Which items are accepted as relevant and thus used? Which items are not used? Should those items be deleted from protocols or should an intervention aim at changing the practice by clinical staff around these information items?

Different levels (macro, meso, and micro) of factors that support or hinder the implementation of interprofessional education and the transfer of learning results were described [65]. On the macro level there seems to be conflicting tendencies in western societies of higher specialization on the one hand and the emphasize to work and learn together on the other hand [66,67,68,69,70,71]. Also, the settings of interprofessional education are diversifying [72]. On the meso level the leadership of the department verbalized strongly their support for the project and acted accordingly. However, on the micro-level, it seems as if the participation of especially the physicians was impaired by conflicting priorities and demands of daily routines.

Despite our focus to work with all stakeholders involved, we focused the needs analysis on the leadership of the ED and the nurses working there. Based on the scope of the project, many of the known and assumed influences [73] on changes in practice could not be accounted for as much as we would have wished. We did get some input from the physicians, but it was difficult to involve them and/or the heads of their departments in a more systematic way. One might say that we included the individual perspectives of physicians from different specialties, but could not include the organizational perspective of the participating departments. One of the consequences was different levels of buy-in into the project in terms of participation. The ED, having initiated the project, made the course mandatory for the nurses working there. But for the physicians from the other departments participation was not mandatory. The take home message here is to carefully review the relevant stakeholder groups involved in a project and to potentially challenge the scope of the project as defined by the entity asking for it. Such a scope might be too narrow, leading to challenges in sustainability of the project results. The stakeholders, who are not considered in the project can still make or break its success on an even longer time scale [74].

We had high-level leadership back-up and that might have helped that we could find behavioral effects still 6 months after the intervention in clinical practice, even if not all observed in six month later participated in the training. Explaining, how the training effect carried over to those, who did not participate in the training is beyond the scope of our data and study. It is also beyond the scope of our project to investigate Kirkpatrick level 4 [62], for example, staff turnover. Due to other organizational priorities, unfortunately, the project was not funded after its closing.

Limitations

Our study has limitations. We worked with a small sample and an un-blinded observer, who was also leading the development and implementation of the intervention. We might therefore overestimate the positive effects of the intervention. We cannot exclude biases in the measurement, yet we think that the way that we measured the completeness of the telephone conversation was reasonably straightforward to allow for relevant measurements be taken by an observer who understands what is going on in the department. LFP, who did the observations, is an intensive care nurse by training and has years of clinical experience as well as simulation-based expertise and had a leading role in an extensive handover project carried out in CAMES [26]. LFP made a judgment about the relevance of the information provided based on her own clinical experience.

Besides a content validation by the heads of the ED department, who are clinical experts, and some adjustments based on pilot testing, our observation tool was not subjected to a check of psychometric quality. Therefore, validity and reliability evidence for the results of the observations, using the tool is not known. The same holds for our evaluation questionnaire.

As with all observational studies, we cannot exclude the effect that participants might have performed better just because they were observed. No control group was included, hence we cannot exclude that other factors actually caused the changes in clinical behavior besides our course and other interventions. We did not keep formally track in the 6-months post measurement, who of the participants actually attended the course, but relied on LFPs memory of the course participants. During the 3-months post measurement, all observed people did participate in the course.

The illustrative examples in Table 4 could contain biases as only one side of the conversation was actually heard by LFP, who deduced questions and answers of the person being called.

We did not have the resources to measure some of the possible influencing factors for the telephone conversations. For example, it might be possible that the physicians called during the post intervention measurements were for some reason less busy than those during the pre intervention measurement. They might thus not only be more willing to come immediately to the ED, they might also have more time to listen and to ask questions. Further it might make a difference, if the physicians were trained themselves.

Possible ways forward

In the light of the still not conclusive findings about the effects that interprofessional education has on the different levels of evaluation [75], it would make sense to work towards building up the evidence for what works or not in terms of reactions, learning, application and system outcome. We also see benefits in focusing more on process levels. Therefore, it would be interesting to supplement an educational intervention as described in our paper with further interventions to make the effects “stick” and to better understand their impact. Brennan et al. [76] described a process in which they help professionals to understand interprofessional processes better by becoming observers in their own organization. Iedema et al. [77] facilitated in-depth interprofessional discussions by helping teams to establish video filming and reflective discussions of the filmed every day practice. Dadich described technology-based ideas, drawing on “crowd sourcing” concepts to make healthcare professionals co-researchers and reporting on effects on educational interventions [78]. Whatever direction is taken and method used for further investigations, we agree with Kitto et al. [79,80,81] emphasizing the need to concentrate on the value proposition that any tool offers. Does it actually improve interprofessional cooperation or does it influence merely the mechanics of interactions between people: having cooperation tools is not equal to having cooperation [79].

Conclusions

Improved interprofessional communication behavior in phone-based conversations about admitted patients between nurses and physicians was seen in a clinical setting after a comprehensive interprofessional intervention that was backed up by leadership of the participating departments. These improvements lasted at least 6 months after the intervention.

Declarations

Author contribution statement

L. Petersen: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

M. Madsen, D. Østergaard AND P. Dieckmann: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by the , Denmark.

Competing interest statement

The authors declare the following conflict of interests: P. Dieckmann holds a professorship with the University of Stavanger that is paid for by an unconditional grant from the Laerdal foundation to the University.

Additional information

No additional information is available for this paper.