Dynamic Communication Quantification Model for Measuring Information Management During Mass-Casualty Incident Simulations.

OBJECTIVE: To develop a new model to quantify information management dynamically and to identify factors that lead to information gaps.
BACKGROUND: Information management is a core task for emergency medical service (EMS) team leaders during the prehospital phase of a mass-casualty incident (MCI). Lessons learned from past MCIs indicate that poor information management can lead to increased mortality. Various instruments are used to evaluate information management during MCI training simulations, but the challenge of measuring and improving team leaders' abilities to manage information remains.
METHOD: The Dynamic Communication Quantification (DCQ) model was developed based on the knowledge representation typology. Using multi point-of-view synchronized video, the model quantifies and visualizes information management. It was applied to six MCI simulations between 2014 and 2019, to identify factors that led to information gaps, and compared with other evaluation methods.
RESULTS: Out of the three methods applied, only the DCQ model revealed two factors that led to information gaps: first, consolidation of numerous casualties from different areas, and second, tracking of casualty arrivals to the medical treatment area and departures from the MCI site.
CONCLUSION: The DCQ model allows information management to be objectively quantified. Thus, it reveals a new layer of knowledge, presenting information gaps during an MCI. Because the model is applicable to all MCI team leaders, it can make MCI simulations more effective. APPLICATION: This DCQ model quantifies information management dynamically during MCI training simulations.

Introduction

A mass-casualty incident (MCI) is an emergency incident where the number of casualties exceeds the medical resources that can be allocated to treat those casualties on-site (Einav et al., 2004; Rimstad & Braut, 2015, Rimstad & Sollid, 2015). These events may result from an accident, a sudden attack, or a disaster. Thus, the variability between MCIs is greater than their similarity. To overcome this variability, MCI protocols have been developed to provide guidance in managing these chaotic events (Bitan, 2017). Information management (also known as “communication” [Rimstad & Braut, 2015] or “gathering and analyzing information” [Wang et al., 2012]) is one of the core tasks of MCI commanders during the chaotic prehospital phase (Rimstad & Braut, 2015). Reports from past MCIs reveal that information gaps due to communication challenges such as excessive radio communications (Rimstad & Sollid, 2015) lead to increased mortality (Assa et al., 2009; Avitzour et al., 2004; Busby & Witucki-Brown, 2011; Raiter et al., 2008; Sloan, 2011; Turner et al., 2016), a prolonged evacuation duration (Assa et al., 2009), overflowing hospitals (Raiter et al., 2008; Turner et al., 2016), delays in hospital preparedness (Avitzour et al., 2004; Sloan, 2011), and a decrease in rescuers’ safety on-site (Turner et al., 2016).

Emergency medical services (EMS) personnel are trained to manage MCIs through simulation drills, with an emphasis on coping with large amounts of information (Perry et al., 2020). Since EMS performance can be measured using a variety of parameters, multiple instruments have been developed to measure the quality of information management during an MCI. These instruments include postsimulation Likert-scale questionnaires (Gordon et al., 2016; Roper et al., 2018), sets of performance indicators scored by experienced observers (Gryth et al., 2010; Rüter et al., 2004a, 2004b), and postsimulation surveys (Atack et al., 2009; Jorm et al., 2016; Zinan et al., 2015). Despite efforts to improve information management between EMS team members during MCI simulations, it remains a major challenge (Assa et al., 2009; Avitzour et al., 2004; Busby & Witucki-Brown, 2011; Raiter et al., 2008; Sloan, 2011; Turner et al., 2016).

The objectives of this study were twofold: (1) to develop a new model to measure the quality of information management during an MCI, and (2) to identify the factors that lead to information gaps. To achieve the first goal, the Dynamic Communication Quantification (DCQ) model was developed, based on the knowledge representation typology (Kozlowski & Chao, 2012). This typology is part of team cognition theory and macro-cognition theories, and aims to represent how knowledge emerges within team members and teams. The objective of this part was to harness the knowledge representation typology to reveal factors that lead to information gaps during MCI simulations. In addition, we compared the DCQ model to existing instruments to evaluate the advantages of each method. To achieve the second goal, data from six MCI simulations were analyzed by the DCQ model. The objective here was to utilize the DCQ model to present the information exchange gaps during the simulations.

This paper comprises five sections: following the introduction, the literature review section presents the knowledge representation typology, including the foundations and conceptual drivers of the typology. In the methods section, based on the knowledge representation typology, we introduce the DCQ model, which adapts the typology to the MCI domain. The results section demonstrates the DCQ implementation, showing how the model can be applied to an MCI simulation. To identify information gaps, we extend the use of the DCQ to six MCI simulations with the focus on the MCI commander. Then, we compare the DCQ model to two other instruments and evaluated the advantages of each of them. The conclusions for the study are presented in the final section.

Background

This section presents the knowledge representation typology and the derivative typology that leads to the DCQ model. To establish the theoretical basis of the DCQ model, we review the typology’s conceptual drivers. Then the seven metrics of the typology are reviewed, from knowledge emergence of an individual (i.e., team members) to knowledge emergence within the collective entities (i.e., groups and teams). We refer to information management as communication to meet the definitions of the team cognition and macro-cognition theories.

Foundation and Conceptual Drivers

Effective communication among team members is vitally important for high-complexity tasks (Salas, 1992). Communication within collective entities (i.e., groups and teams) is based on the knowledge each team member acquires and shares within the team. The theory of macro-cognition was developed to describe how team members form required and meaningful knowledge to solve a problem. As a part of the theory, the knowledge representation typology measures how knowledge can be dynamically measured, both at the individual level (e.g., for each team member) and at the team level (Kozlowski & Chao, 2012). Three conceptual drivers form the foundations of the typology: self-regulation, knowledge compilation, and knowledge emergence.

The first driver comprises self-regulation models, which describe individual learning as an iterative and cyclic process for achieving a goal (DeShon et al., 2004). The process begins by defining the goal and the strategy of the individual. At each iteration of the process, the individual’s performance is measured and compared to its goal. If a discrepancy is identified, the goal and strategy are revised. Self-regulation leads to individual knowledge compilation that improves performance toward achieving that goal. Thus, knowledge formation is compiled over several iterations of the self-regulation process.

The second conceptual driver is knowledge compilation, which forms over multiple iterations of the self-regulation process (Anderson, 1982; Kozlowski & Bell, 2007). Declarative knowledge (i.e., data) is initially collected and then forms the basis of procedural knowledge (i.e., categorized knowledge). As procedural knowledge accumulates, strategic knowledge is formed, leading to a specific problem-solving strategy. Procedural knowledge evolves into adaptive knowledge and a generalization of the problem, and solution strategies can be created.

In the MCI context, declarative knowledge can be exemplified in terms of the casualties at the MCI site. After each casualty is triaged (i.e., medically examined and classified), procedural knowledge emerges. As the amount of procedural knowledge grows (i.e., most of the casualties have been triaged and classified), a problem-solving strategy can be planned in the form of casualty evacuation procedures. Upon initiating the evacuation procedure, the EMS teams gain experience, and adaptive knowledge could develop into several solution strategies to improve the evacuation procedure, which might decrease mortality.

In contrast to self-regulation and knowledge compilation, which represent how knowledge is formed at the individual level, team knowledge, the third conceptual driver, is a multilevel process, which can be represented and measured in two different ways. Team knowledge emergence can be measured as either composition or compilation (Kozlowski & Klein, 2000). Composition is based on the assumption of isomorphism, where identical lower-level properties yield a higher-level property. Examples of this include shared mental models (Cannon-Bowers et al., 1990; Converse et al., 1993) and classical decision-making (Klein et al., 1993). In these models, team members can have the same knowledge (i.e., same lower-level properties). Their composition creates a higher-level property (i.e., all team members’ knowledge is synchronized) in the form of improved team performance. On the other hand, compilation is based on discontinuity, in which different lower-level properties yield a higher-level property. It can be seen in transactive memory (Wegner, 1987), naturalistic decision-making (Klein, 2008), and organizational learning with an emphasis on knowledge spirals (Nonaka, 1994). In these models, team members have different knowledge (i.e., different lower-level properties). The compilation combines all the different team members’ knowledge into 1 shared pool of knowledge, which forms the basis for decision-making.

The knowledge representation typology was designed to capture all three of these conceptual drivers, and is outlined in the next subsection.

Knowledge Representation Typology

Seven knowledge metrics form the knowledge representation typology (Kozlowski & Chao, 2012). Each team member has access to common and unique knowledge, and the sum represents the total pool of knowledge for solving a problem. The first metric is the individual knowledge metric that measures the amount of knowledge (common and unique) of each team member at a given time. The individual knowledge of each team member is combined in the second metric, the knowledge pool of the team, which represents the total collective knowledge of the team. Third, the knowledge configuration metric is an integral part of the knowledge pool and represents the knowledge that is held by one or more team members. The individual knowledge, knowledge pool, and knowledge configuration metrics address the amount of knowledge at a specific time. To represent the dynamic growth of knowledge, the fourth metric, knowledge acquisition, captures how knowledge is acquired across a timeline. The knowledge acquisition of each team member composes the fifth metric, knowledge variability, which captures how the knowledge of the collective grows over time. The six and seventh metrics, knowledge emergence within the team and knowledge emergence between teams, allow us to compare growth rates for the knowledge variability, knowledge pool, and knowledge configuration across different teams.

To achieve the goal of developing a new model to measure the quality of information management during an MCI, the typology requires adaptation to the MCI context. This adaptation includes the following phases: (1) review of the EMS team response procedures and team roles, (2) formalization of the DCQ model based on the knowledge representation typology, (3) review of the MCI simulation data for the study, and (4) a novel approach for data collection during the MCI that will serve as an input to the model. For the ease of representing the typology adaptation to the MCI context, we refer to data and knowledge as information (Kozlowski & Chao, 2012).

Methods

EMS Response to an MCI

The EMS response to an MCI aims to maximize the number of casualties saved, which represents the collective goal of the team (Figure 1). To achieve the team goal, and based on previous MCI operations, the EMS team members follow an MCI protocol that includes three phases: (1) evaluation and triage, (2) treatment and preparedness for evacuation, and (3) evacuation according to triage classifications (Assa et al., 2009; Avitzour et al., 2004; Blancher et al., 2018; Busby & Witucki-Brown, 2011; Ellis & Sorene, 2008; Garner et al., 2001; Peleg et al., 2003; Raiter et al., 2008; Shapira & Shemer, 2002; Sloan, 2011; Smith & Dowell, 2000). The evaluation goal is to count the number of casualties and to quickly assess injuries throughout the entire scene. Then, casualties are gathered into the treatment area, where each casualty receives a triage classification and emergency medical treatment. As more ambulances arrive at the scene, casualties are loaded into ambulances for evacuation to receive further medical treatment at hospitals.

Figure 1

EMS team roles for an MCI response. EMS = emergency medical service; MCI = mass-casualty incident.

The medical aspects of an MCI are managed by a commander, who is responsible for management of the entire scene. Two main team leaders support the commander: the medical leader, who is responsible for triage and lifesaving treatments (Peleg et al., 2003; Rimstad & Braut, 2015; Shapira & Shemer, 2002); and the evacuation coordinator, who is responsible for dispatching casualties from the scene to hospitals (Rimstad & Braut, 2015). The MCI commander is also responsible for communication with the dispatch center and with other emergency agencies at the scene (e.g., police, firefighters, etc.; Assa et al., 2009; Blancher et al., 2018; Busby & Witucki-Brown, 2011; Rimstad & Braut, 2015; Sloan, 2011).

DCQ Model

The DCQ model aims at identifying information gaps between the team leaders. To tune the model, we denote casualties at the MCI by the set and each casualty as , where is the triage classification for the ’th casualty. Each casualty, , has a set of information items, , relating to the evacuation procedure of the EMS: (1) the time of first triage (defined as ), (2) the time of arrival to the treatment area (defined as at), (3) the time of departure from the treatment area to the ambulances (defined as dt), and (4) the time of loading into an ambulance (defined as ua). For example, the set of information items for urgent casualty number 5 is = {03:00, 05:15, 06:45, 09:20}. Overall, the problem space is denoted as (i.e., the union of all information items for all the casualties). We note that all items of information of each specific type will be represented as . For example, all triage classification information items are represented as (i.e., the counts of all triage classification items for all the casualties on-site). Table 1 shows how information forms during the MCI simulation, and which information items are required by each team leader.

Table 1

Information Items, Formation, and Acquisition by EMS Team Leaders

Information Items	Information Formation	Information Acquisition, by Team Leader Role
Casualties at the scene and triage classifications	Collected in a quick assessment as EMS teams arrive at the scene	Medical leader: triages casualties at the scene or interacts with other EMS team membersCommander: verbally interacts with the medical leader or other EMS team members
Casualties arrival at the treatment areas	Casualties arrive at the treatment area on foot or by stretcher	Medical leader: interacts with casualties performs emergency treatments as needed asks other EMS team members to perform emergency treatments on a specific casualty counts the number of casualties in the treatment area prioritizes casualties for evacuation Commander: verbally interacts with the medical leader
Casualties departure from the treatment area	Casualties leave the treatment area on foot or by stretcher	Medical leader: conveys casualties from the treatment area.Evacuation coordinator: conveys casualties from the treatment area.Commander: interacts with the medical leader or evacuation coordinator
Casualty loading into an ambulance	Casualties are loaded into ambulances	Evacuation coordinator: loads casualties into an ambulance or verbally interacts with other EMS team members who have already loaded casualties into ambulances

Note. EMS = emergency medical service.

Individual and Team Information Acquisition and Exchange

For each team member, the individual information compilation (i.e., the fourth metric of the knowledge representation typology) is processed in four phases: (1) Information formation, (2) acquisition, (3) integration, and (4) exchange. These phases facilitate the tracking of every information item from the moment it forms until it is exchanged. Thus, if an information item forms but is not exchanged, these phases reveal a gap where this item was lost. In addition, we defined an integration period between information acquisition and exchange, to evaluate if there was a gap between them.

To identify information gaps, three continuous measurements were defined: two for information acquisition and one for information integration. Equations and formulation calculations are given in Appendix B for each of the different EMS team leaders. Information acquisition was measured by (1) the ratio of the number of information items that were acquired to the total number (Equations 1, 3, and 5), and (2) the time between information items being available and their acquisition, measured in seconds (Equations 2, 4, and 6). Information integration was measured by the difference between the number of items that were acquired and the number of items that were exchanged (i.e., reported by the end of the simulation).

For example, consider a case of two casualties, (i.e., the first indexed urgent casualty and the first indexed nonurgent casualty in the simulation), each with relevant information items, respectively, = {03:00, 05:15, 06:45, 09:20}, and = {05:00, 07:20, 10:30, 13:40}. If the medical leader acquired information regarding the triage classification of both casualties, the ratio would be calculated as (i.e., all the information items were acquired); if only one item of information was acquired, then . For the time calculation, only information items that were acquired will be summed, so for both casualties Table 2 summarizes the integration between the knowledge representation typology and the MCI context.

Table 2

Knowledge Representation Typology and Adaptation to the MCI Context

Information Metrics	Brief Description	Example		MCI Context Adaptation
Individual information	The proportion of the total pool of possible information possessed by each team member separately		The amount of information individuals i, j, and k each possess within the problem space	The number of information items each of the team leaders acquired: 1. MCI commander – ∣C∣,tc for each cip∈C , and ∣c^p∣∈p  2. Medical leader – ∣C∣ , tc for each cip∈C , at and dt for each cip∈C  3. Evacuation coordinator – dt and ua for each cip∈C
Information pool	The proportion of the total pool of possible information possessed by the team collectively		The proportion of the total information among individual team members accounting for overlap	The proportion of the total items of information among the team leaders
Information configuration	The proportion of the total pool shared in common by team members and the pattern of unique knowledge held across individuals		Understanding what is common and what is unique information among team members	Unique and common information items (common information items are types of information item relevant to more than one team member)
Information acquisition	The rate of information compiled by each team member over time		How fast an individual learns (expending the circle above)	Individual information compilation, and how fast a team member acquires information
Information variability	Within team variability in the rates of information acquisition		Different rates of information acquisition can affect a team’s learning	How different rates of information acquisition by the team leaders affect team learning
Information emergence (within team)	The rates of growth for information pool and information configuration			Information emerges within the team over time
Information emergence (between teams)	Comparing growth rates for information variability, information pool, and information configuration across teams			Comparing the quality of information management of MCI commanding teams in different MCI simulations

Note. Modified from Salas et al. (2013).

MCI = mass-casualty incident.

MCI Simulations

To demonstrate how the DCQ model can better evaluate information management, we applied it to MCI simulations conducted as part of a paramedic training course. Participants in the simulation were chosen from a homogenous group of second-year paramedic students at a midsized university. The majority of the students are medics at a city EMS with 2 or 3 years of military experience.

The training of second-year students included one simulation. The scenario exercised in the simulation included a missile attack on an urban area, with approximately 20 victims that were simulated by volunteers (for the exact number of victims at each simulation see sections 4.1 and 4.2). Prior to the simulation, each volunteer received an injury description tag (i.e., a script) and was instructed to act according to it. The victims were first-year students, scattered across different locations at the MCI site and instructed to remain in their locations unless the participants instructed them otherwise.

The city EMS participated in the simulation and provided the required resources, which included ambulances and ambulance drivers, EMS medical personnel, stretchers, and communication equipment. The city EMS personnel that participated in the simulation were instructed to follow the participants’ instructions. To simulate the evacuation process of casualties to hospitals, a unique dispatch station was created within the city’s EMS dispatch station, which included an MCI dispatcher to communicate with the simulation participants and EMS city personnel.

This research complied with the American Psychological Association Code of Ethics and was approved by the Institutional Review Board at the Department of Industrial Engineering and Management at Ben-Gurion University. Informed consent was obtained from each participant

Data Collection and Statistical Analysis

This section will present the data collection and statistical analysis of the three parts of the study. For the first part, the DCQ model application, the multi-point-of-view synchronized video will be presented, followed by data and measurements applied. For the second part, the MCI commanders’ information management, the data collected from six MCI simulations and measurements used to identify information gaps will be presented. For the third part, instrument evaluation, two additional instruments for evaluating information management will be reviewed, with emphasis on questions and performance indicators used for the evaluation.

DCQ Model Application

To support the DCQ analysis, we developed a novel approach for data collection that captures information formation, acquisition, and exchange during an MCI simulation. Capturing information acquisition and exchange between the three EMS team leaders required four video cameras. We attached one camera to the MCI commander’s vest to capture visual and vocal interactions, and to allow the commander to move freely around the MCI site. Two cameras were operated by video photographers near the treatment area and the evacuation area, managed by the medical leader and the coordinator, respectively. This allowed not only the capture of verbal interactions but also the activities in these two areas. We attached the fourth camera to a drone that captured an aerial view of the entire scene. To minimize observer-expectancy effects (Burghardt et al., 2012) while maximizing data collection, we instructed the photographers to keep a distance from the subjects while filming, and to remain in one position.

We synchronized the four video sources and combined them into one multi-point-of-view video that was tagged interdependently by two observers (O.P. and T.K.) using OBSERVER XT software. A third observer (Y.B.) resolved any disagreement between the other two. This data collection and analysis approach was presented at the International Symposium on Human Factors and Ergonomics in Health Care 2019 (Perry et al., 2019).

In this simulation, casualties included five that were urgent, 11 nonurgent, and three resulting in death from their injuries. Continuous data are presented as means with 95% confidence intervals (CIs), and information acquisition versus information formation as percentages. The databases for the calculations and formulation are depicted in Appendices A and B, respectively. SPSS V17 and Microsoft Excel 2016 were used for statistical analysis. The model measured three team leaders.

MCI Commanders’ Information Management

Six MCI simulations between the years 2014 and 2019 were analyzed by the DCQ model. Since our goal was to identify information gaps, four measurements were included: (1) the difference between the actual number of casualties at the scene and the number of casualties that were reported to the commander (marked as (2) the difference between the actual number of casualties and the number of casualties that were reported as evacuated by the commander at the end of the simulation (marked as ), (3) the time at which the commander started to acquire information about the overall number of casualties (marked as ), and (4) the time at which the commander started to acquire information about the evacuation of casualties (marked as ). For this analysis, we focused only on casualties and not on fatalities that were determined upon arrival to the MCI site.

For both and , the results are reported as deviations from the overall number of casualties, including factors that led to incorrect casualty counting. If no deviations were found, and do not appear in the graphs. For both and , the time at which the commander started acquiring information is reported in minutes from the start of the simulation.

Instrument Evaluation

We chose three instruments for the evaluation: (1) the self-assessment teamwork tool for students (SATTS; Gordon et al., 2016; Roper et al., 2018) as a postsimulation Likert-scale questionnaire, (2) a list of performance indicators scored by an experienced observer (Gryth et al., 2010; Rüter et al., 2004a, 2004b), and (3) the DCQ that was developed by the authors.

The SATTS aims to measure the quality of information management, coordination, and teamwork of all the EMS teams from the participant’s perspective. It comprises 14 questions; of these, four are relevant to the quality of information management. Unlike the SATTS, which includes all the students who participated in the simulation, the performance indicators place the focus on the MCI commander. During the simulation, an experienced observer accompanies the MCI commander and ranks the performance by a score ranging from 0 to 2 (i.e., 0–incorrect decision, 1–partly correct, 2–correct). Out of 12 performance indicators, four are relevant to the ability of the commander to manage information and are focused on timely reports of the number of casualties and injury severity.

Three EMS team leaders manned the role of the MCI commander according to the drill manager’s instructions. Since the evaluation of the three instruments is relevant to information management, only questions and performance indicators that included information management were analyzed (SATTS: ; performance indicators: – see appendix C for the questions and performance indicators). Results for the SATTS questionnaire and the performance indicators are represented as means and standard deviations. The results for the DCQ are represented as percentages.

Results

This part demonstrates how the DCQ model can be used to quantify information management. The model was applied to data collected during a 2018 MCI simulation that included 22 participants and 19 casualties distributed between 11 nonurgent, five urgent, and three fatalities.

Medical leader: casualties and triage classifications

Figure 2 represents the medical leader’s information acquisition versus information formation at the beginning of the simulation. The medical leader arrived after 08:00 min from the simulation initiation and started to perform triage and life-saving treatments at casualties dispersed around the site. Seven casualties were triaged between 09:46 and 11:18 (two urgent, three nonurgent, and two declared dead). At 12:41, the medical leader received information regarding another eight nonurgent casualties that another EMS team member triaged. Lastly, while scanning the site for the second time (between 18:10 and 18:42), the medical leader triaged four more casualties (three urgent and one declared dead). Overall, the medical leader managed to acquire 100% (i.e., ) of the pertinent information items that formed (Eq. 1). The mean time (Eq. 2) between information formation from the first ambulance arrival to information acquisition was s (95% CI [375.09, 542.69]). However, although the medical leader acquired 100% of information items (i.e., acquired information regarding all 19 casualties), the last report at the end of the simulation included 15 casualties. This result indicates that information integration (i.e., the sum of all information items that were acquired) led the medical leader to understand that there were 15 casualties at the site when there were actually 19, a four information-item difference.

Figure 2

Information acquisition by the medical leader during the simulation timeline. The S-shaped lines represent information acquisition, and the black horizontal dotted lines, the actual number of casualties. Both are divided according to the casualties’ triage classifications.

Medical leader: casualties’ arrival to the treatment area

The medical leader started to manage the treatment area from 13:10 min from the simulation initiation until the end of the simulation. There were 23 casualty arrivals (i.e., 23 items formed) to the treatment area instead of 19. Four casualties were ordered to leave the treatment area and get into an ambulance. However, the ambulance that they were instructed to get into was fully occupied. Thus, the casualties returned to the treatment area, increasing the 19 arrivals to 23 arrivals. Out of the 23 arrivals, the medical leader managed to acquire 34.78% of the information items that formed (i.e., ), including five urgent casualties and three dead (Eq. 3) between 16:26 and 28:40. The mean time (Eq. 4) between information formation and acquisition was s (95% CI [5.86, 80.63]) for urgent casualties.

Evacuation coordinator: casualties’ loading into an ambulance

The evacuation coordinator started to manage the loading area at 10:20 min from the simulation initiation. Overall, 16 casualties were evacuated, the first one at 12:35 and the last one at 25:40. The evacuation coordinator acquired 68.75% (i.e., ) of the information items formed (Eq. 5) including three urgent casualties and eight nonurgent casualties. Information regarding five more casualties was not acquired since the evacuation coordinator was busy loading other casualties. The mean time (Eq. 6) between information formation and acquisition was s (95% CI [0, 45.25]).

Commander: overall casualty integration

Figure 3 represents the information exchange between the medical leader and the commander regarding the overall casualties at the scene, over the simulation timeline. The commander arrived at 05:30 min from the simulation initiation and gathered information regarding the number of casualties. There were four main reports regarding the number of casualties. At 07:19, the commander reported to the EMS dispatch about eight casualties (five urgent and three nonurgent). The second report to the EMS dispatch occurred at 08:28 and included 21 casualties without triage classification. At 10:07, the medical leader and the commander counted 20 casualties (12 nonurgent, eight urgent). Finally, at 15:15, the medical leader reported 15 casualties (12 nonurgent, three urgent). Reports on another three casualties that were declared dead arrived between 18:00 and 25:00. In terms of information integration, the distance between the number of reported casualties and overall casualties was four. The overall number of items that were integrated (i.e., urgent, nonurgent, and dead) does not sum to the overall number of items at the scene (i.e., overall casualties), which could be caused by a counting error.

Figure 3

Information exchange and integration by the commander during the simulation timeline. The S-shaped lines represent information acquisition, and the black horizontal dotted lines, the actual number of casualties. Both are divided according to the casualties’ triage classifications.

Our goal in this part was to use the DCQ model to identify information gaps that led to an incorrect estimation of the number of casualties and the number of casualties that were evacuated. In addition, we wanted to evaluate which commanders were more persistent in acquiring information items. The analysis results are reported in Table 3 and illustrated in Figure 4.

Table 3

Analysis Results of MCI Simulation 2014–2019

Simulation and Casualties	Δ_1	Δ_2	Δ_3	Δ_4
Simulation A(upper right)19 Casualties	Incorrect counting due to the merging of two different areas of casualties (deviations of two and four casualties).	Incorrect counting of casualties that were evacuated – lost track of arrival and departure of casualties from the treatment area (deviation of seven casualties).	5 min	13 min
Simulation B(upper left)18 Casualties	Correct counting of the overall casualties.	Correct counting of evacuated casualties.	21 min	21 min
Simulation C(middle right)17 Casualties	Correct counting of the overall casualties.	Correct counting of evacuated casualties.	7 min	17 min
Simulation D(middle left)22 Casualties	Correct counting of the overall casualties.	Correct counting of evacuated casualties.	2 min	3 min
Simulation E(bottom right)16 Casualties	Error in counting due to the merging of two different areas of casualties (deviations of two and four casualties).	Incorrect counting of casualties that were evacuated – lost track of arrival and departure of casualties from the treatment area (deviation of six casualties).	8 min	15 min
Simulation F(bottom left)24 Casualties	Error in counting due to the merging of two different areas of casualties (deviation of 10 casualties).	Incorrect counting of casualties that were evacuated – lost track of arrival and departure of casualties from the treatment area (deviation of nine casualties).	4 min	15 min

Note. MCI = mass-casualty incident

Figure 4

Information exchange during MCI simulation: A–2014, B–2015, C–2016, D–2017, E–2018, F–2019. MCI = mass-casualty incident.

The mean and standard deviation (Figure 5) of the answers relevant to information management were ( ) , , , . Six experienced observers evaluated the three MCI commanders; thus, each performance indicator (Figure 6) was scored 18 times ( ). The results of scoring the indicators that were relevant to the information management were , , , . Results of applying the DCQ (Figure 7) revealed that while there were 24 casualties in the MCI simulation, the EMS team reported 34 at the end of the simulation (41% more than the actual number).

Figure 5

Results of the SATTS. SATTS = self-assessment teamwork tool for students.

Figure 6

Results of the performance indicators.

Figure 7

Results of the DCQ model. DCQ = Dynamic Communication Quantification.

Discussion

We have presented the Dynamic Communication Quantification (DCQ) model, a new model to quantify information during MCI simulations, which reveals a new layer of knowledge regarding the challenges EMS team leaders face while managing information. Results from the first part of the study demonstrate the advantages of the model in providing an in-depth view of the ability of every member of the EMS commanding team to acquire, integrate, and share information. The results reveal that while EMS team leaders managed to acquire large amounts of information, information integration was the main challenge during the simulation. Information integration led to an incorrect understanding of the number of casualties that were at the scene and the number of casualties that were evacuated. This demonstration focuses on six metrics of the knowledge representation typology (Kozlowski & Chao, 2012). It provides an in-depth perspective of individual (i.e., team member) information acquisition, how information accumulates over time, and how it emerges within the team.

The second part of this study represents metric number seven in the knowledge representation typology (information emerges between teams), and focuses on the evaluation of different commanding teams during different simulations. Extending the analysis to six MCI simulations allowed us to identify factors that lead to information gaps, specifically mismatches regarding the number of overall casualties and evacuated casualties. Results from this study indicate that MCI commanders cope with two main challenges while managing information. The first is assessing the overall number of casualties, since the MCI site is initially divided into several parts. Therefore, the commander is required to consolidate many items of information within a short time period. The second factor is due to the arrival and departure of casualties from the treatment area over a short time, which creates a challenge to track the number of evacuated casualties. Thus, our hypothesis about the ability of the knowledge representation typology to dynamically quantify information management was confirmed.

The model accounts for error in counting and misinformation by following the verbal interaction of the commander with other entities. For example, suppose the commander acquired a number of information items by verbal interactions with other team members, and reported a different number to the EMS dispatch. In this case, we can assume that the error was made due to a problem in information integration. The model will represent this error in the differing numbers acquired by the commander and reported by her to the EMS dispatch.

A comparison of the DCQ model with two other instruments that were presented in part three of the study reveals that, while the DCQ provides an objective point of view about the ability of the commanding team to manage information, other instruments yield only partial conclusions. The SATTS (Gordon et al., 2016; Roper et al., 2018) results might lead to the conclusion that the ability of the EMS team to manage information was relatively high, but results from the performance indicator (Gryth et al., 2010; Rüter et al., 2004a, 2004b) do now provide a clear conclusion about the ability of the team to manage information. These results could be explained by examining three factors: who the team members are whose performance is being evaluated, the effect of measurement biases, and the dynamic measurement of information management.

The SATTS does not focus on the commanding team, but rather on all participants in the simulation, which does not allow us to evaluate how those with the main roles in the team managed information. In contrast to the SATTS, the performance indicators focused only on the commander, while the response in most cases is a team effort (Peleg et al., 2003; Rimstad & Braut, 2015; Shapira & Shemer, 2002). Thus, both instruments provide a partial view (i.e., only the commander) or a wide point of view (i.e., all participants), which makes it difficult to assess the ability of the team to manage information. In terms of measurement biases, the SATTS captures nonobjective data (participants are scoring their own performance). Thus, it might be biased (Hurd & Kapteyn, 1999), especially when sample sizes are small (Eysenck & Hartley, 2014). For the performance indicators, a high variance in scores between observers might lead to a different conclusion, as can be seen in the study results. The DCQ overcomes these factors since it focuses on the commanding team while minimizing biases such as observer-expectancy effects (Burghardt et al., 2012) and capturing nonobjective data (Hurd & Kapteyn, 1999). Lastly, both the SATTS and the performance indicators evaluate team leaders’ abilities to manage information at the end of the simulation or at specific time intervals. Thus, they reveal only part of the MCI simulation, whereas the DCQ model measures information management dynamically throughout the simulation.

The DCQ meets the definition of both team cognition and macro-cognition for measuring information emergence within a team (Kozlowski & Chao, 2012) and could comprise the performance evaluation for medical command and control teams from previous studies (Gryth et al., 2010; Rüter et al., 2004a, 2004b). In addition, the multi point-of-view synchronized video can be used as a debriefing tool at the end of a simulation. Thus, trainees, instructors, and researchers could identify challenges and develop new tools and methods that could improve EMS service performance (Bitan, 2017). Information management is a core task of MCI commanders and remains one of the main challenges. Decision-making that prioritizes casualty treatment and evacuation based on inaccurate information could decrease casualties’ probabilities of survival. The DCQ model application allows us to reveal information gaps and the factors that led to them. Healthcare practitioners could use the model and the information it provides to train MCI commanders, emphasizing information management, the factors that led to inaccurate information, and how to avoid such inaccuracies.

Like other measurement instruments and models, the DCQ also has some limitations. While it can objectively answer the question “What happened?” in terms of information management, “why” an event happened remains unanswered. Thus, we suggest conducting interviews with EMS team leaders to gain a better understanding of the causes behind the results. The second limitation relates to the EMS team leaders’ roles. In addition to the various roles in this study, other studies have reported additional roles such as two commanders (Rimstad & Sollid, 2015) or an air evacuation coordinator (Assa et al., 2009; Rimstad & Braut, 2015). Thus, a change in the team leaders’ roles will need to be made to adapt the instrument to different team members. The third limitation relates to the method by which EMS team members acquire information. This study measured information exchange as verbal interactions between EMS team members. However, in a few specific cases, EMS team leaders operated with visual data (e.g., hand signaling) before verbal interaction. Thus, the assumption that EMS team leaders acquire visual data as knowledge should be taken into consideration and can be reviewed in the postsimulation interviews. Lastly, the participants in the simulation were second-year paramedic students. Different participants (e.g., paramedics with some years of experience) could lead to different performance in terms of information management.

Conclusion

In conclusion, the DCQ model provides access to a new objective layer of knowledge to aid EMS team leaders in information management during MCI simulations. An integral part of the model is the novel data collection approach, which integrates all the levels of data (from the individual to the team level), provides data for in-depth analysis, and serves as an input to the model. Findings from the current study should encourage the use of the DCQ in MCI simulations during routine training programs and qualification courses for doctors, paramedics, and nurses. To support training, the instrument should be applied to emphasize the importance of avoiding using inaccurate information and how to identify and cope with factors that lead to it. In addition, for future research, we suggest further exploring the DCQ in terms of teamwork and coordination, which serve as complementary constructs to the information management (also known as communication) construct according to the team cognition theory. For application, we suggest exploring the DCQ model in other environments in which accurate information is key for decision-making, such as command and control centers and in hospital emergency units.

Measuring information during MCI simulation is important to identify information gaps. Inaccurate information may lead to wrong decisions about prioritizing casualties for treatment and evacuation, and could decrease casualties’ probabilities of survival.

The DCQ model reveals that consolidating the number of casualties and tracking casualty evacuation were the two main factors that led to information gaps during MCI simulations.

The evaluation of different instruments for measuring information management reveals that the choice of instruments could lead to differing results.

Casualty Ci and	Information type	tc – Medical leader	at – Medical leader	dt – Medical leader	ua – Evacuation coordinator
1. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:07:31	0:22:58	0:24:05
	Acquisition	0:10:51	NA	NA	NA
2. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:16:40	0:22:58	0:24:05
	Acquisition	0:09:58	NA	NA	NA
3. Dead	Formation	First ambulance arrival – 0:05:30	0:16:23	0:16:24	NA – Have not been loaded
	Acquisition	0:10:52	0:16:26	0:16:26
4. Dead	Formation	First ambulance arrival – 0:05:30	0:18:23	NA – Left in the treatment area	NA – Left in the treatment area
	Acquisition	0:09:46	0:18:45
5. Urgent	Formation	First ambulance arrival – 0:05:30	0:17:32	0:20:45	0:25:20
	Acquisition	0:11:11	0:18:46	0:20:50	NA
6. Urgent	Formation	First ambulance arrival – 0:05:30	0:16:03	0:19:15	0:20:03
	Acquisition	0:11:13	0:17:00	0:19:25	0:20:05
7. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:08:48	0:14:15	0:14:40
	Acquisition	0:11:18	NA	NA	0:15:08
8. Dead	Formation	First ambulance arrival – 0:05:30	0:27:50	NA – Left in the treatment area	NA – Left in the treatment area
	Acquisition	0:18:10	0:28:40
9. Urgent	Formation	First ambulance arrival – 0:05:30	0:17:24	0:19:51	0:23:30
	Acquisition	0:18:42	0:17:33	0:19:56	0:23:37
10. Urgent	Formation	First ambulance arrival – 0:05:30	0:21:50	0:21:52	0:22:11
	Acquisition	0:18:10	0:21:50	0:21:52	0:23:50
11. Urgent	Formation	First ambulance arrival – 0:05:30	0:17:26	0:20:51	0:22:26
	Acquisition	0:18:10	0:19:37	0:20:55	NA
12. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:08:02	0:12:20	0:12:35
	Acquisition	0:12:41	NA	NA	0:12:35
13. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:07:10	0:14:02	0:14:02
	Acquisition	0:12:41	NA	NA	0:15:19
14. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:07:14	0:14:15	0:14:40
	Acquisition	0:12:41	NA	NA	0:15:08
15. Nonurgent	Formation	First ambulance arrival – 0:05:30	First Arrival – 0:07:19Re-arrival (1) – 0:22:20Re-arrival (2) – 0:24:30	First Departure – 0:19:50Second Departure – 0:23:18Third Departure – 0:25:14	0:25:30
	Acquisition	0:12:41	NA – for all three arrivals	NA – for all three departures	0:25:30
16. Nonurgent	Formation	First ambulance arrival – 0:05:30	First Arrival – 0:13:17Re-arrival (1) – 0:22:20Re-arrival (2) – 0:24:30	First Departure – 0:19:50Second Departure – 0:23:18Third Departure – 0:25:14	0:25:30
	Acquisition	0:12:41	NA – for all three arrivals	NA – for all three departures	0:25:30
17. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:07:24	0:20:25	0:20:27
	Acquisition	0:12:41	NA	NA	0:20:27
18. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:07:37	0:20:25	0:20:27
	Acquisition	0:12:41	NA	NA	0:20:27
19. Nonurgent	Formation	First ambulance arrival – 0:05:30	0:14:18	0:24:05	0:25:40
	Acquisition	0:12:41	NA	NA	NA

Measurement		Equation	Formulation—Rows in Appendix A
Medical Leader: Overall Casualty Acquisition and Integration	Accuracy	Equation 1	r_tc=⋃k∈(u,nu,d)tcip⋃k∈(u,nu,d)TCip,∀tcip≠NA
	Mean Duration	Equation 2	tc¯=Σ_k∈(u,nu,d)Σ_itc_i⋃k∈(u,nu,d)tcip,∀tc_i≠NA
Medical Leader: Casualty Arrivals to Treatment Areas	Accuracy	Equation 3	r_at=⋃k∈(u,nu,d)atip⋃k∈(u,nu,d)ATip,∀atip≠NA
	Mean Duration	Equation 4	at¯=Σ_k∈(u,nu,d)Σ_iat_i⋃k∈(u,nu,d)ATip,∀at_i≠NA
Evacuation Coordinator: Loading Casualties into Ambulances	Accuracy	Equation 5	r_ua=⋃k∈(u,nu,d)uaip⋃k∈(u,nu,d)UAip,∀uaip≠NA
	Mean Duration	Equation 6	ua¯=Σ_k∈(u,nu,d)Σ_iua_i⋃k∈(u,nu,d)UAip,∀ua_i≠NA

Qi	Poor			Average			Excellent	Description
Q 4—Instructions and verbal communications were directed.	1	2	3	4	5	6	7	Excellent: Use of person’s name (not just directed nonverbally)Average: Generally it was clear who the communication was directed at, names not used all the time, the occasional use of “someone” or undirected communication.Poor: Use of “someone”, directed to the room, no nonverbal indication of who the communication was meant for.
Q 8— Situational information was verbalized.	1	2	3	4	5	6	7	Excellent: Team members verbalized patient information, situation information, monitor data or other information.Average: All team members verbalized situational information occasionally. Occasional essential opportunities missed. Distracting/unnecessary communication may occur occasionally.Poor: Team members didn’t say anything when there was a situation which may have needed action. E.g. no other team member saying the BP is low.
Q 9— Instructions were explicit.	1	2	3	4	5	6	7	Excellent: Instructions were clearly audible, easy to understand, and there was sufficient detail (e.g., dose/ dilution of drug) to avoid any potential confusion.Average: Most instructions were clear, but with the occasional vague instructions.Poor: Unclear, inaudible, or imprecise instructions
Q 11— Possible future developments or requirements were communicated clearly.	1	2	3	4	5	6	7	Excellent: Following the initial plan, or following a change in patient status, anticipated events and what might be required were verbalized clearly.Average: There was some anticipation of future developments but with insufficient detail for team members to know what may be required of them.Poor: There was no information communicated about possible developments to prepare for.

Activity	Objective/goal to Be Achieved (All Time Are From Arrival on Site)
PI2—First report to dispatch center	Within 2 min
PI3—Correct content of first report	Missile attack
PI9 — Establishing continuous communication with dispatch center	Within 5 min
PI11 — Second report to dispatch center	Within 10 min
PI12 — Correct content of Second report	Verifying first report and indicating when first patient transport can take place