The QUality of Interhospital Transportation in the Euregion Meuse-Rhine (QUIT-EMR) score: a cross-validation study.

OBJECTIVES: Interhospital transports of critically ill patients are high-risk medical interventions. Well-established parameters to quantify the quality of transports are currently lacking. We aimed to develop and cross-validate a score for interhospital transports.
SETTING: An expert panel developed a score for interhospital transport by a Mobile Intensive Care Unit (MICU), the QUality of Interhospital Transportation in the Euregion Meuse-Rhine (QUIT-EMR) score. The QUIT-EMR score is an overall sum score that includes component scores of monitoring and intervention variables of the neurological (proxy for airway patency), respiratory and circulatory organ systems, ranging from -12 to +12. A score of 0 or higher defines an adequate transport. The QUIT-EMR score was tested to help to quantify the quality of transport. PARTICIPANTS: One hundred adult patients were randomly included and the transport charts were independently reviewed and classified as adequate or inadequate by four transport experts (ie, anaesthetists/intensivists). OUTCOME MEASURES: Subsequently, the level of agreement between the QUIT-EMR score and expert classification was calculated using Gwet's AC1.
RESULTS: From April 2012 to May 2014, a total of 100 MICU transports were studied. The median (IQR) QUIT-EMR score was 1 (0-2). Experts classified six transports as inadequate. The percentage agreement between the QUIT-EMR score and experts' classification for adequate/inadequate transport ranged from 84% to 92% (Gwet's AC10.81-0.91). The interobserver agreement between experts was 87% to 94% (Gwet's AC10.89-0.98).
CONCLUSION: The QUIT-EMR score is a novel validated tool to score MICU transportation adequacy in future studies contributing to quality control and improvement. TRIAL REGISTRATION NUMBER: NTR 4937. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The QUality of Interhospital Transportation in the Euregion Meuse-Rhine (QUIT-EMR) score is airway, breathing, circulation and disability derived, which is widely known to physicians, therefore it is easily applicable.

The QUIT-EMR score has the advantage of including additional points for interventions.

The use of retrospective data revealed some missing data, most probably due to lack of entry due to the stability of clinical parameters.

Introduction

For critically ill patients transferred from an Intensive Care Unit (ICU) to another ICU, transport modalities usually consist of specially designed ambulances carrying standard ICU equipment on board with dedicated, ICU trained physicians and nurses caring for the patient.1 2 These so-called Mobile Intensive Care Units (MICU) are expected to deliver high-end, maximum quality care and are often based at a tertiary medical centre. MICU generally does not cover emergency transportations, especially in rural areas.1–3 Transportation modalities of critically ill vary nationally and internationally, regarding whether physicians specialised in advanced supportive care, such as anaesthetists and intensivists, are mandatory to be present on a transport. Since intrahospital and interhospital transports pose a severe threat to patient safety, the quality of transports is of great importance to developing safe transportation methods.1 3 4 Momentarily, no gold standard to review the quality of interhospital transport exists. Transport parameters on airway, breathing, circulation and disability (ABCD) are established to predict patients’ outcomes. We conceived the QUality of Interhospital Transportation in the Euregion Meuse-Rhine (QUIT-EMR) score based on variables regarding ABCD; consciousness and a patent airway enable adequate breathing and support an effective and stable circulation. In order to (1) diagnose clinical deterioration during transportation, (2) evaluate the effectiveness of interventions during future transportations and (3) to provide a tool to predict what categories of patients are likely to deteriorate during the planned transportation, the present study cross-validates the QUIT-EMR score with an expert panel to gain more insights in MICU transportation quality.

Materials and methods

Patient population

We studied 100 randomly chosen interhospital transportations of critically ill, adult (>18 years) patients between April 2012 and May 2014 from the MICU database Maastricht, the Netherlands. This database includes all interhospital MICU transports of intensive care patients in the Intensive Care Units Zuid-Oost Nederland (ICUZON) region and is prospectively collected.4 Transportation is coordinated by the MUMC+ as follows. In summary, the hospital requesting transportation calls the specialist transportation nurse at the coordinating centre, who discusses the case with a qualified physician (anaesthetist or intensivist), after which the transport is accepted or declined. The transportation nurse subsequently plans the transport with the Regional Ambulance Department. The specialist transportation team (including a qualified physician and a nurse specialised in the transportation of critically ill patients) assesses the patient locally. Next, they transfer the patient using the transportation trolley to the desired location. The transporting anaesthetist/intensivist registers vital, laboratory, therapy and other transportation parameters on the clinical report form at the initial call, at the start, during and at the end of the transport. For the present study, we first shuffled all files (~350) of the transports conducted by the Maastricht UMC+ between 2012 and 2014. Next, we selected and included every third file. Due to the lack of published data on the incidence of (in)adequate transportations, no formal power calculation was performed, and a sample of 100 transport files was pragmatically chosen.

The QUality of Interhospital Transportation in the Euregion Meuse-Rhine (QUIT-EMR) score

An expert panel defined the guiding principles underlying the score. The expert panel consisted of medical directors of the Emergency Medical Service of the city of Aachen, the region of Aachen, Germany, and the medical coordinator of the MICU, Maastricht UMC+, the Netherlands. The transported patient’s clinical condition was considered to be determined primarily by ABCD parameters focusing on the following three major organ systems: the neurological system, the respiratory system and the circulatory system. We chose, a priori, to select parameters widely used for vocational training in critical care practice to develop the score. This strategy reflects daily critical care practice and facilitates the implementation potential of the score. Although no formal Delphi method was employed, the final selection of parameters was discussed until consensus between experts. The neurological status determines airway patency and is affected by the clinical condition and sedation therapy. The respiratory system encompasses breathing support during transportation and includes oxygenation as well as mechanical ventilator support. The circulatory system is optimised during transportation by fluid resuscitation and vasopressor administration. The QUIT-EMR score is shown in table 1.

Table 1

QUIT-EMR score

Neurological parameters during transportation
	Neurological improvement			Neurological deterioration
	Departure	Arrival	Points	Departure	Arrival	Points
1a	Comatose	Altered (ie, responds to stimulus) or awake	+1	Awake	Altered (ie, responds to stimulus) or comatose	−1
1b	Altered (ie, responds to stimulus)	Awake		Altered (ie, responds to stimulus)	Comatose
1c	Pupillary light reflex absent	Pupillary light reflex present		Pupillary light reflex present	Pupillary light reflex absent
2	Agitated	Calm	+1	Calm	Agitated	−1
	Intervention undertaken related to neurological improvement			No intervention is undertaken despite the neurological deterioration
3	Change sedatives/analgesics or bolus sedatives		+1	No change sedatives/analgesics or bolus sedatives		−1

FiO2, a fraction of inspired oxygen; NIV, non-invasive ventilation; SpO22, pulse oximetry.

In summary, the QUIT-EMR score is an overall sum score that includes component scores of monitoring and supportive treatment variables of the neurological, respiratory and circulatory organ systems. Within one organ system, a point can be scored per variable, and with more variables per organ system, more than one point can be scored. A clinically stable variable scores 0 points. A clinically improving variable scores +1 point. A clinically deteriorating variable scores −1 point. Interventions on the neurological, respiratory and circulatory organ systems add additional points. The maximum number of intervention points per organ system is limited to 1 point per organ system as follows: for the central nervous system, intervention points are scored for interventions affecting the patients’ mental status (eg, bolus application of sedatives or analgesics). For the respiratory organ system, intervention points are scored for interventions affecting the oxygenation of a mechanically ventilated patient (eg, changes in Positive End Expiratory Pressure level and medication concerning ventilation, such as muscle relaxants). For the circulatory organ system, intervention points are scored for interventions affecting blood pressure and heart rhythm (eg, volume therapy, change in number or dosage of vasoactive medications). The total sum score can range from −12 up to +12. A transport was defined as adequate if an overall QUIT-EMR sum score of zero or higher was found. A transport was defined as inadequate in case a QUIT-EMR sum score was below 0 points.

Scoring transportation quality by a QUIT-EMR and by independent experts without knowledge of a QUIT-EMR score

The QUIT-EMR sum scores were calculated by two investigators independently (US and MF) based on clinical report forms containing the information about vital parameters throughout the transport using a uniform study datasheet. However, missing values occurred as the physician registered vital parameters at the start of the transport only (most encountered) or did not register specific values, reflecting real-life practice. Any disagreement between the two investigators was resolved by discussion until consensus was reached. To enable calculation of the QUIT-EMR score, the general assumption was that the missing values remained unchanged during transport. Next, four experts (1–4), who were not involved in the expert panel or calculating QUIT-EMR scores, independently assessed the same 100 clinical report forms of 100 transported patients. All experts were anaesthetists and/or intensivists from the Maastricht UMC+ experienced in interhospital transportation. These experts were blinded to the QUIT-EMR score. They denominated each of the 100 transports as either adequate or inadequate, based on expert judgement, without further instructions. Inadequate transports were characterised by physiologic instability or physiologic deterioration with either no intervention or inadequate interventions, as defined independently by the four experts.

Statistical analyses

One hundred transported patients’ characteristics were described using mean±SD, median (IQR) or percentages where appropriate, calculated using IBM SPSS Statistics for Windows (V.25.0.0.2, Armonk, NY, USA, IBM Corp.). Missing descriptive data were reported. Next to the percentage agreement regarding adequate or inadequate transport, the level of agreement between QUIT-EMR and expert opinion and between experts was computed using Gwet’s AC1 instead of Cohen’s kappa due to the high proportion of adequate transports.5 6 Gwet’s AC1 was calculated with AgreeStat2015.6. (http://agreestat.com/agreestat)).

Patient and public involvement

No patient involved.

Results

The transported patients’ mean age was 61±15 years, 31% of transported patients were women and the mean transportation time was 74±30 min. Components of the QUIT-EMR score are shown in table 2.

Table 2

Study population

General characteristics
Age, years, mean±SD	61±15
Women, %	31
Transport time, minutes, mean±SD*	74±30
Reason for transfer†
Lack of capacity, %	6
Higher-level ICU, %	63
Repatriation, %	4
Intervention, %	16
Other, %	6
Missing, %	5
Neurological parameters	Departure	Arrival	Intervention
GCS
Comatose, %‡	33	14
Altered, %§	6	2
Awake, %¶	29	11
Missing, %	32	73
Pupillary light reflex
Absent, %	3	–
Present, %	71	20
Missing, %	26	80
Arousal
Agitated, %**	3	–
Calm, %	83	–
Missing, %	14	–
Intervention
Increase sedatives/analgesics or bolus sedatives, %	n.a.	n.a.	11
Decrease sedatives/analgesics, %	n.a.	n.a.	1
Ventilatory parameters
Breaths per minute, median (IQR)	20 (18–25)	21 (18–26)
Missing, %	15	31
SpO2, %, median (IQR)	98 (96–100)	98 (95–100)
Missing, %	1	6
Mode of ventilation
No additional oxygen, %	1	–
Nasal oxygen, %	12	7
Oxygen mask, %	4	3
NIV, %	3	3
Invasive ventilation, %	78	73
Missing, %	2	14
Intervention:
Increase of positive end-expiratory pressure, %	n.a.	n.a.	3
Decrease of positive end-expiratory pressure, %	n.a.	n.a.	–
Bolus muscle relaxants, %	n.a.	n.a.	15
Circulatory parameters
Rhythm
Sinus rhythm, %	56	52
Sinus bradycardia, %	3	1
Sinus tachycardia, %	18	16
Atrial fibrillation, %	9	9
Atrial flutter, %	1	–
Pacemaker or other %	2	2
Missing, %	11	20
Heartbeats per minute, mean number ±SD	93±22	93±23
Missing, %	13	18
Systolic blood pressure, mean mm Hg ±SD	130±25	127±23
Missing, %	0	5
Intervention
Increase of vasoactive medication, %	n.a.	n.a.	6
Decrease of vasoactive medication, %	n.a.	n.a.	14
Fluid resuscitation, %††	n.a.	n.a.	22
Administration of erythrocytes, %‡‡	n.a.	n.a.	3
Additional interventions to control bleeding, %	n.a.	n.a.	–
Increase in the number of vasoactive medication, %	n.a.	n.a.	3
Decrease in the number of vasoactive medication, %	n.a.	n.a.	–

*Transport time, time departure patient transportation until arrival.

†Reason for transfer, mutually exclusive.

‡Comatose, EMV 3 (Eye-opening, best Motor response, best Verbal response).

§Altered, EMV 4-14 (i.e., responds to stimulus).

¶Awake, EMV 15.

**Agitated, as mentioned in Sedation-Agitation Scale, scored before departure.

††Fluid resuscitation, fluid administration ≥ 500 mL.

‡‡Erythrocytes, 280 mL of erythrocyte concentrate/packed red blood cells. n.a., not applicable, changes during transport reported as an intervention.

GCS, Glasgow Coma Scale; NIV, non-invasive ventilation; SpO2, peripheral capillary oxygen saturation.

QUIT-EMR scores

The median QUIT-EMR score was 1, with an IQR of 0–2. QUIT-EMR scored 94 transports adequate (0 points or higher) and six transports inadequate (below 0 points). In the category with adequate transports, 78 interventions were performed. In several transports, more than one intervention was performed. Clinical improvement was documented in 20 transports. During 13 of these transports, at least one intervention was performed, whereas no interventions were performed during seven transports. During nine transports, the patient’s condition deteriorated, despite interventions by the transportation team. During the six inadequate transports, no interventions were performed by the transport team.

Expert opinion

All experts scored 100 cases, except expert four, who considered the documentation of two transport charts as insufficient to evaluate. The experts rated the transports using the transportation forms according to their clinical judgement, knowledge and experience, without any knowledge or information on the QUIT-EMR score. The percentage of transports defined as adequate by the independent experts ranged from 90% to 95%.

Level of agreement

The percentage agreement between the QUIT-EMR score and experts’ opinions ranged from 84% to 92%, corresponding with a good to very good6 level of agreement (Gwet’s AC1 0.81–0.91; table 3). The interobserver agreement between experts ranged from 85% to 92%, corresponding to a (very) high interobserver agreement (Gwet’s AC1 0.82–0.91) (table 3).

Table 3

Agreement between quality of interhospital transportation in the Euregion Meuse-Rhine (QUIT-EMR) score: score and experts

	QUIT-EMR score	Overall expert score	Expert 1	Expert 2	Expert 3	Expert 4
Adequate transports scored, %	94		90	92	95	92
Percentage agreement
QUIT-EMR score, %	–
Overall expert score, %	87*	–
Expert 1, %	84	87*	–
Expert 2, %	86	94*	86	–
Expert 3, %	92	92*	85	91	–
Expert 4, %	84	92*	84	91	89	–
Level of agreement, Gwet’s AC1	QUIT-EMR score		Expert 1	Expert 2	Expert 3	Expert 4
QUIT-EMR score	–
Overall expert score, %	0.90*	–
Expert 1	0.81	0.89*	–
Expert 2	0.84	0.98*	0.83	–
Expert 3	0.91	0.95*	0.83	0.90	–
Expert 4	0.84	0.95*	0.83	0.92	0.90	–

Data are percentages or Gwet’s AC1, overall expert score is the mean of four experts.

*Calculated as a similar score for 3 out of 4 experts, n=96.

Discussion

The QUIT-EMR scoring system concerning the critically ill patients’ interhospital transport showed to be adequate and valid. The results show that the ABCD-derived QUIT-EMR score has a high level of agreement with experts regarding the classification of critically ill patients’ transport as either adequate or inadequate.

Prior research has proven the value of specialised transport teams.7–9 Patient safety is increasingly becoming a core item in healthcare, with numerous interventions to minimise the risks of treatment-related complications being performed.10–12 Interhospital transport is feasible by an ambulance staffed by paramedics or through specialised retrieval teams. Nevertheless, an interhospital transfer is one of the most challenging and high-risk procedures in terms of coordination and patient safety. The perspective of interhospital transport of critically ill patients as an Intensive Care intervention with a high potential for adverse events due to human and technical errors13–15 and its resulting quality of care is a field of increasing interest over recent years in general, and during the recent pandemic specifically. In 2016, van Lieshout et al discussed the problem that no validated and standardised way to score the quality of the team’s response to an event exists.16 In the accompanying editorial, Valentin and Schwebel again highlighted that ‘the response and the ability to resolve a critical event might be a more relevant performance indicator for a transport team than the pure rate of events’.17 We agree that critical events’ incidence does not necessarily reflect the quality of an interhospital transport system. In the QUIT-EMR scoring system, we combined changes in predefined physiologic parameters with intervention-related items to better describe the transport teams’ quality compared with the sole focus on the incidence of adverse events. The developed scoring system proves to be a valid tool for research purposes. The occurrence and/or prevention of adverse events was not specifically included in the QUIT-EMR score. Adverse events are important with regard to quality. However, with regard to transportation, these adverse events are multicausal (eg, medication error, or vehicle engine failure), thus did not solely reflect transportation quality. In addition, the prevalence of adverse events during transportation was rather low. However, actions made by the transportation team were more common, and thus not always focused at adverse events once they had occurred, yet more commonly preventive in nature. For these reasons, interventions by the transportation team were included in the QUIT-EMR score.

The study has several strengths and limitations. First, the QUIT-EMR score is ABCD-derived. The ABCD-method is widely used in the clinical assessment of critically ill and thus familiar to physicians. This makes the QUIT-EMR score easily applicable. Although each ABCD-component has a similar weight, likely reflecting critical illness pathophysiology suboptimally, the QUIT-EMR score has the advantage of including additional points for interventions, which adds information per ABCD-component. QUIT-EMR thereby incorporates the ability to detect, prevent and resolve a critical event into the score, which is perhaps more relevant concerning adequate transportation than the rate of events during transport.16 18 Second, the prevalence of inadequate transport was fortunately low, at 6%. Since agreement assessed by Cohen’s kappa is known to be biased by very low or very high prevalence, Gwet’s AC1, which appropriately takes prevalence into account, was used to evaluate the agreement between the QUIT-EMR score and expert opinion.5 6 Third, the study included 100 real-life transportation report forms. The retrospective results, however, revealed some missing data. We assumed missing values were mainly caused by lack of entry due to the stability of clinical parameters, which were therefore entered only once, whereas changes were considered to be entered more frequently. Even if this assumption were incorrect, the main result concerning the high level of agreement between QUIT-EMR and experts would remain unchanged. As our study was based on data logged on transport forms, data on rejected transports due to instability of patients were unavailable. In addition, data prior to the initial clinical evaluation by the transporting physician was likewise lacking. Nevertheless, it is common practice for a dispatching ICU to assess the patient’s clinical condition and perform stabilisation before arrival of and together with the transportation team before initiating transportation. Another limitation of the QUIT-EMR score is that it does not provide individual fit for each patient and/or patient category. For example, permissive hypotension is scored as negative, while this can be the appropriate measure for a particular patient. The next step in developing this score therefore would be to validate the score on a novel transport patient population.

This study shows the validity and adequacy of the QUIT-EMR score for identifying clinical deterioration during transportation and evaluating interventions’ effectiveness during transportation. Future studies can further explore the potential use of the QUIT-EMR score in assisting physicians to predict which patients are likely to deteriorate during transportations. Furthermore, the presence of specific patterns in patients transported adequately, respectively, inadequately, and the association with possible (adequate or inadequate) interventions could be explored.

Conclusion

A high level of agreement between the QUIT-EMR score and experts’ opinion was found, suggesting adequate validity of the score for research purposes. Several patterns of adequate and inadequate transportations with or without interventions were identified. The QUIT-EMR score is valid and thereby has the potential: to identify patients at risk before planned transportation, to objectify clinical deterioration during transportation and to evaluate the association of interventions during transportation on the outcome. Prospective application of the QUIT-EMR on a larger transport cohort will enable to classify patients into groups of (eg, best and worst) QUIT-EMR scores. These groups can subsequently be used to study the association between patient characteristics and outcome based on the QUIT-EMR score.