Assessment of novel ICT-EMS systems to improve emergency patient transportation during the COVID-19 pandemic.

OBJECTIVES: This study aimed to examine the usability, feasibility, acceptability, and appropriateness of the information and communication technology for emergency medical services (ICT-EMS) systems to improve the transportation of emergency patients during the COVID-19 pandemic.
METHODS: Emergency medical technicians (EMTs) (n = 229) employed at 7 fire stations operated by the North Chungcheong Fire Service Headquarters, South Korea were trained to use ICT-EMS devices prior to a 1-month implementation period. System Usability Scale (SUS), Feasibility of Intervention Measure (FIM), Acceptability of Intervention Measure (AIM), and Intervention Appropriateness Measure (IAM) questionnaires were conducted in the 4th week of the 1-month implementation period to assess the perceived usability, feasibility, acceptability, and appropriateness of the ICT-EMS systems.
RESULTS: Among a total of 229 EMTs, 187 EMTs (81.7%) completed the survey. The overall SUS score was significantly low (score of 35.6) indicating an overall negative perception of the ICT-EMS systems. With regard to the feasibility, acceptability, and intervention appropriateness of ICT-EMS, roughly 50 (26.7%) participants agreed that ICT-EMS implementation was possible, appealing, and suitable.
CONCLUSION: Many potential areas of improvement were identified within the ICT-EMS systems. System alterations regarding usability, feasibility, acceptability, and appropriateness may be necessary to successfully implement the ICT-EMS systems.

Introduction

The emergency medical services (EMS) system is an expansion of emergency medical care into the prehospital phase Many countries, including Korea, trace basic concepts in EMS systems development back to the United States EMS Systems Act of 1973, which addressed fifteen key elements: 1) access to care, 2) patient transfers, 3) communications, 4) mutual aid, 5) training, 6) facilities, 7) critical care units, 8) disaster plans, 9) manpower, 10) transportation, 11) public safety agencies, 12) consumer participation, 13) coordinated patient record keeping, 14) public information and education, and 15) reviews and evaluations [1]. EMS systems serve the communities by mobilizing ambulances in response to emergency situations, transporting patients to appropriate medical facilities, and providing emergency medical care [2]. The COVID-19 pandemic, a worldwide historical event, has affected the healthcare system as well as the society at large, including social, economic, political, and cultural aspects. The COVID-19 pandemic has had a negative impact on EMS systems worldwide. During the early pandemic phase, transportation of critically ill patients through EMS to emergency departments (EDs) decreased, while prehospital deaths increased because of termination of resuscitation before transportation to the ED and death at home without resuscitation [[3], [4], [5]].

During the pandemic, the prehospital time required to transport suspected COVID-19 patients as well as those who were critically ill to appropriate hospitals has increased due to the lack of isolation rooms [6]. Accepting suspected COVID-19 patients can be a burden for local hospitals with limited resources because of the risk of nosocomial infection of medical personnel or non-infected patients. In order to overcome these issues during the COVID-19 pandemic, solutions for improving the EMS systems have been proposed, such as regionalization of EMS systems, improvement of emergency room infrastructure, sharing of emergency medical resources using information and communication technology, and use of rapid COVID-19 diagnostic devices [2,[7], [8], [9]].

A coordinated and integrated response across the community is necessary to maintain essential EMS services on the frontlines of the COVID-19 pandemic to minimize the spread of the SARS-CoV-2 and support care for critically ill patients. Information and Communications Technology (ICT) systems have been applied to the healthcare industry, and the demand for related ICT services such as mobile health care is increasing [10]. Cheongju, the capital of North Chungcheong Province in South Korea, is preparing to establish an ICT system for EMS (ICT-EMS) to coordinate EMS activities with the regional fire agency and emergency medical centers to determine hospital destinations during the prehospital phase. Before implementation of a technology-based intervention and for its suitability, it is important to assess the perceived utility and ease of use, feasibility and acceptability of the new ICT-EMS systems by the end users. The purpose of this study was to examine usability, feasibility, acceptability, and appropriateness of the ICT-EMS systems by emergency medical technicians (EMTs) acting in Cheongju.

Materials and methods

Study design and setting

We conducted a questionnaire survey to determine the usability, feasibility and acceptability of the use of ICT-EMS systems to improve emergency patient transport during the COVID-19 pandemic prior establishment of ICT-EMS systems in the central area of North Chungcheong Province. This prospective study was conducted between February and August 2021 at North Chungcheong Fire Service Headquarters in South Korea. North Chungcheong Fire Service Headquarters operates 81 ambulances with 461 EMTs in 12 fire stations that cover 1.6 million residents in North Chungcheong Province, South Korea. North Chungcheong Province has 3 cities and 8 counties and consists of north, south, and central areas. The central area has 0.8 million residents serviced by 229 EMTs of 7 fire stations and has 1 regional emergency medical center, 3 local emergency medical centers, and 2 local emergency medical agencies. The regional emergency medical center is in charge of treating critical patients and coordinating between community stakeholders during crises such as pandemics and disasters.

Participants

Study participants were EMTs who consented to the study and completed the study survey. EMTs (n = 229) employed at 7 fire stations operated by the North Chungcheong Fire Service Headquarter were recruited. We used a stepped wedge study design for this community-based pilot study of ICT-EMS establishment. Over a 7-month period, each of the 7 fire stations covering the central area of North Chungcheong Province participated in the intervention for a 1-month period, in succession. The order of participation was randomly assigned. We introduced ICT-EMS and provided training to EMTs employed by each of the 7 fire stations before implementing the ICT-EMS systems during each station's scheduled month. EMTs were trained on ICT-EMS application use and designation of the appropriate destination hospital. The function of indirect education for EMTS by checking the clinical results of transported patients was also included in the ICT-EMS systems. The questionnaire was conducted in the 4th week of the 1-month ICT-EMS system implementation period at each fire station. Questionnaires were completed by the entire EMT staff of each of the 7 fire stations.

This study was approved by the Institution Review Board (IRB) of Chungbuk National University Hospital through expedited review with the requirement for informed consent waived (IRB Number: CBNUH 2020–11-013).

ICT-EMS systems

We developed ICT-EMS systems to improve the EMS transport process for emergency patients (cardiac arrest, severe trauma, cardio-cerebrovascular disease) and patients with suspected COVID-19. We investigated emergency medical resources of participant hospitals in the central region of North Chungcheong Province, including equipment and space necessary to perform surgery, percutaneous intervention, thrombolysis, and extracorporeal membrane oxygenation, and the number of isolation rooms available for the treatment of critical patients suspected of COVID-19. EMTs are the end user of the ICT-EMS systems and used an application developed to triage patients at the scene and decide the appropriate destination hospital for critical patients (defined as patients with cardiac arrest, severe trauma, chest pain/presumed myocardial infarction, and/or positive cerebral infarction screening), and for patients with suspected COVID-19.

The ICT-EMS application was used to designate the appropriate destination hospital based on real-time emergency medical resources, patients' previous hospital visits for treatment of major diseases, and distance from the scene to the destination hospital. Coordinators in the participant hospitals updated the number of usable emergency medical resources in real-time between 9 AM and 6 PM on weekdays. EMTs were instructed to place paper wristbands with embedded near field communication (NFC) chips on patients' wrists during EMS response. The ICT-EMS application was then used to match the patient-unique prehospital rescue number associated with the NFC wristband with the patient's medical record number at the hospital. The NFC chip-embedded wristband was implemented as a means of building a community-based emergency database by combining patient prehospital information with hospital clinical records (Fig. 1 ).

Fig. 1

Schematic diagram of the ICT-EMS systems.

ICT-EMS; information and communication technology for emergency medical services.

Measuring tools for evaluating the implementation of ICS-EMS: SUS, FIM, AIM, IAM

Reliable and validated tools, including the System Usability Scale (SUS), Feasibility of Intervention Measure (FIM), Acceptability of Intervention Measure (AIM), and Intervention Appropriateness Measure (IAM) were used for evaluating whether the ICT-EMS systems were implementable [[11], [12]]. The SUS contains 10 questions to be answered using the 5-point Likert scale (1; Strongly disagree; 2, Disagree; 3, Neither agree nor disagree; 4, Agree; 5, Strongly agree). Of these ten questions, five questions are positively formulated (statements with odd numbers) and the other five ones are negatively formulated (statements with even numbers). We explained the alternating formulation of these questions to all participants and asked them to answer each question with caution. The FIM questionnaire contains 4 statements assessing the extent to which a new system can be successfully used within a given setting, while AIM comprises 4 statements measuring the perception among implementation stakeholders of whether a delivered service is agreeable or satisfactory. IAM is used to measure the perceived fitness or compatibility of the new service with a given practice setting. Table 1 shows the statements and response categories for each SUS, FIM, AIM, and IAM item.

Table 1

Questions Used in Standard SUS, AIM, IAM, and FIM Surveys [11,12].

	Question
System usability scale, SUS	1. I would like to use the ICT application for EMS improvement frequently2. I found the ICT application for EMS improvement unnecessarily complex3. I thought the ICT application for EMS improvement was easy to use.4. I think that I would require technical support to be able to use the ICT application for EMS improvement.5. I thought the various functions in the ICT application for EMS improvement were well integrated.6. I thought there was too much inconsistency in the ICT application for EMS improvement.7. I think that most people would learn to use the ICT application for EMS improvement very quickly8. I found the ICT application for EMS improvement very cumbersome to use.9. I felt very confident using the ICT application for EMS improvement.10. I needed to learn a lot of things before I could begin using the ICT application for EMS improvement.
Feasibility of Intervention Measure, FIM	1. The EMS improvement project using ICT seems implementable.2. The application of the EMS improvement project using ICT seems possible.3. The EMS improvement project using ICT seems doable.4. The application of the EMS improvement project using ICT seems easy to use.
Acceptability of Intervention Measure	1. The application of the EMS improvement project using ICT meets my approval.2. The application of the EMS improvement project using ICT is appealing to me.3. I like the application of the EMS improvement project using ICT4. I welcome the application of the EMS improvement project using ICT
Intervention Appropriateness Measure	1. The application of the EMS improvement project using ICT seems fitting.2. The application of the EMS improvement project using ICT seems suitable.3. The application of the EMS improvement project using ICT seems applicable.4. The application of the EMS improvement project using ICT seems like a good match.

AIM, Acceptability of Intervention Measure; IAM, Intervention Appropriateness Measure; FIM, Feasibility of Intervention Measure; EMS, Emergency medical service; ICT, Information and Communication Technology.

Study outcomes

Usability and acceptability of the ICT-EMS systems to facilitate community-based patient transportation during the COVID-19 pandemic and feasibility and participant's satisfaction regarding the use of the ICT-EMS systems were assessed to determine how well the ICT-EMS systems were received by participating EMTs.

Statistical analysis

We used Excel (ver. 2013, Microsoft®, Santa Rosa, CA, USA) to complete all data analyses. The mean (SD) scores of SUS responses were obtained and converted into an overall SUS score out of 100 in where each of the questions had a weight of 10 points.

Results

Demographic factors of study participants

The general characteristics of the study participants are presented in Table 2 . Among a total of 229 EMTs employed in the central region of North Chungcheong Province, 187 EMTs (81.7%) participated in the survey, 44% of which were male. The median age was 31 years (IQR, 28–36). A total of 179 participants (95.7%) were level 1 EMTs, and the median duration of EMS employment was 4 years.

Table 2

General characteristics of the study subjects.

Variable	Label	N	%
Total	–	187	100
Sex	Male	83	44.4
	Female	104	55.6
Age	Median (IQR) 31 (28–36)
	20–29	66	35.3
	30–39	95	50.8
	40–49	25	13.4
	missing	1	0.5
Certification	EMT I (including nurse)	179	95.7
	EMT II `	8	4.3
Job experience (years)	Median (IQR) 4 (2−10)
	0–3	90	48.1
	4–6	29	15.5
	7–9	17	9.1
	10–12	23	12.3
	13–15	13	7.0
	16 ~	14	7.5
	missing	1	0.5

EMT, emergency medical technician; IQR, interquartile range.

ICT-EMS usability (SUS survey)

The mean SUS score was significantly low at 35.6 out of a possible 100. Participants' perception of the usability of the ICT-EMS systems was as follows: the system was very cumbersome (mean ± SD, 3.8 ± 1.0), unnecessarily complex (3.7 ± 1.1), had too many inconsistencies (3.7 ± 0.9), required technical support (3.6 ± 1.1), and required extensive learning before use (3.4 ± 0.9), which are all negative assessments of the system's usability. In contrast, mean scores for responses to positive statements were approximately neutral (i.e., ranged from 2.4 to 2.7) (Table 3 ).

Table 3

System Usability Scores (SUS) of study participants.

SUSa	Scorebmean (SD)	Convertedc Score
1. I would like to use the ICT application for EMS improvement frequently	2.5 (1.1)	1.5
2. I found the ICT application for EMS improvement unnecessarily complex	3.7 (1.1)	1.3
3. I thought the ICT application for EMS improvement was easy to use.	2.4 (0.9)	1.4
4. I think that I would require technical support to be able to use the ICT application for EMS improvement.	3.6 (1.1)	1.4
5. I thought the various functions in the ICT application for EMS improvement were well integrated.	2.4 (1.0)	1.4
6. I thought there was too much inconsistency in the ICT application for EMS improvement.	3.7 (0.9)	1.3
7. I think that most people would learn to use the ICT application for EMS improvement very quickly	2.7 (1.0)	1.7
8. I found the ICT application for EMS improvement very cumbersome to use.	3.8 (1.0)	1.2
9. I felt very confident using the ICT application for EMS improvement.	2.4 (0.9)	1.4
10. I needed to learn a lot of things before I could begin using the ICT application for EMS improvement.	3.4 (0.9)	1.6
Total converted mean scores × 2.5 (overall SUS score)	N/Ad	35.6

SUS: System Usability Scale

1: strongly disagree, 2: somewhat disagree, 3: neutral or no opinion, 4: somewhat agree, and 5: strongly agree.

For items 1, 3, 5, 7, and 9, the converted score is the mean score minus 1. For items 2, 4, 6, 8, and 10, the converted score is 5 minus the mean score.

N/A: not applicable.

ICT-EMS feasibility and acceptability (FIM, AIM, and IAM survey)

In the FIM survey used to assess the feasibility of the ICT-EMS systems, the responses “agree” or “strongly agree” were selected by 53 participants (28.3%) in response to the statement “ICT-EMS seems implementable,” and by 25 participants (13.4%) in response to the statement “ICT-EMS seems easy to use,” the latter of which had the lowest agreement rate. In response to the statement “ICT-EMS seems doable,” the responses “agree” or “strongly agree” and “disagree” or “strongly disagree” were equally selected by 52 participants (27.8%) (Fig. 2A).

Fig. 2

Participant Responses to FIM, AIM, and IAM as feasibility tests for the introduction of new ICT-EMS systems to EMTs.

FIM; feasibility of intervention measure, AIM; acceptability of intervention measure, IAM; intervention appropriateness measure, EMTs; Emergency medical technicians.

In the AIM survey regarding the acceptability of the ICT-EMS systems, the responses “disagree” or “strongly disagree” in all statements were higher in number compared with “agree” or “strongly agree.” The responses “agree” or “strongly agree” were selected by 19 participants (10.2%) in response to the statement “ICT-EMS meets my approval”, which had the lowest agreement rate (Fig. 2B).

In the IAM survey used to assess the appropriateness of the ICT-EMS systems as an intervention, “agree” or “strongly agree” were more selected than “disagree” or “strongly disagree,” by 54 (28.9%) and 51 participants (27.3%), respectively, in response to the statement “ICT-EMS seems fitting.” The responses “agree” or “strongly agree” in other statements were lower in number compared with “disagree” or “strongly disagree. The agreement rate (14.4%) by 27 participants was the lowest in response to the statement “ICT-EMS seems suitable” (Fig. 2C).

Discussion

This study was aimed to assess the usability, feasibility, acceptability, and appropriateness of ICT-EMS systems developed for coordinating the transportation of patients with suspected COVID-19, as well as critically ill patients during the COIVD-19 pandemic. The overall SUS score in our study was 35.6, which was significantly lower compared with the standard average score of 68 or higher, which typically indicates a sufficiently usable ICT-EMS systems [12]. In the FIM, AIM, and IAM surveys, the agreement rate of the approvability, ease of use, and suitability of the questionnaires were the lowest. The results of this study indicate that there is resistance among EMTs due to usability, feasibility, acceptability, and appropriateness flaws within the ICT-EMS systems.

Through the construction of digital platforms for sharing patient data and information regarding emergency resources at community hospitals, EMTs and medical personnel in participant hospitals can communicate to determine where to transport patients for optimal patient care within an appropriate amount of time. ICTs have the potential to improve the treatment of critical patients diagnosed with acute myocardial infarction, acute stroke, and COVID-19 [13,14]. The use of applications and ICTs in EMS systems is expected to have a positive effect on treatment outcome and administrative and supporting processes. User experience could be a vital factor affecting the SUS score and the agreement rate of the FIM, AIM, and IAM surveys obtained in this study.

The SUS, originally created by John Brooke in 1986, was designed to provide a reliable and expedient scale for assessing the perceived usability of a wide variety of products, services, or systems, including applications, mobile devices, and websites [15]. Items were presented alternatively, switching between positively- and negatively-phrased statements. This alternating structure increases the cognitive burden on the respondents answering the questionnaire [16]. In the analysis of responses to the SUS questionnaire, the mean of negatively-phrased items was lower than that of positively-phrased items (2.5 vs. 3.6). After negatively-phrased items were inverted, a mean SUS score of 35.6 was obtained, indicating that the overall SUS response of EMTs regarding the ICT-EMS systems was negative.

We believe that the low SUS scores and the low agreement rate of the FIM, AIM, and IAM surveys regarding the ICT-EMS systems are due to the EMTs' dissatisfaction with the system configuration, such as required multitasking, complexity, and inconsistency. EMTs were required to manage the patient and enter patient information into the ICT-EMS systems during transportation. Previously, EMTs used an existing tablet PC to enter basic patient information during transport, so it may have felt cumbersome to use an additional device to enter data into the ICT-EMS systems.

ICT-EMS systems modification is necessary to be effectively integrated into the workflow of the community EMS [17]. Given that the system's usability, feasibility, acceptability, and appropriateness scores were related to user satisfaction, it may be improved upon by collecting the opinions of users, including EMTs and medical personnel, to fully identify issues [18]. Then, training EMTs and medical personnel on these upgraded ICT-EMS will be needed. Modifying EMT behaviors related to the ICT-EMS may take time, but it can be done by generating positive motivation to reach better outcomes for emergency patients [19].

This study has some limitations which should be acknowledged. First, study surveys were conducted at the early stages of a project after 3 weeks of ICT-EMS system implementation, and participating EMTs' perceptions of the ICT-EMS systems were not positive overall. The one-month implementation period may have been too short for sufficient exposure to the ICT-EMS systems prior to perception assessment. In addition, EMTs employed at fire stations that are located at some distance from the central area of North Chungcheong Province occasionally transport patients to neighboring communities where ICT-EMS systems are not used; therefore, they may not have had ample opportunities to use the ICT-EMS systems during patients' transportation. Third, half of the participants (47.8%) were employed at two fire stations in the study community. The average time of transportation was about 10 min, which is insufficient time to use ICT-EMS application during transportation. This was the primary point of dissatisfaction reported by EMTs regarding the ICT-EMS systems. Improvements to ease of use and EMT familiarity with the ICT-EMS systems are necessary to promote the use of ICT-EMS systems during patients' transportation across short distances.

Conclusion

The perceived usability, feasibility, acceptability, and appropriateness of ICT-EMS systems implementation reported by EMTs was poor, indicating that there are many areas of improvement that can be addressed to improve the ICT-EMS systems. Before the widespread establishment of ICT-EMS systems for facilitating emergency transport of patients during the COVID-19 pandemic, additional surveys and interviews of proposed ICT-EMS systems can be helpful to understand the issues associated with the ICT-EMS systems. After addressing areas of improvement, such as reducing multitasking and simplifying the complexity of the ICT-EMS systems, this survey may be repeated to assess the perception of the modified ICT-EMS systems.

Funding

This research was supported by the

Declaration of Competing Interest

None.