Quality measurement in physician-staffed emergency medical services: a systematic literature review.

PURPOSE: Quality measurement of physician-staffed emergency medical services (P-EMS) is necessary to improve service quality. Knowledge and consensus on this topic are scarce, making quality measurement of P-EMS a high-priority research area. The aim of this review was to identify, describe and evaluate studies of quality measurement in P-EMS. DATA SOURCES: The databases of MEDLINE and Embase were searched initially, followed by a search for included article citations in Scopus. STUDY SELECTION: The study eligibility criteria were: (1) articles describing the use of one quality indicator (QI) or more in P-EMS, (2) original manuscripts, (3) articles published from 1 January 1968 until 5 October 2016. The literature search identified 4699 records. 4543 were excluded after reviewing title and abstract. An additional 129 were excluded based on a full-text review. The remaining 27 papers were included in the analysis. Methodological quality was assessed using an adapted critical appraisal tool. DATA EXTRACTION: The description of used QIs and methods of quality measurement was extracted. Variables describing the involved P-EMSs were extracted as well. RESULTS OF DATA SYNTHESIS: In the included papers, a common understanding of which QIs to use in P-EMS did not exist. Fifteen papers used only a single QI. The most widely used QIs were 'Adherence to medical protocols', 'Provision of advanced interventions', 'Response time' and 'Adverse events'.
CONCLUSION: The review demonstrated a lack of shared understanding of which QIs to use in P-EMS. Moreover, papers using only one QI dominated the literature, thus increasing the risk of a narrow perspective in quality measurement. Future quality measurement in P-EMS should rely on a set of consensus-based QIs, ensuring a comprehensive approach to quality measurement.

Background

Emergency medical services (EMS) provide pre-hospital treatment and transportation to definitive care for patients in need of urgent medical care. EMSs are well integrated in health services in most countries and normally consist of ground ambulances staffed by paramedics, emergency medical technicians or nurses [1-6]. As a supplement to regular EMS, physician-staffed rapid response vehicles or helicopters exist in some areas [7]. Although the operational concept of this physician-staffed EMS (P-EMS) may differ, a common feature is the involvement of a specially trained physician in pre-hospital care of critically ill or injured patients. Depending on the country, these physicians are often anaesthesiologists, surgeons, internists or emergency physicians [1, 3].

The focus on quality measurement in healthcare is increasing [8-12]. As an example, quality dimensions such as efficiency, patient-centeredness and safety have been assessed in several emergency departments [13-15]. In P-EMS, a valid model for quality assessment is needed to achieve appropriate governance, quality assurance and quality improvement [16]. Snooks et al. [17] define the development of meaningful quality indicators (QIs) for EMS as the most important issue for future research in emergency pre-hospital care. For P-EMS, measuring quality of care is considered a priority area of research [18].

Quality measurement can be defined as measuring the extent to which set targets are achieved [19]. A QI is used to measure performance against a recognized standard of care. Donabedian defines three categories of QIs: structure, process and outcome of healthcare [20, 21]. Structure indicators describe the infrastructure of a healthcare system, such as competence of the staff, equipment and deployment and response times. Process indicators evaluate the care provided to the patient, and outcome indicators address the change in patient health status. None of these categories of indicators provide a complete description of the quality of care but address single components. Thus, different types of QIs should be combined to assess the quality of a service [19].

QIs inform clinicians and organizations how the health system performs and aid in the improvement in care. Ideally, all QIs are based upon evidence of their relevance and importance. The process of developing QIs generally includes stakeholders who evaluate the evidence and define the QI parameters [22]. These QIs for P-EMS should be evaluated against patient-oriented outcomes, e.g. pain intensity, morbidity or mortality. However, P-EMS quality can also relate to system factors such as training of traditional EMS, major incident management and the concept of providing equity of access to healthcare. Different stakeholders have different perspectives on what represents quality in healthcare [23, 24], and various QIs for P-EMS are possible.

A widely cited definition of quality that also might be applicable for P-EMS systems is ‘the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge’ [9]. Often, a few high impact clinical conditions are identified (in terms of morbidity, mortality, costs and incidence). These conditions are termed ‘tracer conditions’. Examples of tracer conditions for emergency medicine are cardiac arrest or trauma patients with severely reduced consciousness (Glasgow Coma Score < 8). Measuring the outcomes of tracer conditions can predict a system’s response to other clinical states and the overall quality of a service [25].

This systematic review aims to identify, describe and appraise the methodological quality of the literature pertaining to the quality assessment of P-EMS.

Methods

For the purpose of this review, physicians who staff P-EMS should be trained in critical care, exceeding the competency of a general practitioner on call [26]. Moreover, we define the term ‘pre-hospital’ as relating to procedures administered or care provided prior to patient arrival at the hospital [8]. The studies identified in the review do not address the potential benefit from P-EMS compared to other EMS.

Literature search strategy

A systematic literature search of MEDLINE and Embase to identify relevant literature was conducted (see Additional file 1 for search strategy). Four different sets of entry terms were applied and combined. These entry terms describe pre-hospital setting, emergency care, physician staffing and finally the concept of quality measurement. All records were collated in an Endnote bibliographic database (©2007 Thompson Reuters).

The study followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, including the PICOS (Population, Intervention, Comparator, Outcome, Study design) methodology [27]. Here, the participants were all the identified articles describing quality estimation in P-EMS. Our evaluation of intervention, comparison and outcomes was carried out using the data extraction and quality appraisal variables in Tables 1 and 3. The study was registered at PROSPERO (http://www.crd.york.ac.uk//prospero/, registration number CRD42015024421).

Table 1

Reasons for excluding 129 out of 156 full-text studies in the eligibility-check of the systematic review

Reasons for exclusion	No.
Wrong study design	77
Not about quality measurement in P-EMS	29
Not enough information for quality appraisal and data extraction	9
Only abstracts	6
Not about P-EMS	3
Comparative studies pertaining to new procedures	2
Commentary, letter to editor or editorial	1
Not original article	1
Duplicate	1

Table 3

Quality indicators used in the included literature

Quality indicator	Category	No. of papers it is used in
Adherence to medical protocols	Process	11
Provision of advanced interventions	Process	8
Response time	Process	7
Adverse events	Process	7
Medication administration	Process	5
Transport to appropriate facility	Process	4
Time on scene	Process	3
Improved care due to clinical decision making	Process	3
Reliability of the primary diagnosis made by the P-EMS physician	Process	3
Survival	Outcome	3
ROSC in cardiac arrest	Outcome	3
Time from alarm to patient handover	Process	2
Time from arrival at patient until hospital admission	Process	2
Altered physiology	Outcome	2
Pain management	Outcome	2
Proportion of intubated patients adequately oxygenated and ventilated	Process	2
Time gain by air transportation	Process	1
The number of patients with a NACA-score ≥ 4 with an intravenous line	Process	1
The proportion of patients successfully intubated	Process	1
Life years gained	Outcome	1
Morbidity/disability	Outcome	1
Amount of yearly CPR training	Structure	1
Precision of dispatch	Process	1
Rate of CPR started within 8 min of the call to the dispatch center	Process	1
Patient satisfaction	Outcome	1

ROSC, return of spontaneous circulation; NACA, National Advisory Committee for Aeronautics (see Additional file 2); CPR, cardiopulmonary resuscitation.

Inclusion/exclusion criteria

Articles were included in the systematic literature review if they fulfilled all the following criteria: (1) literature describing methods of quality estimation in P-EMS, i.e. the use of one or more QIs based on quantitative methods, qualitative methods or both; (2) original manuscripts and (3) literature published after 1 January 1968 and until the date of the literature search, 5 October 2016. The rationale for including literature from 1 January 1968, is the establishment of the world’s first civilian physician-staffed helicopter EMS in Munich this year [28].

Articles without abstract, book chapters, editorials, comments and letters to the editor were excluded. Articles in English, French, German and Scandinavian languages were identified. The translation competency of these languages was present in the author group.

Literature identification

The records from the literature search were exported to www.covidence.org. Here, all titles and corresponding abstracts were screened independently by two of the authors for eligibility (O.U. and H.H.) (Figure 1). In the case of uncertainty, a third reviewer (M.R.) reviewed the title/abstract. Articles clearly not meeting the inclusion criteria were excluded. Articles accepted for full-text screening were assessed in pairs of authors (O.U. and M.R., A.J.K. and H.H.) using the inclusion/exclusion criteria listed above. Excluded articles were listed with the reason for exclusion. If there was any uncertainty about whether an article should be included, there was a discussion until consensus was reached among the authors. One author (H.H.) performed data extraction and quality appraisal and consulted another author (M.R.) in uncertain cases. Further, one author hand-searched references in included articles to identify additional relevant articles (H.H.). Finally, a search in Scopus was conducted to identify articles citing the included articles.

Figure 1

Information flow through the different phases of the systematic review.

Prior to the literature search, the authors made templates for data extraction and quality appraisal. The data extraction and quality appraisal variables were based on the authors’ assumptions on what is important to report in quality measurement studies in P-EMS. However, these variables do not represent a reference standard, since such a standard does not exist, to our knowledge. As a part of data extraction, fixed-system variables were included. Fixed-system variables relate to system characteristics concerning the organization, staffing and operational capacities of the service and are necessary for interpreting the results [29].

Results

A total of 4699 articles were identified by the search strategy, 156 of which were accepted for full-text screening. Of these, 129 articles were excluded. The main reason for excluding articles were ‘Wrong study design’, pertaining to articles exclusively comparing different treatment modalities and without any quality measurement objectives (Table 1). A total of 26 articles from the main database search were included for data extraction and quality appraisal [30-55]. One additional article was included from the Scopus search for citing articles [56]. A review of the literature lists of included articles did not result in additional findings.

None of the papers gave a complete report of fixed-system variables, thus complicating the comparison of involved P-EMS concepts (Table 2). Twenty-four of the 27 papers use QIs that can be identified as process indicators. Structure indicators and outcome indicators are used less frequently, in two and seven papers, respectively. Twenty-five different QIs were identified, all of which were considered suitable for international use and transferable to other P-EMSs (Table 3). The most widely used QI was ‘Adherence to medical protocols’. This QI measures if medical guidelines are followed, as done by Viergutz et al. [53] who investigated whether guidelines for preclinical care of patients with traumatic head injury were followed. The second most used QI was ‘Provision of advanced interventions’, investigating if the P-EMS unit provided treatment that exceeds the competences of the attending EMS, as done by Mikkelsen et al. [44]. The two following QIs are ‘Response time’ and ‘Adverse events’, the latter exemplified by Nakstad et al. [45], who studied the incidence of desaturation during pre-hospital rapid sequence intubation. Fifteen papers used one single QI, and twelve papers applied a set of QIs. Moreover, three papers used tracer conditions as their approach to quality assessment (Table 2). Pertaining to the internal validity of the papers, ten of the 27 papers did not clearly explain the methodology for developing the QIs (Table 4).

Table 2

Data extraction of included articles

	Are the following fixed-system variables reported?									Quality indicators used				Multiple, mixed quality indicators	Tracer conditions used
	Transportation mode	SAR-capability	Rural and/or urban	Response types	Speciality of physician	Physician experience	Funding	Description of dispatch system	Other fixed-system variables	Structure	Process	Outcome	Patient satisfaction
Akin Paker [30]	✓	✗	✗	✓	✗	✗	✗	✗	✗	✗	✓	✗	✗	✓	✗
Anadelic [31]	✓	✗	✓	Partly	✗	✗	✗	✓	✓	✓	✓	✗	✗	✓	✗
Arntz [32]	✓	✗	✓	✓	✓	Partly	✓	✓	✓	✗	✓	✗	✗	✓	✗
Breckwoldt [33]	✓	✗	✓	✗	✓	✓	✗	✗	✓	✗	✓	✓	✗	✗	✗
Duchateau [34]	✓	✗	✓	✗	✗	✗	✗	✗	✓	✗	✓	✗	✗	✗	✗
Fjaeldstad [35]	✓	✗	✓	✓	✓	✗	✓	✓	✓	✗	✓	✗	✗	✗	✗
Flabouris [36]	✓	✓	✗	✓	✓	✓	✗	✓	✓	✗	✓	✗	✗	✓	✗
Helm [37]	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗	✗	✗	✗
Helm [38]	✓	✗	✗	✗	✓	✗	✗	✗	✗	✗	✓	✗	✗	✗	✗
Hennes [39]	✓	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗	✓	✗	✗	✗
Klemenc-Ketis [40]	✓	✓	✓	✓	✗	✗	✓	✓	✓	✗	✓	✗	✗	✓	✗
Leicht [41]	✓	✗	✗	✓	✓	✓	✗	✗	✓	✗	✓	✗	✗	✗	✗
Lossius [42]	✓	✗	✓	✓	✓	✓	✗	Partly	✗	✗	✗	✓	✗	✗	✗
Messelken [43]	✓	✗	✓	✓	✓	✗	✗	✗	✗	✗	✓	✓	✗	✓	✓
Mikkelsen [44]	✓	✗	✓	✓	✓	✓	✗	✓	✓	✗	✓	✓	✗	✗	✗
Nakstad [45]	✓	✗	✓	✓	✓	✓	✗	✗	✓	✗	✓	✗	✗	✗	✗
Neukamm [46]	✓	✗	✓	✓	✗	✗	✗	Partly	✓	✓	✓	✓	✗	✓	✗
Pedersen [47]	✓	✗	✓	✓	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗	✓
Regel [48]	✓	✗	✓	✓	✗	✗	✗	✗	✓	✗	✓	✗	✗	✓	✗
Rognås [49]	✓	✗	✓	✓	✓	✓	✗	✓	✓	✗	✓	✗	✗	✗	✗
Rognås [50]	✓	✗	✓	✓	✓	✗	✗	Partly	✓	✗	✓	✗	✗	✗	✗
Sollid [51]	✓	✗	✓	✓	✓	✗	✗	✗	✓	✗	✓	✗	✗	✗	✗
Van der Velden [52]	✓	✗	✓	✓	✗	✗	✗	✓	✓	✗	✓	✗	✗	✗	✗
Viergutz [53]	✓	✗	✓	✓	✗	✗	✗	✗	✓	✗	✓	✗	✗	✓	✗
Von Knobelsdorff [54]	✓	✗	✓	✓	✗	✗	✗	✗	✗	✗	✓	✓	✗	✓	✗
Weltermann [55]	✓	✗	✓	✓	✗	✗	✗	✗	✗	✗	✓	✗	✗	✓	✗
Schlechtriemen [56]	✓	✗	✗	✗	✗	✗	✗	✗	✗	✗	✓	✗	✗	✓	✓

✓ = yes; ✗ = no.

Table 4

Quality appraisal of included articles

	Internal validity						External validity
	Was the methodology for developing the QIs clearly explained?	Are the QIs clearly defined?	Is the rationale for the QIs sufficiently described?	Is handling of missing data sufficiently described?	Do the authors address conflicts of interest?	Has an ethics committee approved the study?	Is professional background and funding of those involved in QI development stated?	Were the QIs developed by the systematic work of a group of experts?	Is it reported in which country/organization the QIs were developed?	Is (are) the P-EMS (s) involved in quality measurement sufficiently described?	Are the QIs transferable to other countries or P-EMSs?	Are limitations of the study discussed?	Are possible sources of bias discussed?	Have the QIs been used in other publications?	Was the feasibility of the QIs evaluated?
Akin Paker [30]	✗	✗	✗	✓	✗	✓	✗	?	✓	✗	✓	✓	✓	?	✓
Anadelic [31]	✓	✓	✓	✗	✗	?	✓	?	✓	✗	✓	✗	✗	?	✓
Arntz [32]	✓	✗	✓	✓	✗	?	✗	?	✓	✓	✓	✗	✗	?	✗
Breckwoldt [33]	✓	✗	✓	✗	✓	?	✗	?	✓	✗	✓	✗	✗	?	✗
Duchateau [34]	✓	✓	✗	✗	✓	?	✗	?	✓	✗	✓	✗	✗	✓	✗
Fjaeldstad [35]	✗	✓	✗	✓	✓	?	✗	?	✓	✗	✓	✓	✗	?	✗
Flabouris [36]	✓	✓	✓	✓	✗	?	✗	?	✓	✓	✓	✓	✓	?	✗
Helm [37]	✓	✓	✓	✗	✓	✓	✗	✗	✓	✗	✓	✓	✗	✓	✗
Helm [38]	✗	✓	✓	✗	✗	?	✗	✗	✓	✗	✓	✓	✗	?	✗
Hennes [39]	✓	✓	✓	✗	✗	?	✗	?	✓	✗	✓	✓	✓	✓	✓
Klemenc-Ketis [40]	✗	✓	✓	✗	✗	✓	✗	✗	✓	✓	✓	✓	✓	?	✗
Leicht [41]	✗	✓	✓	✗	✗	?	✗	✗	✓	✗	✓	✓	✓	?	✗
Lossius [42]	✓	✓	✓	✗	✗	✓	✓	?	✓	✓	✓	✓	✓	✓	✗
Messelken [43]	✓	✓	✓	✓	✓	?	✗	?	✓	✗	✓	✓	✗	✓	✗
Mikkelsen [44]	✓	✓	✓	✓	✓	?	✓	✗	✓	✓	✓	✓	✓	?	✗
Nakstad [45]	✗	✓	✓	✓	✗	✓	✗	?	✓	✓	✓	✗	✓	✓	✗
Neukamm [46]	✗	✓	✗	✗	✓	✓	✗	?	✓	✓	✓	✓	✓	?	✗
Pedersen [47]	✗	✓	✓	✗	✓	✓	✗	✗	✓	✗	✓	✓	✓	✗	✗
Regel [48]	✓	✓	✓	✗	✗	?	✗	?	✓	✗	✓	✓	✓	✗	✗
Rognås [49]	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✗	✗
Rognås [50]	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✓	✗
Sollid [51]	✗	✗	✓	✗	✓	✓	✓	✗	✓	✓	✓	✓	✓	?	✗
Van der Velden [52]	✓	✓	✓	✗	✓	?	✓	✗	✓	✗	✓	✓	✓	?	✗
Viergutz [53]	✓	✓	✓	✓	✓	✓	✗	✗	✓	✗	✓	✓	✓	?	✗
Von Knobelsdorff [54]	✗	✓	✓	✓	✗	?	✗	?	✓	✗	✓	✗	✓	?	✗
Weltermann [55]	✓	✓	✓	✗	✗	?	✗	?	✓	✗	✓	✓	✗	✓	✗
Schlechtriemen [56]	✓	✓	✓	–	✗	–	✗	?	✓	–	✓	✓	✓	?	✗

✓ = yes; ✗ = no; ? = uncertain; – = not relevant.

Discussion

This systematic review identified 27 papers that reported the use of QIs in P-EMS. Fifteen papers used one single QI, and twelve papers applied a set of QIs. Twenty-four of the 27 papers used QIs pertaining to process. Generally, the systematic review demonstrated a lack of a shared understanding of which QI to use in P-EMS.

The lack of agreement on QIs has also been described for regular EMS [57, 58]. This lack of consensus on QI, the heterogeneity of diagnosis, and the challenge of isolating the effect of pre-hospital care from the effect of in-hospital care complicates quality measurement in pre-hospital emergency medicine [59, 60].

The characteristics of the P-EMSs in the identified literature vary and are described sufficiently in only ten papers. This complicates the comparison of studies because the concept of involved P-EMSs remains unclear. Five services use helicopters, eleven use rapid response cars, and another eleven use both transportation modes. Further, nine services are urban, one is rural and eleven are both urban and rural. This mixed representation of transportation modes and urban versus rural profile seems to reflect the heterogeneity of P-EMS. Another relevant aspect of P-EMS is the specialty training and competency of the staffing physicians. Fourteen papers do not report the physicians’ medical specialty. Emergency medicine and anesthesiology are reported as the physicians’ specialties in one and seven papers, respectively. Five studies report a mix of medical specialties in the actual services. All mixes are different, with anesthesiology as the only medical specialty represented in all five papers. Regarding country, eleven of the 27 papers are German. Germany has a long history of P-EMS as an integrated and natural part of the emergency medical system. However, the considerable contribution of German papers is not only because of the service’s long existence. Quality measurement in German hospitals has developed substantially during the last two decades [61], and gradually, German P-EMS has adopted this quality measurement initiative. In addition to a formalized and common understanding of the need for quality measurement in P-EMS [62], the establishment of common documentation systems seems to have been the first necessary step towards quality measurement in German P-EMS and might set an example for QI work in P-EMS in other countries [63, 64].

In 23 papers, the QIs are well defined. However, for many of the QIs identified in this systematic review, the development process does not seem optimal. Ten papers do not clearly describe the methodology for developing the QIs, and only in two papers were the QIs systematically developed by a group of experts. Finally, only seven papers report the professional background and funding of those involved in the development of QIs. A key point in the development of QIs is a systematical and objective approach; this allows for the assessment of the evidence base for the QIs and secures legitimacy. Inadequate QI development may influence the validity, reliability and feasibility of the QIs.

Structure indicators and outcome indicators are used by only two and seven papers, respectively. This may be because process indicators are easier to collect. Routine use of outcome indicators will require automated data exchange between pre- and in-hospital databases, which might not be feasible. This calls for a more integrated electronic patient chart system covering patient data capture through all phases of care. Nevertheless, the major role of process indicators in quality measurement literature of P-EMS is as expected. Process indicators are considered useful for short time frames and when it is difficult to adjust for patient factors [65], and process indicators are therefore particularly relevant for P-EMS. Moreover, process indicators provide a direct assessment of quality of care, as opposed to structure and outcome indicators, which measure care quality by an indirect approach [66]. Process indicators are easy to interpret and well suited for the evaluation of adherence to medical protocols and other quality improvement programmes.

As seen in Table 3, many of the process indicators are time variables. When setting targets and measuring EMS quality, time variables have been widely applied [17, 66]. EMS evolved as a response to the need for rapid access to healthcare in time critical conditions (war injuries, cardiac arrest, major trauma) [66, 67]. However, the current patient population for P-EMS is increasingly heterogeneous, including conditions that are not highly time critical. For a high proportion of patients, shorter pre-hospital time intervals do not improve outcome, and in some cases, longer on-scene time for initial treatment and stabilization is desirable [68, 69]. Accordingly, if P-EMS quality measurement focuses too much on time variables, the results will have poor relevance for a high proportion of the services’ patient population. Moreover, it can lead to an undesired attention shift, resulting in decreased quality for quality dimensions not measured. However, this undesired attention shift is not specific for time variables but rather a universal challenge in quality measurement when the used QIs are few. Finally, our findings indicate that literature pertaining to pre-hospital time variables often lacks information about the competency of the responding unit and the quality of provided care, resulting in even less information from the time variables measured.

Adherence to medical protocols is the most frequently used QI in the included papers. Guidelines and protocols are developed to improve quality of care and to reduce unwanted variation in care. Quality measurement can be defined as measuring the extent to which set targets are achieved [19], and to explore the gap between guidelines and clinical practice is an adequate approach in quality measurement. Ebben et al. [70] demonstrated a wide variation in different EMS professionals’ adherence to guidelines and protocols, indicating that a substantial number of patients do not receive appropriate pre-hospital care. However, this conclusion presumes that there is an evident relationship between adherence to guidelines and patient outcome. Few studies have explored this relationship, and in the review by Ebben et al., only three studies showed that adherence to guidelines improved patient outcomes [71-73]. Finally, it is recommended that guidelines should define QIs to aid monitoring and assessment of guideline adherence [74, 75]. Thus, QIs should ideally be a part of the guideline development process.

For the fifteen papers relying on only one single QI, there is a risk for narrowing the perspective on quality, and important aspects of quality in the actual healthcare service may be ignored. Using a set of mixed QIs that cover different aspects of the service is preferable [76]. Three papers evaluated the care quality of selected ‘tracer conditions’, i.e. high-priority clinical conditions [77]. The evaluation of a service’s response to tracer conditions with condition-specific QIs is used to predict the overall performance of P-EMS. The chosen tracer conditions are not identical, illustrating the following challenge: when different clinical conditions are used for quality measurement, it may complicate the evaluation and comparison of the quality of different P-EMS. To overcome this, the Institute for Healthcare Improvement (IHI) has recommended the use of ‘whole system measures’, defined as a set of QIs aligned with the Institute of Medicine’s (IOM’s) six dimensions of quality, which are not disease- or condition-specific [78, 79]. The six quality dimensions that define high-quality care are timeliness, safety, efficiency, equity, effectiveness and patient-centeredness. Each of these is distinct, and all are equally important. To obtain an adequate and comprehensive quality measurement of P-EMS, future quality measurement should therefore cover these six dimensions as far as possible. This will require the use of multiple QIs, developed specifically for P-EMS. A set of QIs for this purpose has been developed recently, with IOM’s six quality dimensions as the most important part of the conceptual framework [76].

Strengths and limitations

We recognize some strengths and limitations of this study. A strength is that in the literature search, several languages were eligible, allowing inclusion of non-English literature. This is important as eleven of the 27 identified papers were non-English. First, a limitation is that as always in a systematic review, screening, eligibility-check and qualitative synthesis of literature is a product of the review authors’ judgements, allowing subjective interpretations of the content of the studies. Although screening and eligibility assessment were conducted in pairs, data extraction and quality appraisal were primarily conducted by one author. Moreover, no established tools for data extraction or quality appraisal were available. Second, some of the included studies are not explicitly presented as quality measurement studies. However, when screening the literature, we recognized that some studies are quality measurement studies despite not using the quality measurement terminology. A possible reason for this might be the fact that quality measurement terminology is still quite new in P-EMS and not widely used. Accordingly, we have focused on the actual content of the screened literature when deciding if a study should be included—not on the presence or absence of correct quality measurement terminology. Doing this, we have appraised if a paper ‘concerns an initiative to improve healthcare; broadly defined to include the quality, safety, effectiveness, patient-centeredness, timeliness, cost, efficiency and equity of healthcare’, as stated by the SQUIRE guidelines [80]. This approach was chosen to avoid overlooking potentially relevant aspects of quality measurement studies in P-EMS.

Conclusion

This systematic literature review served the purpose of identifying, describing and evaluating studies of quality measurement in P-EMS. The review demonstrated a lack of a shared understanding of which QIs to use in P-EMS. Process indicators were dominant in the included papers, and the most emphasized QIs were ‘Adherence to medical protocols’, ‘Provision of advanced interventions’, ‘Response time’ and ‘Adverse events’. Moreover, fifteen of the 27 papers used only a single QI to measure quality, thus increasing the risk of a narrow perspective on quality. The remaining papers used multiple QIs in their quality measurement, which is considered preferable.

Future quality measurement in P-EMS should rely on a set of consensus-based QIs, securing a comprehensive approach to quality measurement and offering the possibility of comparing results from different P-EMS systems.

Click here for additional data file.

Click here for additional data file.