Developing metrics for emergency care research in low- and middle-income countries.

INTRODUCTION: There is little research on emergency care delivery in low- and middle-income countries (LMICs). To facilitate future research, we aimed to assess the set of key metrics currently used by researchers in these settings and to propose a set of standard metrics to facilitate future research.
METHODS: Systematic literature review of 43,109 published reports on general emergency care from 139 LMICs. Studies describing care for subsets of emergency conditions, subsets of populations, and data aggregated across multiple facilities were excluded. All facility- and patient-level statistics reported in these studies were recorded and the most commonly used metrics were identified.
RESULTS: We identified 195 studies on emergency care delivery in LMICs. There was little uniformity in either patient- or facility-level metrics reported. Patient demographics were inconsistently reported: only 33% noted average age and 63% the gender breakdown. The upper age boundary used for paediatric data varied widely, from 5 to 20 years of age. Emergency centre capacity was reported using a variety of metrics including annual patient volume (n = 175, 90%); bed count (n = 60, 31%), number of rooms (n = 48, 25%); frequently none of these metrics were reported (n = 16, 8%). Many characteristics essential to describe capabilities and performance of emergency care were not reported, including use and type of triage; level of provider training; admission rate; time to evaluation; and length of EC stay.
CONCLUSION: We found considerable heterogeneity in reporting practices for studies of emergency care in LMICs. Standardised metrics could facilitate future analysis and interpretation of such studies, and expand the ability to generalise and compare findings across emergency care settings.

African relevance

Emergency systems development is a foundational part of developing emergency care in Africa.

Emergency care research in Africa is small in comparison to the rest of the world.

Defining key metrics used concurrently throughout African emergency care research is important to facilitate future research.

Introduction

Emergency medicine plays an integral role in many health systems around the world, primarily in high-income countries.1, 2, 3, 4, 5, 6, 7 There is mounting evidence that high-quality emergency care has the potential to address a significant proportion of the global burden of disease, as advocates have called for the development of emergency care in low- and middle-income countries (LMICs).8, 9, 10, 11, 12, 13, 14, 15, 16 Recent outbreaks of pandemic infectious diseases such as MERS-CoV in the Middle East and Asia, and Ebola in West Africa highlighted the need for equipped emergency care facilities, staffed with trained personnel to stem the tide of such outbreaks and to form the front lines in the treatment of more common but increasingly important conditions such as non-communicable disease (NCDs) and injuries in LMICs.

Despite the compelling need for more data on emergency care in LMICs, research in this area has been largely neglected. A 2015 systematic review of emergency facilities in LMICs found no published reports on emergency care in over half of LMICs. Where data were available, the review found that only a small set of metrics on emergency care delivery was reported consistently across facilities, and that researchers frequently used a wide array of ill-defined measures to describe EC characteristics and performance. This inconsistency has complicated inter-facility data comparison and study replication.16, 19 More data are needed to understand current capabilities, expose deficits, and ultimately improve emergency care delivery in these resource-constrained settings.10, 16, 18, 19, 20

In this study, we identify a set of key metrics commonly used by researchers to describe emergency care in LMICs and propose a standard set of data elements that would be practical to collect. A consensus on basic terminology and methodology has advanced the field of emergency medicine in high-income countries, and a clearly defined core set of metrics for describing emergency care in LMICs would similarly not only advance local emergency care research and quality improvement, but also allow for more effective cross pollination and systems development.

Methods

We conducted a systematic review (PROSPERO registry: CRD42014007617) to identify published reports describing general emergency care delivered to an undifferentiated patient population in all 139 LMICs. For each LMIC, we searched PubMed, CINAHL, and all World Health Organization (WHO) regional indices using “[country name] + emerg∗” as the search term. We performed a similar search on Google Scholar, but limited the search to within article titles given the large number of results. We also manually screened select non-indexed journals known to frequently publish research on emergency care. Reference lists of all studies included were further screened manually.

Results were screened by title and abstract, and selected for inclusion if they described facility-based emergency care provided to all patients, regardless of disease category or chief complaint. Articles published after 1989 in all languages were included provided an English or French abstract was available. We excluded studies focussed only on specific emergent conditions (e.g., stroke only), subsets of the general emergency patient population (e.g., women only), or data aggregated from multiple departments or facilities, unless they provided general emergency facility statistics or data on the burden of diseases.

We evaluated the reporting frequency of data elements commonly found in published emergency care literature originating from LMICs. We created a database that continually expanded with new data fields as the systematic review progressed and new reported metrics were encountered. At the conclusion of the review, we selected publications that provided comprehensive descriptions of their facilities and patients, and used them as models to structure our recommendations for future research.

At the conclusion of our review, we presented our findings at the African Federation of Emergency Medicine consensus conferences, Addis Ababa 2014 and Cape Town 2015. Small group discussion amongst attendees provided invaluable insight into some of the local determinants and limitations to data collection and publication in certain settings. We incorporated lessons learnt from the group discussions on how to improve data standards into our recommendations. We highlight specific information to be recorded and reported by individual emergency facilities, to enable reliable inter-facility data comparisons and expose areas for improvement in specific locales.

Results

Fig. 1 shows the search strategy used to screen 43,109 published reports with 195 studies meeting our inclusion criteria. This resulted in descriptions of 192 unique facilities in 139 LMICs, as shown geographically in the map in Fig. 2. Fig. 3 presents the proportion of these publications that reported the data elements of interest.

Figure 1

The flow diagram shows the results of the database and journal searches, acquisition of unpublished data, and application of exclusion criteria.

Figure 2

Number of publications, geocoded to location of facilities described (Lighter colour indicates more publications. High income countries were removed from the map to show only LMICs.

Figure 3

Proportion (%) of publications reporting data.

As a whole, hospital characteristics were nearly universally reported, with the exception of whether it was located in an urban or rural setting. As a result, researchers curious as to the kinds of patients the facility served or its geographic access would have to use mapping software (e.g., Google Maps) to find an approximate location for the facility. Similarly, indicators of overall hospital size and patient capacity, such as the number of inpatient beds, were infrequently reported; however, could be found occasionally on the hospital webpage, if one existed. The physical layout of the emergency centre itself, including the number of emergency centre beds, was documented more often. The annual patient volume in the emergency centre was the most commonly reported marker for facility size, but the simultaneous reporting of the catchment area for the hospital or the number of outpatient visits per year was rare.

Although half of the studies referenced the availability of triage, further details on the processes for stratification of patient acuity were not routinely provided. For example, only 20% described the level of training of the healthcare provider performing the triage assessment and even less frequently the protocol used, if any. The level of training for the physicians staffing the EC was reported in only half of the publications, and less so for nurse staffing.

Approximately one-fifth of publications were general facility descriptions, without individual emergency facility patient sampling. When patient-level data for those study subjects presenting for emergency care were utilised, 27% were prospective observational or cross-sectional studies and 73% retrospective studies. Continuous sampling was the most commonly used method to gather data on study subjects. But overall, when employing other sampling methods, there was lack of clarity and detail in the study methods to explain the procedures used to ensure adequate selection of a representative study sample.

The emergency patient population was defined as paediatric, adult, or general in 90% of reports, 33% reported median or average age for patients, and 63% the proportion of either sex. Fig. 4 shows the frequency of chosen paediatric to adult age cut-offs used by facilities in our review. While the median age was 15 (IQR 14–18), the range was wide from 5 (4 studies in 3 countries)21, 22, 23, 24 to 20 years of age (1 study in 1 country).

Figure 4

Distribution of paediatric to adult age cut-offs found in emergency care literature originating from LMICs.

Patient outcomes of emergency care, which would serve as a marker for the standard of acute care delivery at each facility, were poorly reported limiting the reader’s ability to draw accurate conclusions from the published reports. Perhaps the most objective and useful metrics for emergency care- mortality rates at specific time interval, such as 24-h or 48-h mortality- were only available in 10% of publications.

The recording of diagnoses was hallmarked by the same lack of consistency as the data describing emergency care. Only 10% of reports used specific diagnosis coding systems, such as ICD-10 Clinical Classification Software. Frequent misclassification, inconsistent, or double classification of conditions can result in over- or underestimation of key conditions. Table 1 lists examples of inconsistencies in the categorisation of various diagnoses and causes of death. Articles commonly classified localized infections with their respective organ system—but also included a separate infectious disease category, presumably for systemic infections (e.g., meningitis and cerebral malaria categorised with the central nervous system, while malaria, tetanus, septicaemia and enteric fever with “various infections”). Similarly, traumatic conditions could be categorised with the organ system affected and with a general trauma category.

Table 1

Burden of disease reporting inconsistencies.

Inconsistencies in reporting and categorisation of burden of disease metrics	Examples
Diagnosis
Infectious disease
Localized infections included with respective organ system categories	Pneumonia categorised with ‘respiratory disease’ (rather than infectious)27
Infectious disease categorised separately from organ system categories	Malaria and pneumonia categorised with “all infectious complaints”28
Inconsistent reporting of infectious diagnoses within a single study	Infective endocarditis categorised with cardiovascular disease, amoebic liver abscess with gastrointestinal disease, but pneumonia, liver abscess, and amoebic dysentery categorised with infectious disease29
Trauma
Specific traumatic diagnoses reported with respective organ system despite separate trauma category	Abdominal bruising and wounds categorised with abdominal conditions despite an abdomen subcategory for trauma30
Cause of death
Infectious disease
Infectious causes included with respective organ system	Meningitis categorised with neurological disease31
Inconsistent reporting of infectious causes within a single study	Cerebral malaria categorised with neurological disease, but also a separate severe malaria category32
Surgical
Malignancies classified as surgical causes of death	Prostate and breast cancer categorised with surgical causes of death31

Discussion

Our review uncovered considerable heterogeneity in reporting practices for studies of emergency care in LMICs. As a result, these research efforts, despite presenting valuable statistics and important healthcare metrics, had limited utility when comparing emergency care delivery and outcomes across different settings and determining trends in time.

Specific deficiencies were identified in data elements that required obtaining data from other departments. For example, inpatient beds (10%) and total outpatient visits per year (5%) were underreported. Metrics obtained directly from within the footprint of the EC were reported more reliably, for example annual EC patient volume (90%) was consistently reported as a measure of EC capacity. Similarly, patient events that occur outside the EC are infrequently reported in publications of emergency care in LMICs. Rates of 48 h mortality (10%) and need for subspecialty care (10%) had low-rates of reporting possibly because they do not occur within the patient’s EC stay, while data such as type of population served (90%) and gender (63%) are both readily obtained during the EC stay itself and thus more frequently reported. It is instructive to note that studies were 3–4× as likely to report important metrics like patient mortality in the EC, compared to reporting that required follow up of patients after disposition to the inpatient service. This may be the result of weaknesses in hospital-wide health information systems in most facilities in LMICs, resulting in inability to consistently capture data on these events as part of the emergency care data recorded. Finally, the rate of patients “brought-in-dead” was only reported 7% of the time—this complicates interpretation of in-EC mortality rates, since some hospitals may report these patients as having died in the EC. This raises questions regarding the consistency of definitions used for these categories and the lack of transparency in the practices regarding reporting mortality statistics in these facilities.

Several important functions of emergency care were also underreported in the studies we reviewed, complicating interpretation of data and its use for future research. First, patient triage is an indispensable first step to high quality emergency care; yet, it was only referenced in approximately one quarter of reports from LMICs. Even fewer publications provided details on triage processes, which are likely to be variable33, 34; indeed, several studies have demonstrated a significant improvement in patient outcomes after the implementation of structured triage.21, 22, 35 Second, the training level for physicians staffing the facility was noted in approximately half of reports. Lack of senior level supervision of physicians in training has been linked to worse patient outcomes,36, 37, 38 Reporting the level of training of physicians and nurses staffing facilities in LMICs is essential to understanding the capabilities to provide various services and for accurately mapping the human resource gaps.

Other statistics reported in the literature used non-uniform parameters, which made it difficult to aggregate data across facilities. For example, the reporting of in-EC mortality rate was the most frequent measure of death; however, the majority of studies failed to also report patient lengths of stay. When lengths of stays were available, the atypical ranges – up to 117 days in a facility in Brazil – and failure to report means and medians for the data made it difficult to compare emergency care across facilities. This heterogeneity in reporting such a critical data element reinforces the importance of having a standard metric for death, such as 24-h mortality.

Likewise, length of stay data were often reported with respect to specific and varying time intervals (e.g., >24 h), rather than averages, complicating comparative analyses.40, 41, 42 Paediatric-adult age cut-offs chosen by individual facilities also varied widely (Fig. 1). This distorted the appropriate classification of these patient groups across different studies and made it difficult to assess the key differences in patient characteristics, their burden of disease, and clinical outcomes.

In addition, there was also discrepancy in reporting burden of disease information. Traumatic and infectious conditions were variably reported with their respective organ systems, i.e., pneumonia categorised with pulmonary disease, or they were grouped together under a broad infectious disease category. This inconsistency in disease categorisation even occurred within a single publication (Table 1). Ideally, having all infectious and traumatic conditions grouped together as standard practice would allow for an accurate understanding of the true burden of disease.

In order to help guide emergency care clinicians and researchers in LMICs, we present a template for manuscript preparation that provides recommendations for how data can be collected, analysed, and reported in a standardised fashion. Consistent research practices across different emergency care settings would maximise the utility of gathered information and allow for more meaningful conclusions to be drawn from aggregated comparable data that could not be gathered in as large a volume from one facility. Furthermore, the ability to aggregate data and to generalise results will make emergency care research in LMICs more useful to other emergency care practitioners who will more confidently be able to assess the ability to translate research findings into their own clinical setting.

Table 2 provides a set of recommendations that stemmed from our systematic review. A publication template can also be found in the Online supplement. Insights from small group discussions at the African Federation of Emergency Medicine consensus conferences have also been incorporated into these recommendations.

Table 2

Recommendations for data collection, analysis, and reporting based on findings from the systematic review.

Data category	Recommendations and examples
Hospital characteristics
Urban–rural status	• Geographic location of facility • Urban, sub-urban, or rural
Facility ownership	• Funding organisation or governing body type • For-profit, non-profit, government or teaching hospital
Size
Inpatient bed count	• Number of inpatient hospital beds available for admissions
Number of outpatient visits per year	• Number outpatient department (OPD) clinic visits annually (separate from emergency visits)
Catchment area	• Physical area and size of the population from which patient population is drawn
Emergency facility characteristics
Size
Layout and number of rooms	• Organisation of the emergency facility • Number of clinical rooms by typeo Treatment or procedure rooms o Resuscitation unit o Observation unit o All other clinical areas
Bed count	• Number of emergency centre beds available for patient care
Annual EC patient volume	• Number of patient visits to the emergency facility annually
Intake
Mode of arrival	• Proportion of patients arriving to the facility by various modes of transport, i.e., ambulance, private care, public transport, or any other methods used
Intake area	• Presence or absence of dedicated portal of entry for emergency care
Triage
Availability	• Use of initial patient assessment at presentation to determine acuity of complaints and prioritisation of medical care
Location	• Physical location where triage assessment occurs, i.e., a separate clinical room at the entrance of the emergency facility
Triage officer	• Identify the type of healthcare worker responsible for stratifying patient acuity, i.e., a physician, a nurse, or another healthcare provider
Protocol used	• The guidelines used for triage assessment, i.e., the World Health Organization’s Emergency Triage Assessment and Treatment (ETAT)
Disposition from triage	• Outcome for subsequent patient care, after triage assessment, i.e., whether all patients are treated in the emergency facility prioritising the most urgent cases, or if some patients are discharged directly from triage with or without treatment
Physician staffing	• Highest level of training for physicians providing patient care in the emergency facility • Stratify by hours of the day and days of the week, emergency medicine attending coverage from 0700–1900 and emergency medicine resident only coverage from 1900–0700
Nurse staffing	• Highest level of training of nurses in the emergency facility, • Document specialty certification or training in emergency care
Study design
Sampling method
Continuous sampling	• All patients presenting to the emergency facility over a specific time interval are chosen for inclusion in the study and is the preferred method of sampling
Simple random sampling	• From the total number of patients visiting the emergency facility over a specific time interval, a predetermined number of study subjects are chosen at random for inclusion in the study- the method used to ensure adequate randomisation should be presented
Systematic random sampling	• Selection of a repeating interval of patients, i.e., every fourth patient presenting to the emergency facility over a specific time interval is chosen for inclusion in the study
Convenience sampling	• Subgroup of total patients presenting to EC, i.e., only between specified hours or over a specific time interval are chosen for inclusion in the study- potential bias to study results should be noted in the discussion section if using convenience sampling
Study population	• Total number of patients available for inclusion in the study initially, and the final number of study subjects selected for inclusion in analyses
Patient information
Patient demographics
Population served	• Type of patients treated in the facility, i.e., paediatric patients only, adults only, or all patients
Patient age	• Report mean, median, and range of patient ages • When stratifying patient data by age (demographic information, burden of disease data, and patient outcomes) we recommend using paediatric-adult age cut-off of 15 years of age • Within the paediatric age group, patients 5 years of age or less may also be grouped separately
Patient outcomes
Length of stay	• Report mean, median, and range for length of EC stay
Inpatient admission rate	• Proportion of emergency patients admitted to the inpatient service from the EC
Mortality rate over a time interval, i.e., 24-h mortality rate	• Number of deaths over the total number of visits to the emergency facility over a time interval from presentation is the preferred metric for death, and this requires patient follow up even after they leave the EC
Mortality rate in the emergency facility	• Number of patients that died in the EC over the total number of visits with simultaneous reporting of the mean, median, and range for patient length of stay in the facility
Patients brought-in-dead	• Number of patients that died BEFORE receiving any treatment in EC • To be reported separately from deaths that occur during or after treatment in the EC
Burden of disease
Chief complaint	• Presenting complaints reported by patients upon arrival to the EC
EC Diagnosis	• Provisional diagnosis made by healthcare provider after completion of EC care
Cause of death	• Cause of death determined by healthcare provider for patients who die in EC
Infectious disease	• All infectious disease, whether localized or systemic, should be grouped together under a separate infectious disease category, and not with the affected organ system
Traumatic conditions	• All traumatic injuries should be grouped together under a separate trauma category, and not with the affected organ system

EC, emergency centre.

Number of clinical rooms by type

In conclusion, the data elements needed to provide an accurate understanding of the current state of emergency care in LMICs are not being routinely captured. The information that is collected is confounded by dissimilar practices for data acquisition and reporting. Many LMICs lack designated emergency centres altogether, and therefore the ability to accurately describe emergent patient presentations and the acute care delivered is lost. Researchers are bound by the constraints of previously collected administrative data, and lack the tools to initiate prospective data collection in order to answer clinical questions regarding the emergent and life-saving care they provide for their patients. Our goals are to increase the scope of publications from LMICs; facilitate data collection and analysis; improve the ability to translate findings of LMIC emergency care research to other similar settings; gain a better understanding of the burden of disease; and potentially improve patient outcomes by reducing the barriers to clinical research. To do so, we provide a framework for reporting results of emergency care research that is simple to use and which has been tailored to resource-constrained settings in LMICs.

The capture of standard data elements and use of common definitions for emergency care indicators facilitates the production of collaborative research networks, and valuable insights can result from analysis of aggregated emergency care data these network produce. Such shared methodology will result in a rapidly growing body of literature capable of painting a clearer picture of the current state of emergency care to inform policymaker decisions. We urge all emergency care researchers in LMICs to adopt such practices to facilitate the growth of such collaborations to help produce the evidence needed to combat the growing global burden of emergency conditions.

Our review aimed to identify all published emergency care literature originating from LMICs; however, our search strategy may have neglected reports that used alternative terminology to signify acute care medicine delivery without using the word stem ‘emerg-’ in the title, keywords, or abstract of articles. The overall paucity of studies that met our inclusion criteria remains our biggest limitation. Our goal during data collection was to record all reported data elements, but the overall infrequency of reporting outside of a few metrics, such as patient volume and demographics, limited the power to draw accurate conclusions from our results. We also acknowledge that the limited data available to researchers, likely related to the overall use of retrospective data collection relying on existing administrative records, is an overarching limitation to this research process, and one that our recommendations aim to change.

Conflicts of interest

The authors declare no conflicts of interest.

Dissemination of results

Results from this research were presented to attendees of the African Federation of Emergency Medicine consensus conferences, Addis Ababa 2014 and Cape Town 2015, both to the general assembly and small group discussion boards. Insights from the group discussions on how to improve data standards were incorporated into our recommendations.

Authors’ contributions

All authors provided substantial contribution to the conception of the work. SA, CYC, and ZO acquired and analysed the data for the study, and all authors interpreted of the data. SA and CYC prepared the manuscript and ZO, TAR, and HM revised the work. All authors gave their final approval of the version to be published.