Overcoming challenges of establishing a hospital-based out-of-hospital cardiac arrest registry: accuracy of case identification using administrative data and clinical registries.

INTRODUCTION: Comprehensive identification of out-of-hospital cardiac arrest (OHCA) cases for inclusion in registries remains challenging due to the inherent diversity of OHCA aetiology, presentation, and management. The Northern Adelaide Local Health Network (NALHN) OHCA registry identifies OHCAs presenting to NALHN hospitals using existing data sources to monitor in-hospital treatment and survival. This study aimed to investigate the accuracy of hospital-based data sources for identifying OHCA cases treated at hospital.
METHODS: Retrospective analysis of all OHCAs aged >18 years included in the NALHN OHCA registry between 2011-16. Registry cases are identified from an emergency medical service (EMS) OHCA registry, Emergency Department (ED) and ICD-10 coding datasets, and key-word searches of two in-hospital clinical registries. Sensitivity and positive predictive values (PPV) of each hospital-based data source were analysed with respect to (a) the number of cases expected to be identified by that source, (b) total OHCA. Non-OHCAs yielded by each source were explored and a sub-analysis of ICD-10 codes was performed.
RESULTS: Between 2011-16, the four hospital-based sources yielded 992 cases, of which 383 were confirmed as OHCA. The ED coding dataset was the most accurate with a sensitivity and PPV of 78%. The ICD-10 coding dataset had good sensitivity but low PPV (33%). The ED coding dataset, combined with the two in-hospital clinical registries, identified 93% of OHCAs.
CONCLUSIONS: No single dataset identified all OHCAs presenting to NALHN hospitals. Combined hospital-based data sources provide a valid method of identifying OHCAs treated at hospital that may be adapted to augment EMS-based data.

Introduction

Out-of-hospital cardiac arrest (OHCA) survival in Australia is only 12% and remains a challenge in medicine. The Global Resuscitation Alliance statement proposes registries as a primary mechanism to improve cardiac arrest survival. Compared with procedure-based registries which have one common entry point (e.g. the procedure), injury- and disease-based registries may have multiple entry points where cases are identified by clinicians or administrative data. The majority of OHCA registries comprise of cases identified by emergency medical services (EMS) but most exclude non-EMS attended OHCAs and lack data on in-hospital management.3, 4, 5, 6, 7, 8, 9 Administrative data, such as the International Statistical Classification of Diseases and Related Health Problems, tenth revision (ICD-10) codes, are designed to allow international comparisons in reporting health trends and statistics but do not reliably identify OHCAs.10, 11 Registries that use comprehensive and valid methods of case identification facilitate improved understanding of epidemiology, process of care, and outcome but remain a particular challenge for OHCA. This paper describes the methods used to identify OHCA cases included in the hospital-based Northern Adelaide Local Health Network (NALHN) OHCA registry. The primary objective was to determine the accuracy of each hospital-based source for identifying OHCA presenting to NALHN hospitals with respect to (a) expected OHCAs (e.g. ICU dataset was analysed with respect to patients admitted to ICU) and (b) total OHCA. Cases identified as OHCA were confirmed by manual hospital medical record review. Secondary objectives included classification of non-OHCAs yielded by each source, and a sub-analysis of ICD-10 coding in admitted patients. Our findings may provide insights for others establishing hospital-based OHCA registries.

Methods

Study design

This is a retrospective analysis of methods used to identify cases for inclusion in the NALHN OHCA registry, a hospital-based cohort registry of all OHCAs aged ≥18 years treated at NALHN hospitals from 2011 onward. Registry variables include all core elements of the Utstein template with additional items on cardiac management, neurological prognostication, and aetiology.

Setting

NALHN comprises two public hospitals that service the northern metropolitan area of Adelaide, South Australia. The Lyell McEwin Hospital is the primary cardiac arrest centre with 15 Intensive Care Unit (ICU) and 26 cardiac unit beds, and Modbury Hospital is a secondary teaching hospital. Combined, there are approximately 200,000 presentations per year to the ED and over 61,000 in-patient admissions. South Australia has a single state-wide two-tier EMS where OHCA patients are treated by SAAS paramedics on-scene. Both receiving hospitals have a resuscitation area in the ED with a multidisciplinary health team led by ED specialist physicians. Traumatic arrests are generally retrieved to an external tertiary hospital for further management once stabilised by EMS or ED staff. SAAS and NALHN hospitals follow the 2010 (revised in 2015) ANZCOR resuscitation guidelines.

Inclusion and exclusion criteria

Cardiac arrest is defined as the absence of signs of circulation.13, 14 According to the Utstein definition we included all OHCAs aged ≥18 years receiving chest compressions or external defibrillation, whether for severe bradycardia with pulses and poor perfusion, as well as ROSC pre-EMS or Medical Emergency Team (MET) arrival. OHCA was defined as cardiac arrest occurring in individuals who do not occupy an ED or inpatient hospital bed. This definition distinguishes between patients who are not receiving advanced care at the time of cardiac arrest and those already under advanced care that aims to (1) prevent cardiac arrest, and (2) provide immediate and timely resuscitation if required, rather than a distinction based on emergency responder (EMS vs. MET). The registry therefore includes all arrests transported to hospital by EMS or private car, including arrests in the community, medical clinics, rehabilitation facilities, nursing homes, and inter-hospital transfers, as well as any arrest on hospital grounds involving staff, visitors, and outpatients, including arrests in the ED waiting room or ambulance bay prior to handover to an ED physician. Paediatric (<18 years) cardiac arrests were excluded because NALHN does not have a paediatric Intensive Care Unit (ICU) and patients are generally retrieved to an external tertiary hospital once stabilised. Automated Implantable Cardioverter-Defibrillator (AICD) shocks without the need for bystander CPR or ongoing resuscitation were also excluded.

Data sources

EMS-based registry

Data linkage with SAAS-CAR was limited to EMS-attended OHCAs occurring within a NALHN catchment postcode and received by a NALHN hospital. Cases were manually linked using age, sex, arrest date, and time of call. Data were not available prior to 2012.

Administrative datasets

The ICD-10 Australian modification (ICD-10-AM) is an expanded version of the World Health Organisation’s ICD-10. The ED coding dataset comprised of the Emergency Department ICD-10-AM Principal Diagnosis Short List (ED Short List) code of cardiac arrest (I46.9) and presenting complaint code of cardiac arrest (0102) extracted from HASS EDIS, a real-time patient tracking tool used in EDs across Australia. The diagnosis and presenting complaint codes were entered from a pull-down menu by the treating doctor and triage nurse, respectively.

The ICD-10 dataset includes the following primary or secondary diagnoses assigned to billable in-patient encounters by clinical coders according to the Australian Coding Standards at the primary treating hospital: Cardiac arrest with successful resuscitation (I46.0), Sudden cardiac death (I46.1), Cardiac arrest unspecified (I46.9), Ventricular fibrillation and ventricular flutter (I49.0), Respiratory arrest (R09.2), and Asphyxiation (T71). We included T71 to maximise sensitivity but excluded Ventricular tachycardia (I47.2) from the final coding set due to the high yield and low expected true positive rates. A post-hoc search of I47.2 yielded 608 cases from 2011 to 2016, of which 75 (12%) were matched to existing cases in the NALHN OHCA registry.

Clinical registries

Cardiac catheterisation registries included the cardiac catheterisation record book, the ‘Code STEMI’ database, and the Coronary Angiogram Database of South Australia (CADOSA),16, 17 and were searched for “OHCA” or similar terms. NAHLN participates in the Australian and New Zealand Intensive Care Society Adult Patient Database (ANZICS APD), an Australia-wide ICU registry used for benchmarking of individual ICU performance. The registry had been used retrospectively to identify an OHCA cohort admitted between 2011–2015, and for 2016 was searched using the diagnosis ‘cardiac arrest’ or ‘respiratory arrest’.

Case identification method and quality control

The medical record number, admission date, name, and date of birth of each case identified from the hospital-based source was recorded, then (1) cases identified as included or excluded from each source were labelled in each consecutive source to avoid duplication, (2) age-based exclusion, (3) exclusion based on electronic summary, (4) inclusion based on manual hospital medical record review, (5) unique identification number assigned; identifiers stored in separate electronic file, (6) medical record abstraction; data import from linked sources, (7) annual review, training, and ongoing education for all data variables and definitions; annual monitoring of 10% of records; inbuilt database checks. Clinician-led identification was not utilised due to constraints in resources and availability of existing data sources.

Statistical analysis

Sensitivity and positive predictive values (PPV) of each hospital-based data source used to identify cases for the NALHN OHCA registry were investigated between 2011–16. The proportion of OHCA and non-OHCA cases yielded from each hospital-based data source was compared to (a) the number of OHCAs expected to be identified by each source, e.g., by limiting the analysis of the ED dataset to cases admitted to ED, and (b) total OHCA. The exact proportion of true negatives for each source was not calculated; however, specificity and negative predictive values were >95% for each data source using estimations of annual ED presentations, in-patient admissions, ICU admissions, and cardiac catheterisation procedures, respectively. A sensitivity analysis was performed excluding cases with ROSC pre-EMS and non-EMS attended OHCAs and accuracy for each dataset was compared. Standards for reporting diagnostic accuracy studies (STARD) were followed. The classification of non-OHCAs yielded by each source was also explored and a sub-analysis of ICD-10 coding was performed for admitted patients. Analyses were performed using MedCalc Statistical Software version 19.2 (MedCalc Software bv, Ostend, Belgium).

Results

Between 2011–16, the NALHN OHCA registry included 393 OHCAs confirmed by manual hospital record review. Patient characteristics are presented for the total cohort (Table 1) and sensitivity analysis inclusion and exclusion groups (Table S1).

Table 1

Characteristics of all adult OHCA cases arriving to NALHN facilities 2011–16, n = 393.

Age	60 ± 18
Male gender	262 (67%)
Possible syncopal episode with CPR (e.g. unmonitored arrest, bradyarrhythmia)	23 (6%)
Arrest location
Home/residence (EMS-attended)	246 (63%)
Other (EMS-attended)	126 (32%)
Vehicle/carpark (non-EMS attended)	14 (4%)
Hospital grounds (non-EMS attended)	6 (2%)
Arrest within NALHN catchment postcode	321 (82%)
Witnessed
Bystander	194 (49%)
Medical	94 (24%)
Unwitnessed	105 (27%)
Bystander CPR	216/299 (72%)
Initial shockable rhythm	183 (47%)
Sustained ROSC	351 (89%)
ROSC pre-SAAS	20 (5%)
Presenting Emergency Department
Lyell McEwin Hospital	317 (81%)
Modbury Hospital	40 (10%)
Non-NALHN	36 (9%)
Presumed cardiac aetiology on arrival to emergency	285 (73%)
Glasgow coma scale >3 on arrival to emergency	105/390 (27%)
Admitted to NALHN facility	316 (80%)
Primary treating: Lyell McEwin Hospital	313 (99%)
Primary treating: Modbury Hospital	3 (1%)
Coronary angiography at Lyell McEwin Hospital	163 (42%)
Targeted temperature management (pre- and in-hospital)	143 (36%)
Admitted to NALHN intensive care/critical care unit	253 (64%)
NALHN discharge disposition
Retrieved to acute care facility <24 h	12 (3%)
Deceased in NALHN Emergency Department	69 (18%)
NALHN inpatient — surviveda	148 (38%)
NALHN inpatient — deceaseda	164 (42%)
Aetiology of arrest according to hospital medical record or autopsy
Cardiac	203 (52%)
Respiratory	72 (18%)
Neurological	12 (3%)
Toxicological	24 (6%)
Other	38 (10%)
Unknown	44 (11%)
Overall survival to hospital dischargea	170 (43%)

Data presented as mean ± standard deviation, or count (percentage). Percentages may not add up to 100% due to rounding.

CPR, cardiopulmonary resuscitation; EMS, emergency medical service; NALHN, Northern Adelaide Local Health Network; ROSC, return of spontaneous circulation.

Includes patients retrieved to non-NALHN hospital(s) during episode of care.

Hospital-based source accuracy

The four hospital-based data sources used to identify cases yielded 992 potential cases, of which 383 were true OHCA (Fig. 1). The EMS reference source yielded an additional 10 (3%) unique OHCAs between 2012–16. Of the 257 cases that arrested within a NALHN postcode and were attended by EMS between 2012–16, 195 (77%) were identified by the EMS registry (see Table S1 for characteristics of cases ‘missed’ by SAAS-CAR). The number of cases yielded, true positives, and accuracy for each source are presented with respect to the number of cases expected to be identified by each source (Table 2) and total OHCA (Table 3). The ED coding dataset was sensitive for both OHCAs admitted to ED (85%), and total OHCA (78%), while the ICD-10 coding dataset had a sensitivity of 79% for admitted OHCAs at the cost of low PPV (33%). Combining the ED coding dataset with the two clinical registries identified 93% of total OHCAs. The sensitivity analysis revealed similar sensitivity and PPVs for each dataset (Tables S2 and S3).

Fig. 1

Cases in the NALHN OHCA registry identified from hospital-based and EMS-based sources. Data displayed is from 2011 to 2016. Cases yielded from each source are cross-referenced with each subsequent source to avoid duplication. Data from the EMS-based source was limited to arrests within NALHN catchment postcode with transport to NALHN hospital between 2012–2016. EMS, emergency medical service; ICU, Intensive Care Unit; OHCA, out-of-hospital cardiac arrest; NALHN, Northern Adelaide Local Health Network.

Table 2

Accuracy of hospital-based sources with respect to total OHCA cases in the NALHN OHCA registry expected to be identified by each source, 2011–16.

	Total yield (n)	True positive (n)	Total OHCA (n)	Sensitivity (%)	PPV (%)
Emergency Department coding dataset	390	305	357 admitted to Emergency Department	85.4	78.2
Cardiac catheterisation registries	197	145	163 underwent cardiac catheterisation	88.96	73.6
Intensive Care Unit registry	250	228	253 admitted to Intensive Care Unit	90.1	91.2
ICD-10 coding dataset	760	251	316 admitted as in-patient	79.4	33.0

ED, Emergency Department; ICU, Intensive Care Unit. Sensitivity = true positive/total OHCA; PPV, Positive Predictive Value = true positive/total yield. Refer Fig. 2 for search criteria used to generate each source.

Table 3

Accuracy of hospital-based sources with respect to total OHCA in the NALHN OHCA registry, 2011–16.

	Total yield (n)	True positive (n)	Total OHCA (n)	Sensitivity (%)	PPV (%)
Emergency Department coding dataset	393	308	393	78.4	78.4
Cardiac catheterisation registries	209	157	393	40.0	75.1
Intensive Care Unit registry	250	228	393	58.0	91.2
ICD-10 coding dataset	760	251	393	63.9	33.0

Sensitivity = true positive/total OHCA; PPV, Positive Predictive Value = true positive/total yield. Refer Fig. 2 for search criteria used to generate each source.

Fig. 2

Primary ICD-10 code categories assigned to out-of-hospital cardiac arrest in-patient encounters grouped according to major diagnosis groups (n = 316).

Non-OHCAs

Table 4 presents a categorisation of cases yielded by each hospital-based source that were not OHCA. ED arrests were further defined as arrests occurring in patients occupying an ED bed or under active care by emergency physicians after handover from EMS or MET and prior to in-patient hospital admission. Of note, the ICD-10 coding dataset yielded more in-hospital cardiac arrests than OHCAs (291 vs. 251), and the ED coding dataset yielded a high proportion of ED arrests (48%). Cases in the ‘not cardiac arrest’ group represented a broad range of diagnoses such as conscious arrhythmias and unconscious collapse.

Table 4

Excluded cases yielded by each hospital-based source searched for potential OHCA cases.

	<18 years of age	ED arrest	In-hospital arrest	AICD shock	Not cardiac arrest
Emergency Department coding dataset (n = 85)	17 (20%)	41 (48%)	4 (5%)	-	23 (27%)
Cardiac catheterisation registries (n = 52)	1 (2%)	17 (33%)	21 (40%)	2 (4%)	11 (21%)
Intensive Care Unit registry (n = 22)	2 (9%)	9 (41%)	10 (45%)	-	1 (5%)
ICD-10 coding dataset (n = 509)	12 (2%)	46 (9%)	291 (57%)	17 (3%)	143 (28%)

Data presented as n (% total non-OHCA). Bold values represent highest values of excluded cases (non-OHCA) for each source. Note: percentages may not add up to 100% due to rounding. ED, Emergency Department; AICD, Automated Implantable Cardioverter-Defibrillator.

Primary and secondary ICD-10 codes

The most common primary ICD-10-AM code for admitted patients in the NALHN OHCA registry (n = 316) was acute myocardial infarction (I21) followed by cardiac arrest (I46) (Fig. 2, Table S4). When the ICD-10 coding set was used to identify admitted OHCA cases using only primary codes, sensitivity and PPV were 32.9% and 61.5%, respectively. The primary diagnosis code of cardiac arrest (I46) had a 14.6% sensitivity and 82.1% PPV for admitted OHCA, compared to 66.8% sensitivity and 40.1% PPV for OHCA when both primary and secondary diagnoses of I46 were searched. A breakdown of cases identified by each ICD-10-AM code in the coding set can be found in Table S5.

Discussion

Although EMS-based registries remain a primary source of OHCA identification and data, hospital-based sources provide additional identification of non-EMS attended OHCAs, as well as data on in-hospital management and outcomes. In this paper we describe the methods used to identify cases in the NALHN OHCA registry using a simple and consistent definition of OHCA. To address the overarching aims of the registry, this definition allowed the inclusion of OHCAs that arrive by private vehicle and thus was not based on the emergency response (EMS vs MET) but rather whether the patient was receiving in-patient care at the time of cardiac arrest. Our analysis of existing hospital-based sources found that the ED coding set identified the most OHCAs overall. Sensitivity analyses excluded non-EMS-attended and -resuscitated OHCAs resulted in similar accuracy for each data source. We confirmed that ICD-10 codes do not provide efficient identification of OHCA cases. Hospital-based data sources, ideally in combination with EMS-based sources, provide a valid method of identifying OHCA cases treated at hospital.

Validation of hospital-based methods of OHCA identification

Due to the heterogenous nature of OHCA presentations and limitations of coding, there is no single data source that correctly identifies all OHCA cases. Combined hospital-based sources identified 97% of OHCAs treated at a NALHN hospital with the remainder identified by the EMS-based registry. Overall, the ED coding dataset yielded the most OHCAs with a sensitivity of 78% for total OHCA and 85% for OHCAs admitted to ED. Existing clinical registries, especially the ICU registry, were highly accurate within their respective patient subgroups, and when combined with the ED coding set, identified 93% of total OHCAs. To the best of our knowledge the NALHN OHCA registry is the first of its kind. Other hospital-based registries either exclude non-EMS attended arrests and patients without ROSC, or do not adequately describe methods of identification, accuracy, and reliability to enable comparison.7, 20, 21 Multiple-source sudden cardiac death registries that additionally utilise autopsy registers and death certificate screening are nearest to comprehensive case capture but are generally limited by their exclusion of non-cardiac cases. Existing data sources are a valid method of hospital-based OHCA identification that minimises the potential burden associated with clinician-led identification.

Limitations of EMS-based registries

Current prospective OHCA data sources are predominately EMS-based.3, 4, 5, 6, 7, 8, 9 Except for PAROS, most OHCA registries do not capture non-EMS-attended OHCAs such as hospital arrivals by private vehicle, arrests on hospital grounds individuals not occupying an ED or inpatient bed, or clearly deceased cases of sudden cardiac death transported directly to coronial services. The latter may be further identified from autopsy registers or by death certificates as in multiple-source SCD registries.22, 23 Although cases may be identified using multiple strategies by EMS personnel and many are linked to hospital records or death registries, missing data remains an issue. Some registries do not routinely audit for missing data and up to 25% of eligible cases have been reported as missing.25, 26 Using hospital-based sources we identified 59 of 254 (23%) eligible cases that were missing from the EMS-based registry, though this number is expected to decrease as the definition of cardiac arrest is standardised between the registries. We also identified 21 (5%) non-EMS attended OHCAs, the majority of which occurred during transport in private vehicles and on arrival to hospital carparks. By combining an EMS-based registry with existing hospital-based sources of OHCA identification we begin to improve identification of non-EMS attended arrests for inclusion in registries in a manner that is not overly resource-intensive.

Limitations of ICD-10 coding

This is the first study to demonstrate that neither single nor primary ICD-10-AM codes are valid methods of identifying hospital-admitted OHCAs. We found that multiple primary and secondary ICD-10-AM codes identified 79% of admitted cases and 64% of all NALHN presentations, albeit with a PPV of only 33% and only 18/251 (7%) unique cases not found in other sources. Only two other North American studies have used ICD-9 coding datasets applied to ED encounters to identify OHCAs and sensitivity varied from 40% to 87%.20, 27 Of note, ICD-10-AM codes are only assigned to in-patient and not ED encounters, so the ED coding dataset used in this study (which incorporates a short version of the ICD-10-AM) may be more useful for future comparisons. Unlike these studies, we did not include I47.2 Ventricular tachycardia because the false positive rate (identification of mainly in-hospital arrests) was expected to be exceptionally high. Although the most common primary diagnosis code for admitted OHCAs was acute myocardial infarction (I21), likely because the I46 code cannot be applied to resuscitated OHCAs when the underlying cause is documented, this code would also not be sensitive enough to identify OHCAs effectively. Future iterations of the ICD-10 coding system should differentiate between IHCA and OHCA to allow effective monitoring of disease trends and allocation of hospital resources.

Study limitations

The NALHN OHCA registry was designed to overcome limitations of incomplete case capture inherent to other types of registries but is also, by design, subject to limitations. The registry was designed for thorough investigation of cases treated at NALHN hospitals but does not provide information on cases attended by EMS in the community that were declared dead on scene or transported to other acute care facilities. Although traumatic OHCAs were not excluded from the registry, bias may be introduced because most are retrieved by EMS to a non-NALHN acute care facility. We included cases according to the Utstein definition and identified 23 (6%) unmonitored events may have been syncopal episodes and not true cardiac arrest according to the treating hospital clinician. The data sources used may not be translatable to a national and international setting. Although ICD-10 codes are designed to allow international comparisons they may be subject to local variations in coding practices. The registry is currently limited to a local cohort, but the small size has allowed us to more effectively test a method that can be adapted for larger population-based registries. Data-linkage with SAAS-CAR was not available for 2011 due to data capture issues, and in future will be expanded to include arrests outside the NALHN catchment. Both NALHN and SAAS-CAR primarily use paper-based records whereas other institutions may be able to conduct electronic searches using appropriate keywords to increase case capture. Confirmation of OHCA and collection of many core data variables requires manual hospital record review, which may be subject to confounding and bias when compared to prospective data collection by a clinician directly involved in the patient care. However, such a method is not feasible in our or other settings and remains subject to missed cases due to the highly varied and time critical nature of OHCA presentations.

Conclusion

We have overcome the challenges of establishing a hospital-based OHCA registry by using existing hospital-based sources as well as linkage with an EMS-based registry. Our analysis confirms that ICD-10 codes do not efficiently identify OHCAs and should not be used in the calculation of cardiac arrest incidence. We found that the ED coding set had the highest sensitivity for total OHCA cases in the NALHN OHCA registry. The methods presented here may be adapted to augment EMS-based data or used where EMS registries are not established or data-linkage with EMS is prohibitive.

Funding

None to declare.

Conflicts of interest

None to declare.

Ethics information

The Central Adelaide Local Health Network Human Research Ethics Committee approved the registry as an ongoing quality improvement activity and separate approval was obtained for data linkage with the South Australian Ambulance Service Cardiac Arrest Registry (SAAS-CAR).

CRediT authorship contribution statement

Melanie R. Wittwer: Conceptualization, Methodology, Investigation, Formal analysis, Writing - original draft. Mohammed Ishaq Ruknuddeen: Methodology, Resources, Writing - review & editing. Mel Thorrowgood: Resources, Writing - review & editing. Chris Zeitz: Supervision, Writing - review & editing. John F. Beltrame: Supervision, Writing - review & editing. Margaret A. Arstall: Conceptualization, Supervision, Writing - review & editing.