Lessons Learned From Rapid Deployment of 100% Mortality Review for Patients With COVID-19 Across a Health System.

Mortality review is one approach to systematically examine delivery of care and identify areas for improvement. Health system leaders sought to ensure hospitals were adapting to the rapidly changing medical guidance for COVID-19 and delivering high-quality care. Thus, all patients with a COVID-19 diagnosis within the 6-hospital system who died between March and July 2020 were reviewed within 72 hours. Concerns for preventability advanced review to level 2 (content experts) or 3 (hospital leadership). Reviews included available autopsy and cardiac arrest data. Overall health system mortality for COVID-19 patient admissions was 12.5% and mortality for mechanically ventilated patients was 34.4%. Significant differences in mortality rates were observed among hospitals due to demographic variations in patient populations at hospitals. Mortality reviews resulted in the dissemination of evolving knowledge among sites using an electronic medical record order set, implementation of proning teams, and development of checklists for converting COVID-19 floors and units.

Introduction

Medical errors were prominently highlighted in the 1999 To Err is Human report[1] and remain common today. Yet, medical systems in the United States are complex, which often contributes to medical errors and preventable deaths. Estimates of the magnitude of preventable deaths range from 2.5% to 85.3%.[2]

As a result, the National Quality Forum tasked health care organizations with systematically identifying and mitigating patient safety risks and hazards to eliminate patient harm.[3] Mortality review is one approach to systematically examine the delivery of care and identify areas for improvement.[4] Some academic medical centers have adopted 100% mortality review and found it to be a valuable tool to improve patient care.[5-9] Mortality is a vital outcome measure of health care quality, and several different measures are publicly reported and linked to pay-for-performance.[10]

Before COVID-19, in 2018, the Johns Hopkins Health System (JHHS) had 2354 total inpatient deaths that ranged by entity from 66 (2.8%) to 882 (37.5%) (unpublished data, January 23, 2020). Leadership of the health system recognized that evaluation of preventable inpatient deaths was inconsistent. Health system hospitals also worked independently on quality improvement efforts, although there was interest in ensuring that what each hospital learned was shared across the health system. Thus, leadership made review of all inpatient deaths across the health system a strategic objective. The design and implementation of this health system-wide process to review all mortalities were temporarily put on hold when COVID-19 admissions across JHHS rapidly increased and quality improvement efforts were redirected to COVID-19 concerns.[11] Several months into the pandemic in 2020, best practices when caring for patients with COVID-19 started to rapidly evolve as health care learned more about the virus.[12] Thus, leadership recognized the benefit of reviewing any patient who died who was COVID-19 positive within a JHHS hospital. The goal was to ensure JHHS was delivering high-quality care and adapting to the rapidly changing medical guidance for COVID-19. This article describes implementation of 100% mortality review for patients with COVID-19 and lessons learned, some of which led to system changes to improve patient care.

Methods

In March 2020, 100% mortality review was implemented across all 6 hospitals in the health system, comprising 2 academic medical centers and 4 community hospitals in Maryland, the District of Columbia, and Florida. All adult and pediatric hospitalized patients with COVID-19 that died in a JHHS hospital between March 1 and July 1, 2020 underwent mortality review within 24–72 hours.

Mortality Review Process

The original mortality review process was modeled after similar processes at other academic medical centers, which involves emailing a screening questionnaire regarding preventability to frontline providers (attending, nurse, and when applicable, house staff). The results of this determined if further review of the death was appropriate.[9] For COVID-19, the mortality review process was modified from the originally designed plan. Rather than email frontline providers to screen for deaths that should undergo further review, every death was reviewed. This modification was done to decrease burden on frontline staff and to respond to the rapid increase in patient volume and rapidly changing standards of care for COVID-19.

An internal medicine/pediatric physician and hospitalist (C.A.H.) and nurse in the quality improvement department (M.D.) independently reviewed all patients diagnosed with COVID-19 who died. Supplemental Digital Content 1, Table S1, available at http://links.lww.com/AJMQ/A71, provides the checklist of information extracted for review. Patients were identified through a report in the electronic medical record (EMR) and confirmed via a separate electronic dashboard maintained by the Hospital Epidemiology and Infection Control department.

The Figure illustrates the review process. If either the physician or nurse reviewer raised concerns about a case in the level 1 review, it was referred to a core group of content experts, composed of intensive care physicians and hospitalists, for a level 2 review by a larger group. Cases in which additional concerns were identified or more information was needed from the frontline care team were referred to the chief medical officer (CMO) at the hospital where the death occurred. The CMO then appointed local leaders to review the case with the care team, which was usually the attending physician and nursing staff providing care at the time of death, to obtain additional information (level 3 review including hospital leadership). Each mortality review was assigned a preventability score using the following 5-point scale. A score of 3 triggered a level 2 review and score 4 or 5 triggered a level 3 review.

The review process after a patient with COVID-19 died in a Johns Hopkins Health System (JHHS) hospital. Concerns about preventability moved the case to the next level for review by a larger group of content experts (level 2) or by leadership and care team members where the patient died (level 3). A rapid cycle root cause analysis (RCA) could be requested at any time if there was concern about a significant system or process of care issue.

Not preventable death, due to terminal illness or condition upon arrival to this hospital.

Not preventable death, and occurred despite the health team taking preventative measures.

Not preventable death, but medical error or system issue was present.

Possible preventable death resulting from medical error or system issue.

Likely preventable death resulting from medical error or system issue.

If at any time in the review process the case was felt to have a significant system or process of care issue, it was referred for a rapid cycle root cause analysis (RCA) by the core content expert group and risk management. The rapid cycle RCA process had a turnaround time of <2 weeks, which is dramatically shorter than a typical RCA. The shorter turnaround time was possible because other routine quality/safety activities (eg, review of events in the safety reporting system) were stopped to allow the entire health system to focus on the COVID-19 response.[11]

Variables and Data Collection

Data for each review were collected in a standard fashion in a password protected Excel document. Variables included demographics: age, gender, and race (Black/African-American, White, Asian, other, or unknown). Also, source of admission (home, different JHHS hospital, different hospital non-JHHS, skilled nursing facility [SNF] or assisted living, different hospital-SNF, and other), code status (full, DNR/DNI, or DNR), palliative care utilization, and specific COVID-related treatments (hydroxychloroquine, remdesivir, venous thromboembolism [VTE] prophylaxis, intubation, etc). The initial physician and nurse reviewers compared autopsy results (if conducted) to the mortality review results and collected cause of death recorded by Pathology. They also obtained the quality scores for cardiopulmonary resuscitation (CPR) interventions for any deceased COVID-19 patient who underwent intubation or CPR. The quality of an intervention is determined by the hospital’s CPR committee and reported in the cardiac arrest report.

Analysis

Descriptive statistics are compared overall for JHHS and by hospital, and hospitals defined as academic medical center (AMC) or community hospital (CH). Descriptive statistics and proportions were used to describe the data. Authors used the chi-square test or Kruskal–Wallis test, as appropriate, to compare demographics between hospitals and chi-square for comparison of total COVID-19 deaths by hospital. Quantitative data trends and qualitative context for each death informed emerging themes and interventions to standardize and communicate changes in practice. No patients died from COVID-19 during the study period at the children’s hospital in Florida, and therefore, this hospital was excluded from the analysis.

Results

Demographic and clinical characteristics of patients varied significantly by hospital (P < 0.001) (Table 1). Mortality was highest for CH 1, reaching 23.8% (124/521 patients) (Table 2). A total of 276 patients were reviewed, of which 27 (9.8%) received a level 2 review by a group of experts, and 2 (0.7%) underwent a level 3 local entity review. Of 276 cases, 269 (97.5%) were scored 1 or 2 (not preventable due to terminal illness on admission, or despite team taking preventative measures), and 7 (2.5%) were scored 3 (not preventable but error/system issue present). No cases were scored 4 or 5. One rapid RCA was completed. At AMC-1, 21 patients admitted with COVID-19 received CPR, and of those, 16 (76.2%) survived the cardiac arrest event. Of those 16 patients, 8 (50%) survived to discharge from the index hospitalization.

Table 1.

Demographics and Clinical Characteristics of COVID-19 Patients Who Died by Health System Hospital Between March 1 and July 1, 2020.

	AMC 1 (n = 66)	AMC 2 (n = 20)	CH 1 (n = 124)	CH 2 (n = 39)	CH 3 (n = 27)	P  a
Age	67	62	80	71	86
Gender, No. (%)
Female	26 (39.4)	6 (30)	52 (41.9)	16 (41.0)	15 (55.6)
Male	40 (60.6)	14 (70)	72 (58.1)	23 (59.0)	12 (44.4)
Race, No. (%)						<0.001
Black/African-American	26 (39.4)	6 (30.0)	45 (36.3)	14 (35.9)	11 (40.7)
White	17 (25.8)	12 (60.0)	62 (50.0)	12 (30.8)	14 (51.9)
Asian	4 (6.1)	1 (5.0)	8 (6.5)	5 (12.8)	1 (3.7)
Other	18 (27.3)	1 (5.0)	8 (6.5)	7 (17.9)	1 (3.7)
Unknown	1 (1.5)	0	1 (0.8)	1 (2.6)	0
Admission source, No. (%)						<0.001
Home	23 (34.8)	8 (40.0)	29 (23.4)	17 (43.6)	19 (70.4)
Different JHM hospital, admitted from home	20 (30.3)	0	0	0	0
Different non-JHM hospital	7 (10.6)	1 (5.0)	1 (0.8)	0	0
SNF/assisted living directly	4 (6.1)	9 (45.0)	93 (75)	21 (53.8)	8 (29.6)
Different hospital, from SNF	12 (18.2)	0	0	0	0
Other	0	2 (10.0)	1 (0.8)	1 (2.6)	0
Mechanically ventilated, No. (%)	60 (90.9)	15 (75.0)	33 (26.6)	23 (59.0)	12 (44.4)
Day 2 code status, No. (%)						<0.001
DNR order	2 (3.0)	0	9 (7.3)	0	2 (7.4)
DNR/DNI order	3 (4.5)	5 (25.0)	32 (25.8)	9 (23.1)	7 (25.9)
Palliative care	1 (1.5)	2 (10.0)	37 (29.8)	7 (17.9)	8 (29.6)
Full code	60 (90.9)	13 (65.0)	46 (37.1)	23 (59.0)	10 (37.0)

Chi-square test or Kruskal–Wallis test used for comparison.

Abbreviations: AMC, academic medical center; CH, community hospital; DNI, do not intubate; DNR, do not resuscitate; JHM, Johns Hopkins Medicine; SNF, skilled nursing facility.

Table 2.

Health System Hospital Characteristics and Mortality of COVID-19 Cases Between March 1 and July 1, 2020.

Characteristics	AMC 1	AMC 2	CH 1	CH 2	CH 3	P  a
Total licensed beds	1162	420	230	220	318
Total COVID-19 admissions	682	341	521	474	195
Total COVID-19 deaths	66	20	124	39	27
% mortality	9.7	5.9	23.8	8.2	13.8	p<0.00001

Chi-square test.

Abbreviations: AMC, academic medical center; CH, community hospital.

Autopsy results were congruent with the cause of death identified by the primary treatment team (Table 3).

Table 3.

Johns Hopkins Health System Pathology COVID-19 Autopsy Findings

Cause of Death	Number
COVID-19 acute lung injury	6
COVID-19 pneumonia	1
Cardiac disease/arrhythmia	3
Sickle cell disease crisis	1
Intracranial hemorrhage	1
Abdominal hemorrhage	1
Sepsis	2
Acute bacterial pneumonia	2
Pulmonary embolism vs pancreatitis	1

Autopsy was not conducted on all patients.

Lessons Learned Prompting System Changes

Several lessons learned from the mortality reviews offered opportunities to optimize care issues (Table 4). As understanding of the disease evolved, frontline providers moved away from the initial approach of early intubation of most critically ill patients to using noninvasive ventilatory strategies and high flow nasal canula as first line therapies for some patients. A few cases revealed difficulties proning intubated patients. One lesson was that it takes skill to safely turn a critically ill patient to a prone position and only intensive care unit (ICU) staff who cared for patients with acute respiratory distress syndrome (ARDS) in the pre-COVID era had this clinical expertise. Because few ICU staff have this experience, trained proning teams were assembled at any hospital where patients with COVID-19 were cared for in ICUs unfamiliar with proning. Another lesson was the need for a central platform to easily and consistently communicate quickly evolving changes in practice to frontline providers across the health system. In response, AgileMD, an application in the JHHS Epic EMR system, was used to build clinical decision support guidelines for COVID-19. A third lesson related to redeployment of providers (eg, nurses, physicians, and respiratory therapists) to new assignments and transitioning of floors to manage different patient populations—COVID-19 or non-COVID-19. Asking clinicians to step into a new situation and take on new duties required checklists to ensure the things that needed to happen for that transition could occur (Supplemental Digital Content 2, Table S2, available at https://links.lww.com/AJMQ/A72). The checklist was extensive and included ensuring code teams knew about any new care location or patient populations (eg, adults being cared for in pediatric ICUs), ensuring appropriate supplies, pharmacy and IT support were established. JHHS leadership also learned to keep staff and providers on the units where they most often worked whenever possible. Finally, the JHHS management team learned to open enough floors/ICUs to enable a limit to be set on the number of admissions per day, especially for ICU patients.

Table 4.

Lessons Learned From Mortality Reviews of COVID-19 Cases and System Changes.

Lessons Learned	System Changes, Dates implemented
Proning critically ill patients requires experienced staff	Proning teams created at each hospital who cared for ICU patients outside of the medical ICU. April 2020
Central platform is vital to easily and consistently communicate evolving care practices to frontline clinicians	Clinical decision support order set for COVID-19 added to EPIC electronic health record for entire health system (easily modifiable as recommendations changed).AMC-1: Adult Emergency Department, March 9, 2020AMC-1: Pediatric Emergency Department, March 13, 2020Inpatient units, April 10, 2020JHHS: July 2020
Redeploying providers to new assignments and transitioning care (non-COVID to COVID) requires instruction	Checklists for converting units to and from biomodeEnd of March 2020–June 2020JHHS May 2020Ensure providers remain on the unit/floors they are most familiar with whenever possible.Late March 2020 (remains the philosophy)
Lessons learned
High mortality can appear to be a clinical care issue, but when drilling down stemmed from the patient population.
When responding to a new disease in a pandemic and learning pathology at the bedside, it is important to approach mortality reviews with humility.
Cardiopulmonary resuscitation outcomes were better than reported in the literature for patients with COVID-19 experiencing cardiac arrest.
Higher central line-associated bloodstream infection and other hospital-acquired infection rates are most likely due to being critically ill and proned for a prolonged period.
When creating new units (both COVID and non-COVID), limiting the number of admissions per unit/floor per day was important.

Abbreviations: AMC-1, academic medical center 1; JHHS, Johns Hopkins Health System.

Discussion

This mortality review found that most patients with COVID-19 that died between March 1 and July 1, 2020 in the health system were older and residents admitted from an SNF, factors known to correlate with higher mortality. These findings align with national and global data reports.[13,14] Approximately one-third of patients mechanically ventilated for COVID-19 died. This finding is similar to the 30%–40% mortality rate in patients with ARDS who die from other causes, such as influenza or severe pneumonia.[15-20] While early reports of survival after CPR were dismal for patients experiencing cardiac arrest,[21,22] the current review data suggested much better survival rates. Though discussion of goals of care with patients and families fighting COVID-19 are important,[23] clinicians must stay current with what is known before having conversations about DNR[12] since CPR may not be hopeless for all cases.

The comparison of hospital characteristics highlighted very different populations of patients with COVID-19 cared for at each hospital. Two CHs received the majority of their COVID-19 cases from SNFs, and the 2 AMCs had younger patients who were more likely to be admitted from other hospitals and remain full code until late into their hospitalization. Thus, the review team discovered that the higher mortality observed at the 2 CHs was a patient population issue and not what appeared on the surface to be due to the quality of care. This is valuable information that would not be captured in a safety or incident reporting system, demonstrating the benefit of 100% mortality reviews.[7] Moreover, public reporting of in-hospital mortality by the CMS and Hospital Quality Alliance should consider adding risk factors, such as admission source, age, and pre-existing DNR status as variables when analyzing and comparing hospital performance.[10,24]

An overarching lesson learned was to approach mortality review with humility when responding to a new disease in a pandemic and learning pathology at the bedside. For example, steroids were initially thought to be harmful, but further evidence shows that systemic corticosteroids help in patients with hypoxia, but appear harmful for patients early in the disease course or with a mild course of COVID-19.[12,25] Along the same theme, the review team observed that patients with COVID-19 had higher rates of central line-associated bloodstream infection and other hospital-acquired infection rates. Yet, this is now felt to be the result of patients being critically ill and laying prone for a prolonged period. This is the case across many COVID-19 ICUs not just within JHHS. This emphasizes the importance of not assuming poor quality care as the default etiology of adverse outcomes for all patient populations, particularly for hospital-acquired complications. VTE prophylaxis in COVID-19 and ventilatory strategies for critically ill patients were also moving targets. There was wide variation in practice early on in the pandemic.[26] High incidence of VTE in patients with COVID-19 is a good example. Researchers at Johns Hopkins hypothesized that incidence was higher because limited patient contact and availability of personal protective equipment was causing missed doses of pharmacologic VTE prophylaxis.[27] What they found was that nearly 75% of patients with COVID-19 and prescribed prophylaxis were receiving all doses, a higher proportion than patients testing COVID-19 negative (61.5%) and not tested (63.9%).

This 100% mortality review allowed for real time review of patterns of clinical care and a good understanding of whether recommended adjustments to clinical care were being adopted across JHHS (eg, eliminating use of hydroxychloroquine, augmenting VTE prophylaxis per local guidelines specific to patients with COVID-19). In addition, the authors learned that bringing experts from the academic medical centers to tour other hospitals, meeting with local physicians, nurses, and other staff to discuss COVID-19 care and share experiences was inspired by the mortality review process. This led to tighter collaboration and consistent care as reflected in the AgileMD guidelines. While 100% mortality reviews take resources, the cost burden is likely balanced by the system changes that made care safer and likely mitigated further harm, and may be considered a value-based health care intervention.[28]

As health systems work to deliver value-based health care, they could use 100% mortality review to reveal inefficiencies in care and redesign processes and systems of care, as the authors did when implementing a modifiable COVID-19 clinical decision support order set and experienced proning teams. Used in this way, mortality review supports the Hospital Value-Based Purchasing Program in which CMS leverages payment to improve patient experience and outcomes in the hospital and to lower costs.[29] When the COVID-19 pandemic started, JHHS was preparing for a planned 100% mortality review program across the entire health care system. Although the roll-out was put on hold for COVID-related priorities,[11] this comprehensive mortality review program has since been implemented across JHHS and operational for nearly a year, as of March 2022. All mortality, including COVID-19 deaths, are now reviewed and evaluation of the 100% mortality review program planned for the future. This program is strongly supported by high-level leadership who provides the resources and placed this initiative on the list of JHHS strategic priorities.

Acknowledgments

Conflicts of Interest

Dr Haut reports research funding from the Patient-Centered Outcomes Research Institute (PCORI), the Agency for Healthcare Research and Quality (AHRQ), the NIH/NHLBI, the DOD/Army Medical Research Acquisition Activity, and the Henry M. Jackson Foundation for the Advancement of Military Medicine (HJF). Dr Haut also receives royalties from Lippincott, Williams, Wilkins for a book, “Avoiding Common ICU Errors.” Dr Haut was a paid speaker for the Vizient Hospital Improvement Innovation Network (HIIN) VTE Prevention Acceleration Network. All the other authors have no conflicts of interest to disclose.

Author Contributions

Dr Herzke is the guarantor of the work and contributed to conception and design of the research, data collection, data analysis and interpretation, and drafting and revision of the manuscript. Holzmueller contributed to the interpretation of data and drafting and revision of the manuscript. Dutton contributed to data collection and analysis, and revision of the manuscript. Drs Kachalia, Hill, and Haut contributed to conception and design of the research and data analysis and interpretation, and revision of the manuscript. All of the authors approved the final version submitted to the journal, and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of the work are appropriately investigated and resolved.