The association of acute COVID-19 infection with Patient Safety Indicator-12 events in a multisite healthcare system.

BACKGROUND: Patient Safety Indicator (PSI)-12, a hospital quality measure designed by Agency for Healthcare Research and Quality (AHRQ) to capture potentially preventable adverse events, captures perioperative venous thromboembolism (VTE). It is unclear how COVID-19 has affected PSI-12 performance.
OBJECTIVE: We sought to compare the cumulative incidence of PSI-12 in patients with and without acute COVID-19 infection. DESIGN, SETTING, AND PARTICIPANTS: This was a retrospective cohort study including PSI-12-eligible events at three Mayo Clinic medical centers (4/1/2020-10/5/2021). EXPOSURE, MAIN OUTCOMES, AND MEASURES: We compared the unadjusted rate and adjusted risk ratio (aRR) for PSI-12 events among patients with and without COVID-19 infection using Fisher's exact χ2  test and the AHRQ risk-adjustment software, respectively. We summarized the clinical outcomes of COVID-19 patients with a PSI-12 event.
RESULTS: Our cohort included 50,400 consecutive hospitalizations. Rates of PSI-12 events were significantly higher among patients with acute COVID-19 infection (8/257 [3.11%; 95% confidence interval {CI}, 1.35%-6.04%]) compared to patients without COVID-19 (210/50,143 [0.42%; 95% CI, 0.36%-0.48%]) with a PSI-12 event during the encounter (p < .001). The risk-adjusted rate of PSI-12 was significantly higher in patients with acute COVID-19 infection (1.50% vs. 0.38%; aRR, 3.90; 95% CI, 2.12-7.17; p < .001). All COVID-19 patients with PSI-12 events had severe disease and 4 died. The most common procedure was tracheostomy (75%); the mean (SD) days from surgical procedure to VTE were 0.12 (7.32) days.
CONCLUSION: Patients with acute COVID-19 infection are at higher risk for PSI-12. The present definition of PSI-12 does not account for COVID-19. This may impact hospitals' quality performance if COVID-19 infection is not accounted for by exclusion or risk adjustment.

INTRODUCTION

The incidence of perioperative venous thromboembolism (VTE), which encompasses deep venous thrombosis (DVT) and pulmonary embolism (PE), ranges from 0.6%–3%. , Evidence‐based practice guidelines recommend prophylactic regimens to reduce the risk of perioperative VTE. The Agency for Healthcare Research and Quality (AHRQ) developed Patient Safety Indicator (PSI) measures to screen for potentially preventable adverse events, using hospital administrative claims data and International Classification Diseases (ICD)‐10 diagnosis and procedure codes. PSI‐12 captures perioperative VTE as the ratio of adult patients with ICD‐10 codes for PE and proximal DVT (numerator) over all inpatients with qualifying procedural codes (denominator), expressed as a rate per 1000 discharges. Denominator exclusions to PSI‐12 are few and include acute VTE that is present on admission, index procedures of pulmonary arterial thrombectomy or inferior vena cava interruption, or acute brain or spinal injuries, primarily hemorrhagic in nature. Based on the use of coding methodology, PSI‐12 may include cases where the ICD‐10 code for VTE precedes the procedure itself.

Since the start of the coronavirus disease 2019 (COVID‐19) pandemic, clinicians have recognized that VTE is a significant complication of COVID‐19 infection. , Factors that increase VTE risk in COVID‐19 infection include disease severity, male sex, older age, history of VTE, and obesity. A meta‐analysis of 66 observational studies reported higher VTE prevalence in hospitalized patients admitted to the intensive care unit (ICU) and in those screened with ultrasound. In surgical patients, Doglietto et al. and COVIDSurg Collaborative and GlobalSurg Collaborative demonstrated a higher risk of mortality and higher odds of thrombotic complications for patients with COVID‐19 infection compared to non‐COVID‐19 patients. The disease‐specific thrombosis risk from COVID‐19 has the potential to complicate VTE measures such as PSI‐12. It is unclear how the increased VTE risk from widespread COVID‐19 infection has affected the rate of PSI‐12, and its impact on quality measures and ratings for United States’ healthcare institutions. Our study aimed to understand the association of acute COVID‐19 infection and the PSI‐12 rate in a multisite health care system by comparing rates of PSI‐12 events between those with acute COVID‐19 infection versus those without COVID‐19 infection during their hospitalization. We sought to also describe the clinical characteristics and outcomes of COVID‐19 individuals with a PSI‐12 event.

METHODS

Study data and population

This was a retrospective cohort study including all consecutive inpatient encounters at the Mayo Clinic hospitals in Minnesota, Arizona, and Florida, from April 1, 2020, through October 5, 2021, which met denominator eligibility criteria for PSI‐12. We obtained all data from patients’ electronic health records. The Mayo Clinic Institutional Review Board deemed the study exempt (21‐006228). We used the hospital protocol for the management of the pro‐coagulable state in COVID‐19, as agreed by the multidisciplinary COVID‐19 treatment review panel. Our management protocols went under continuous iterations as the pandemic evolved.

Outcome

The primary outcome was PSI‐12, which included perioperative VTE (PE or proximal DVT) as defined by AHRQ. We used publicly available AHRQ Quality Indicators (QI) Windows Application (WinQI) software v2020 to identify PSI‐12 events in our cohort. The software identifies the events based on surgical procedure codes upon discharge. The exposure was acute COVID‐19 infection status. During the study period, all patients admitted to Mayo Clinic hospitals underwent a polymerase chain reaction (PCR) test for the presence of the severe acute respiratory syndrome coronavirus type 2 (SARS‐CoV‐2) at the time of admission. For elective surgical patients, outpatient preprocedural screening for COVID‐19 with PCR was performed. All patients who screened positive had elective surgery postponed for 20 days or until isolation precautions were removed in the event of the development of COVID‐19 symptoms, per facility policy. The presence of an ICD‐10 diagnosis code of U07.1 or a positive SARS‐CoV‐2 PCR or antigen test 14 days prior to admission, at the time of admission, or during the hospitalization (positive test after an initial negative test on admission) defined acute COVID‐19 infection. Those without acute COVID‐19 infection formed the comparator group.

Statistical analysis

We tabulated and compared patient demographics and characteristics by acute COVID‐19 status (infected versus uninfected), using χ 2 tests for categorical variables and Wilcoxon rank‐sum tests for continuous variables. We calculated unadjusted PSI‐12 rates (with exact 95% confidence intervals), defined as the cumulative proportion of surgical discharges which experienced a PSI‐12 event, both overall and by acute COVID‐19 status. We compared these unadjusted PSI‐12 rates by COVID‐19 infection status using an exact χ 2 test. We also calculated and compared the adjusted PSI‐12 rates (with an adjusted risk ratio [aRR] and 95% Wald confidence intervals) by acute COVID‐19 infection status using log‐linear regression, with adjustment for the expected risk of PSI‐12 as calculated by the AHRQ WinQI software algorithm.

Descriptive case‐series analysis

For all patients with acute COVID‐19 infection who experienced a PSI‐12, we abstracted additional clinical details to qualitatively describe their course of care. Specifically, we abstracted each patient's indication for hospitalization, history of VTE, hypercoagulability, International Medical Prevention Registry on Venous Thromboembolism and D‐Dimer (IMPROVEDD) VTE risk score, index surgical procedure, time (in days) from positive PCR prior to admission, time (in days) from positive PCR to VTE diagnosis, time (in days) from admission to VTE diagnosis, time (in days) from surgery to VTE diagnosis, pharmacological thromboprophylaxis on admission, need for ICU, length of ICU stay (in days), length of hospital stay (in days), and discharge disposition (alive, deceased). We summarized these data using frequencies for categorical variables and mean (SD) for continuous variables. The IMPROVEDD VTE risk score is a tool to risk stratify hospitalized, medically ill patients and predict their 42 and 77‐day VTE risk.

RESULTS

Our cohort included 50,400 consecutive hospitalizations which met eligibility criteria for the PSI‐12 denominator during the study period, of which 257 (0.51%) had acute COVID‐19 infection and 50,143 (99.49%) did not have acute COVID‐19 infection. The mean (SD) age of the population was 61.69 (15.76) years, and those with acute COVID‐19 infection were significantly younger than those without acute COVID‐19 infection (Table 1). Patients with acute COVID‐19 infection were more likely to be male, non‐White, have emergent or urgent admission types, and have Medicaid as their primary payer as compared to patients without acute COVID‐19 infection.

Table 1

Demographic characteristics by acute COVID‐19 status

Characteristics	Acute COVID‐19 infection (n = 257, 0.51%)	Absence of acute COVID‐19 infection  (n = 50,143, 99.49%)	p Valuea
Age, mean (SD)	60.16 (15.95)	61.70 (15.76)	.066
Female, n (%)	85 (33.07%)	23,400 (46.67%)	<.001
Race/ethnicity, n (%)			<.001
White	201 (78.21%)	45,311 (90.36%)
Black/African American	21 (8.17%)	1837 (3.66%)
Hispanic of any race	6 (2.33%)	962 (1.92%)
Native American	16 (6.23%)	374 (0.75%)
Other	13 (5.06%)	1659 (3.31%)
Admission type, n (%)			<.001
Emergent	131 (50.97%)	8768 (17.49%)
Urgent	88 (34.24%)	5887 (11.74%)
Elective	36 (14.01%)	35,197 (70.19%)
Other/unknown	2 (0.78%)	2 (0.78%)
Primary payer, n (%)			<.001
Medicare	103 (40.08%)	20,218 (40.32%)
Medicaid	28 (10.89%)	2003 (3.99%)
Private	80 (31.13%)	20,961 (41.80%)
Self‐pay/other	46 (17.90%)	6961 (13.88%)
Hospital, n (%)			<.001
Rochester	112 (43.58%)	30,745 (61.31%)
Arizona	86 (33.46%)	9028 (18.00%)
Florida	59 (22.96%)	10,370 (20.68%)
Expected PSI‐12, median, IQR	0.76% (0.38%, 1.20%)	0.34% (0.20%, 0.56%)	<.001
PSI‐12			<.001
Yes	8 (3.11%)	210 (0.42%)
No	249 (96.89%)	49,933 (99.58%)

Abbreviations: IQR, interquartile range; PSI‐12, patient safety indicator‐12; SD, standard deviation.

Wilcoxon rank‐sum for continuous variables and χ 2 test for categorical variables.

With respect to the primary outcome, 218 of 50,400 (0.43%; 95% CI, 0.45%–0.58%) patients experienced a PSI‐12 event. Unadjusted rates of PSI‐12 were significantly higher among patients with acute COVID‐19 infection: 8 of 257 (3.11%; 95% CI, 1.35%–6.04%) patients with acute COVID‐19 infection and 210 of 50,143 (0.42%; 95% CI, 0.36%–0.48%) uninfected patients experienced a PSI‐12 during the encounter (p < .001; Table 2). Patients with acute COVID‐19 infection had higher median expected risks of PSI‐12 than non‐COVID cases per the AHRQ risk‐adjustment WinQI software. The risk‐adjusted rate of PSI‐12 was significantly higher in patients with acute COVID‐19 infection (1.50% vs. 0.38%; aRR, 3.90; 95% CI, 2.12–7.17; p < .001).

Table 2

Outcomes

Characteristics	Acute COVID‐19 infection (n = 257, 0.5%)	Absence of acute COVID‐19 infection  (n = 50,143, 99.5%)	p Valuea
Unadjusted PSI‐12, n	8/257	210/50,143	<.001
Rate, 95% CI	3.11% (1.35%, 6.04%)	0.42% (0.36%, 0.48%)
Adjustedb PSI‐12, n	8/257	210/50,143	<.001
Rate, 95% CI	1.50% (0.81%, 2.76%)	0.38% (0.33%, 0.44%)

Abbreviations: CI, confidence interval; PSI‐12, patient safety indicator‐12.

Exact χ 2 test for unadjusted, Wald χ 2 for adjusted.

For AHRQ Quality Indicators Windows Application (WinQI) software derived the expected risk of PSI‐12.

Case series

The eight perioperative COVID‐19 patients who qualified for PSI‐12 (Table 3) included five PEs and three DVTs. Seven of the eight were admitted for COVID‐19 infection and underwent urgent surgical procedures during their hospital stay. One patient was admitted for a neurosurgical procedure and subsequently developed a positive COVID‐19 PCR test. None of the patients in the cohort were admitted for elective surgery while known to have acute COVID‐19 infection. The patients had a mean (SD) age of 61.75 (8.91) years. Seven were male, all required ICU care, and four died in the hospital. None of the patients had previous history of VTE, but five had hypercoagulable states prior to COVID‐19 infection, including morbid obesity (n = 2) and malignancy (n = 3). All patients received VTE chemoprophylaxis except for one, who was deemed ineligible by clinicians due to elective neurosurgical procedures. The most common procedure was tracheostomy (75%); the mean (SD) days from surgical procedure to VTE were 0.12 (7.32) days and from positive PCR to VTE were 12.87 (15.94) days.

Table 3

Characteristics of acute COVID‐19 infected patients with PSI‐12 events

Patient	Age (years)	Sex	Reason for hospitalization	History of VTE	Hypercoagulability	IMPROVEDD VTE risk scorea	PSI‐12 event	Index surgical procedure	Days from positive PCR prior to admission	Days from positive PCR to VTE diagnosis	Days from admission to VTE diagnosis	Days from admission to surgery	Days from index surgical procedure (day zero) to VTE diagnosis	Pharmacological thromboprophylaxis on admissionb	Need for ICU	Length of ICU stay (days)	Length of hospital stay (days)	Alive at discharge
1	59	F	Pneumonia due to COVID‐19	No	Morbid obesity	2	PE	Tracheostomy	0	24	14	28	− 4	Yes	Yes	30	69	Yes
2	70	M	Pneumonia due to COVID‐19	No	Meningioma	4	PE	Tracheostomy	4	23	29	18	11	Yes	Yes	37	39	No
3	53	M	Pneumonia due to COVID‐19	No	None	4	PE	Tracheostomy	5	23	18	16	2	Yes	Yes	27	34	Yes
4	62	M	Stroke, pneumonia due to COVID‐19	No	None	7	PE	Tracheostomy	0	11	11	6	5	Yes	Yes	16	16	No
5	60	M	Altered mental status, brain mass	No	Glioblastoma multiforme	5	DVT	Right craniotomy endoscopic approach for excisional brain tumor	N/Ad	−22	7	2	5	Noc	Yes	38	43	Yes
6	79	M	Altered mental status, pneumonia due to COVID‐19	No	Squamous cell carcinoma of the scalp	6	PE	Blowhole skin incisions	0	4	4	10	− 6	Yes	Yes	9	18	No
7	59	M	Pneumonia due to COVID‐19	No	Morbid obesity	4	DVT	Tracheostomy	10	15	5	17	− 12	Yes	Yes	123	123	No
8	52	M	Pneumonia due to COVID‐19	No	None	4	DVT	Right thoracoscopy converted to thoracotomy, washout and decortication, tracheostomy	0	25	26	26	0	Yes	Yes	30	47	Yes

Abbreviations: DVT, deep vein thrombosis; F, female; ICU, intensive care unit; IMPROVEDD, International Medical Prevention Registry on Venous Thromboembolism and D‐Dimer; M, male; PSI‐12, patient safety indicator‐12; PE, pulmonary embolism; VTE, venous thromboembolism.

International Medical Prevention Registry on Venous Thromboembolism and D‐Dimer (IMPROVEDD) VTE risk score predict risk of VTE in hospitalized patients. A score greater or equal than 2 identifies patients with increased risk for venous thromboembolic events through 77 days.

Pharmacological prophylaxis on admission was low molecular weight heparin unless renal failure was present, in which case un‐fractioned heparin was used.

Patient tested positive for COVID‐19 infection after surgery.

Patient did not receive pharmacological thromboprophylaxis due to elective neurosurgical procedure.

DISCUSSION

Our study demonstrated a significantly higher rate of PSI‐12 (by a factor of nearly fourfold) in patients with acute COVID‐19 infection when compared to uninfected patients, even when accounting for risk adjustment. These eight patients with acute COVID‐19 infection and PSI‐12 were all, except for one, urgently admitted for acute COVID‐19 pneumonia. All required ICU care, and they all developed VTE despite receiving protocol‐directed pharmacological and/or mechanical prophylaxis.

COVID‐19 dysregulated host response leads to an increased risk of VTE in many patients. , A study by Pasha et al. identified a higher rate of VTE in patients with COVID‐19 infection during initial hospitalization and the first week after positive PCR compared to their pre‐COVID‐19 group. In our case series, every COVID‐19 patient with a PSI‐12 event required ICU admission during their hospital stay, and the mean days from positive PCR for SARS‐CoV‐2 to VTE diagnosis was 12.87 days. Moreover, Rali et al. reported higher mortality in hospitalized COVID‐19 patients diagnosed with VTE than those without VTE. Their COVID‐19 patients diagnosed with VTE had 48% mortality, which is comparable to our COVID‐19‐positive PSI‐12 cohort's 50% mortality.

The eight cases illustrate several critiques of the PSI‐12 measure, including some that existed prior to the COVID‐19 pandemic. PSI‐12 aims to evaluate perioperative VTE, but the broad list of procedural codes included also identifies many patients admitted for urgent medical needs who required a secondary surgical procedure. PSI‐12 does not exclude patients with bleeding disorders or in whom intracranial bleeding developed after admissions, such as in large ischemic strokes or intracranial tumors; such patients may not be candidates for chemoprophylaxis. Patients with serious illnesses may still require urgent procedures such as tracheostomy and are still at risk of being included in the PSI‐12 numerator under the current methodology. Large tertiary centers perform surgery on high‐risk patients who might otherwise be deemed ineligible for procedures, such as those with coagulation disorders. The experience of our center is that critically ill patients, such as those admitted for sepsis following bone marrow transplant, may still be included in the PSI‐12 numerator. PSI‐12 includes a large and heterogeneous population of patients who vary widely both clinically and qualitatively, such as a patient admitted for elective joint surgery versus a patient admitted emergently with COVID‐19 and respiratory failure who undergoes tracheostomy after days of mechanical ventilation. By contrast, other AHRQ PSI measures have more comprehensive denominator exclusions, such as PSI‐6 (iatrogenic pneumothorax) and PSI‐9 (postoperative hemorrhage).

Another methodological concern with PSI‐12 is the inability to exclude cases where VTE was diagnosed prior to the qualifying surgical procedure. In our COVID‐positive PSI‐12 cohort, three of the eight patients were diagnosed with VTE days prior to the qualifying procedure, which were all performed due to severe COVID‐19 pneumonia: two tracheostomies and one blowhole incision for subcutaneous emphysema. The current VTE prophylaxis guidelines in surgical patients rely on evidence from postoperative VTE studies; the preoperative VTEs are excluded due to violation of the principles of cause and effect, with the “effect” coming before the “cause.” Clinically, these VTEs are clearly associated with the underlying medical illness rather than the surgical procedure. While this is not unique to COVID‐19 patients, this issue highlights the difficulty with quality measures that rely solely on coding. A consideration to improve the PSI‐12 methodology might be to exclude cases in which the VTE event precedes the qualifying surgical procedure, which would require reliable capture of dates by hospital billing departments in coding data. Alternatively, a new perioperative VTE measure could be constructed using a combination of clinical and claims‐based data, which may allow greater accuracy and specificity of the measure.

AHRQ currently adjusts for patients’ underlying risks of VTE using multivariate logistic regression models considering four major categories: demographics, the severity of illness, comorbidities, and discharge specific information. The cases in our cohort identified COVID‐19‐related VTE that occurred exclusively in severely ill patients, occurred despite appropriate prophylaxis, and often preceded the index operative procedure. Current risk adjustment using AHRQ WinQI (v2020) software did not fully account for this increase in PSI‐12 related to acute COVID‐19 infection. AHRQ updates their software and criteria annually, based off claims data from 2 years prior. AHRQ v2021 software allows the option to exclude all COVID‐19 discharges, but without risk adjustment. As the pandemic evolves, the quality measurement will also need to change and adapt. Future PSI methodology and software must allow for risk adjustment in addition to (or in place of) exclusion from the denominator. Studies like this one can be used to improve and adjust future methodological decisions regarding the treatment of COVID‐era data in PSI‐12 and other quality outcomes. Our study supports that exclusion of COVID‐19, or updated risk adjustment to account for disease‐specific thrombosis risk, in the PSI‐12 methodology must be incorporated in the assessment of PSI‐12 in future years.

PSIs are currently part of publicly reported quality measures for hospitals. PSI‐90, a composite metric of ten PSIs including PSI‐12, has previously been used by the Centers for Medicare and Medicaid Services (CMS) in both the Hospital‐Acquired Condition (HAC) Reduction Program and the Hospital Value‐Based Purchasing Program. PSI‐90 is undergoing revisions relating to the transition to ICD‐10 and has been removed from the VBP starting in FY2019 but will be reinstated in FY2023. It is important that safety and quality measures are appropriately risk adjusted for the complexity of care required by patients at any given institution. The inadequately adjusted risk may lead to refusal of care for older, sicker, and more medically complex patients whose expenditures are greater than predicted. Blay et al. and Vartak et al. examined the effects of removing VTE from PSI‐90 on hospital performance. They found improvement in PSI‐90 for hospitals that were larger, were major teaching centers, had greater diagnostic resources, or cared for sicker patients. Holding hospitals liable for unavoidable VTEs in COVID‐19 patients may lead to penalties against tertiary and academic facilities caring for large volumes of the sickest patients. This may have the unintended outcome of lower‐quality care being provided to COVID‐19 patients who need urgent surgical procedures by facilities concerned about public reporting and federal pay‐for‐performance consequences surrounding such measures.

Our study has some important limitations. This multicenter study encompasses three destination academic medical centers, potentially limiting generalizability to smaller or nonacademic hospitals. Despite the multicenter design, the sample size was relatively small, and further research is needed to verify these conclusions. In addition, the retrospective design does not allow us to draw cause‐and‐effect conclusions nor allow us to control for changing management and prophylaxis of COVID‐19 associated hypercoagulability over the course of the pandemic. As noted, our risk adjustment may have been confounded by the additional comorbid severity of COVID‐19 patients; however, this is precisely the key takeaway of this analysis: to provide evidence that current AHRQ risk adjustment for PSI‐12 is confounded by patients’ acute COVID‐19 infection and the accompanying comorbid severity.

CONCLUSION

The ongoing COVID‐19 pandemic has strained healthcare systems around the world. Determining how to measure the quality of care during this time is challenging, not only for those acutely ill with COVID‐19 but those who present for other routine care. Improving the accuracy of measures like PSI‐12 will better allow hospitals to review their outcomes and quality of care. Our study found that patients with acute cases of COVID‐19 are at higher risk for meeting the criteria for PSI‐12 than patients without COVID‐19, despite guideline‐appropriate VTE prophylaxis. The current AHRQ risk adjustment does not fully account for this difference. This information should be taken into consideration to update PSI‐12 and other quality measures in the pandemic era, to avoid biasing the publicly reported performance of hospitals caring for critically ill COVID‐19 patients.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.

Supporting information.

Click here for additional data file.