A detailed analysis of patients included in the Summary Hospital-level Mortality Indicator (SHMI) for myocardial infarction (MI)-all is not what it seems?

BACKGROUND: The Summary Hospital-level Mortality Indicator (SHMI) for Myocardial Infarction (MI) is the ratio of the observed to the expected number of deaths due to MI. We aimed to assess (1) the accuracy of MI as a diagnosis in the SHMI for MI and (2) the healthcare received by patients with type 1 MI included in the SHMI for MI.
METHODS: Retrospective review of patients included in SHMI for MI from April 2017 to March 2018. The diagnosis of MI was divided into type 1, type 2 and non-MI. For patients with type 1 MI who underwent intervention, we applied the prognostic Toronto Risk Score (TRS) and classified into group 0: score <13 (mortality risk 0%-4%, lowest risk), group 1: score 13-16 (mortality risk 6%-19.6%), group 2: score 17-19 (mortality risk 27.4%-47.6%) and group 3: score ≥20 (mortality risk 58%-92%). For patients with type 1 MI who underwent conservative management, we reviewed appropriateness of conservative management.
RESULTS: SHMI for MI was 96 (41/42.83) falling to 65.4 with the inclusion of only type 1 MI (28 patients, 28/42.83). About 41.5% (n=17) underwent intervention of whom three were in the lowest risk TRS (group 0) and all received appropriate healthcare. Conservative management was appropriate for the 26.8% (n=11) treated medically, the most common reason was severe cognitive dysfunction.
CONCLUSIONS: We have demonstrated that SHMI for MI can be inaccurate due to the inclusion of type 2 MI or non-MI. Grouping patients into intervention versus conservative management helps in assessment of healthcare. © Author(s) (or their employer(s)) 2020. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Quality Improvement (QI) in the National Health Service (NHS) directly impacts patient care and improves outcomes.1 The use of standardised hospital mortality ratios helps to assess healthcare and identify poor performance with consequent opportunity for QI.2 Summary Hospital-level Mortality Indicator (SHMI)3 is a mortality measure that includes all in-hospital DEATHS, or DEATHS within 30 days (inclusive) of discharge from an individual NHS Trust (figure 1B). Inclusion of postdischarge mortality in the calculation of standardised mortality ratios is known to change outcomes and improve performance of individual hospitals.4 SHMI is calculated as follows [Observed deaths/Expected deaths] × 100=a%. Expected deaths are calculated from a statistical model that adjusts for factors such as age, primary diagnosis, comorbidity and so on.5 SHMI gives an indication if the observed number of deaths within 30 days of discharge from hospital was 'higher than expected' (SHMI banding=1), 'as expected' (SHMI banding=2) or 'lower than expected' (SHMI banding=3) compared with the national baseline.3

Figure 1

SHMI pre and post reclassification. SHMI for all hospitalisation deaths from April 2017 to March 2018. SHMI, Summary Hospital-level Mortality Index.

SHMI for acute myocardial Infarction (MI) at the Trust level is based on all diagnoses coded as MI and available from the Hospital Episode Statistics (HES).6 7 The diagnoses are coded by using WHO’s International Statistical Classification of Diseases (latest version) and Related Health Problems book.8 Inaccuracies in data collection, coding and cause of death certification can lead to the wrong conclusion about quality of care delivered to patients.9 To assess quality of care received for MI, only type 1 MI (secondary to a primary coronary artery event) should be included in the SHMI for MI. However, a type 2 MI (eg, secondary to increased oxygen demand or decreased supply)10 11 may also be included. In such cases death may still be as attributed to MI and SHMI for MI may be inflated by the inclusion of type 2 MI. One-fourth of all MI in hospital patients are type 2 MI and at least half of these patients will have no demonstrable coronary artery disease (CAD).12

In a majority of patients this diagnosis of CAD would have been made based on clinical features and some baseline investigations. However, these investigations may not adequately define CAD. The Fleming method for tissue and vascular differentiation and metabolism has the ability to truly quantify ischaemic vascular disease and differentiate between hibernating and infarcted myocytes on single photon emission computed tomography myocardial perfusion scan (MPS).13 Utility of such a methodology would be of immense help in identifying true CAD in the future.

The SHMI for MI 2017–2018 at our Trust was at the upper limit, hence we conducted this study as a QI exercise.

We aimed to

Assess the accuracy of MI as a diagnosis for patients included in the SHMI for MI

Assess the healthcare received by patients with type 1 MI included in the SHMI for MI.

Methods

A retrospective review of the 41 patients included in the SHMI for patients with MI between April 2017 and March 2018 was performed by two independent cardiologists. In case of disparity regarding type of MI, a consensus was reached by discussion between the two reviewers.

Baseline demographics, cardiovascular risk factors, white cell count, C-reactive protein (CRP) and the Charlson Comorbidity Index data were collected. Patients who underwent percutaneous coronary Intervention (PCI) were compared with those who were managed conservatively.

The study was performed as a departmental QI project and adhered to the SQUIRE guidelines.14 It was approved by the departmental clinical lead and registered as an audit

(audit approval number 1190).

Patient and public involvement

Patients and public were not involved in this study.

Accuracy of the diagnosis of MI

Independent review of electronic case notes (history, presentation), biochemical investigations (including troponin), cardiac investigations (coronary angiography, echocardiogram) and death certificates was undertaken by two Consultant Interventional Cardiologists.

Patients included in the SHMI were divided into non-MI, type 1 MI and type 2 MI.10 11 The SHMI was then recalculated to include only type 1 MI.

Patients with type 1 MI who underwent PCI

The TRS is a prognostic score with excellent precision for assessment of inpatient mortality risk in patients undergoing PCI. The score utilises history, demographics, clinical and procedural variables (table 1).15 We externally validated and recalibrated (inclusion of cardiac arrest) the TRS in our patient cohort. We applied the TRS to patients with type 1 MI included in SHMI who underwent PCI.

Table 1

The TRS

Variable	Risk weight
Age (years)
40–49	1
50–59	2
60–69	3
70–79	4
≥80	5
Diabetes mellitus	2
Renal insufficiency	2
NYHA IV	3
LVEF <20%	3
Multivessel disease	1
Left main disease	2
Recent MI (<1 m)	3
Post-thrombolysis	3
Primary PCI	4
Shock	6
Cardiac arrest with ROSC	6

LVEF, left ventricular ejection fraction; MI, myocardial infarction; NYHA, New York Heart Association; PCI, Percutaneous Coronary Intervention; ROSC, return of spontaneous circulation; TRS, Toronto Risk Score.

Patients were divided into tertiles based on their risk of death as group 0: score <13 (mortality risk 0%–4%), group 1: score 13–16 (mortality risk 6%–19.6%), group 2: score 17–19 (mortality risk 27.4%–47.6%) and group 3: score ≥20 (mortality risk 58%–92%, table 1). The risk of mortality following PCI is low at 0.5%–4%16 17 and based on this risk, we classified group 0 (risk 0%–4%) as the lowest risk group and felt that it was important to assess why patients in this group were non-survivors. We hence evaluated these patients’ procedural and healthcare factors in detail.

Patients with type 1 MI treated conservatively

For patients who underwent conservative management, a detailed review of diagnosis and healthcare received including reason and appropriateness of conservative management was undertaken.

Statistical analysis

Continuous variables are presented as median (IQR) or as mean (±SD) and categorical variables are presented as numbers (%). Continuous variables were tested for normalcy and log transformed if non-normal for comparison by Student’s t-test or by the signed rank sum test.

For external validation of the TRS, the associations of the individual variables of the TRS were assessed with the primary outcome using univariate logistic regression analysis. ORs with 95% CI and significance (p value) were obtained individually for each of the score variables and for the whole score. The predictive ability of the TRS for the primary outcome was tested by (1) area under the curve of the receiver-operating characteristics analysis to assess discrimination and recalibrated by including the variable ‘Cardiac arrest with ROSC’ (weighted like cardiogenic shock=6). Both cardiogenic shock and cardiac arrest (even with return of spontaneous circulation (ROSC)) have poor outcomes. Based on our work on the derivation of a score for Out of Hospital Cardiac Arrest patients to risk stratify the need for invasive angiography called the NULL-PLEASE score, we feel the poor prognostic outcome of cardiogenic shock and cardiac arrest are similar.18 Hence the weighting for these two variables was similar.

(2) Hosmer-Lemeshow (H-L) test to assess calibration (goodness-of- fit of the prediction model).

Statistical analyses were performed using MedCalc. Two-sided p values of <0.05 were considered statistically significant.

In addition survival curves with the TRS were also computed.

Results

There were 41 patients in total and almost half had diabetes mellitus (DM, 41.5%, n=17). Mean age was 75 (range 52–94) years and 36.6% (n=15) were female.

Accuracy of the diagnosis of MI

SHMI for MI was 96% (41/42.83, figure 1A). 68.3% (n=28, figure 2) patients were type 1 MI, 24.3% (n=10) were type 2 MI and 7.3% (n=3) were non-MI.

Figure 2

Accuracy of the diagnosis of MI. MI, myocardial infarction.

The best SHMI result was for type 1 MI as recalculation of the SHMI to include only type 1 MI reduced the SHMI from 96 to 65.4 (28/42.83, figure 1A).

Patients with type 1 MI had a mean age of 74.6 years (range 64.0 to 85.3) and 39.3% (n=11) were female. Of the 28 patients with type 1 MI, 15 patients underwent PCI, 2 patients underwent diagnostic coronary angiography and 11 patients were treated medically.

Patients with type 2 MI were older (mean age 79.3 years, range 78–81.5) and had a primary non-cardiac diagnosis. Forty per cent of these patients were female (n=4). None of the patients with type 2 MI underwent coronary intervention.

TRS demonstrated high discriminatory ability for in-hospital mortality (C-statistic 0.922, 95% CI 0.876 to 0.955, p<0.0001). Youden Index J associated criterion (optimum cut-off point) for in-hospital mortality was >8 with a sensitivity of 86.02% and specificity of 85.09%. H-L test demonstrated good calibration of the model (p=0.787, chi2=3.94). Recalibration of the TRS with addition of cardiac arrest as an extra variable improved the discriminatory ability of the TRS (C-statistic 0.939, 95% CI 0.897 to 0.967, p<0.0001, figure 3). Youden Index J associated criterion was >11 with sensitivity of 78.5% and specificity of 95.6%. H-L test continued to demonstrate good calibration with p=0.871 and χ2=3.84.

Figure 3

Risk stratification by TRS tertiles of patients who underwent PCI. PCI, percutaneous coronary intervention; TRS, Toronto Risk Score.

Assessment of healthcare for the type 1 MI deaths included in the SHMI for MI

Patients with type 1 MI who underwent PCI

Out of a total of 28 patients with type 1 MI, two-third underwent PCI (53.6%, n=15) and one-third were treated conservatively. Two patients had a diagnostic coronary angiogram only. About 86.7% (n=13) presented with an ST elevation MI (STEMI) and underwent Primary PCI (PPCI). Most patients (46.7%, n=7) were in group 1 (mortality risk 6%–19.6%, figure 3).

There were four patients in the low-risk group 0 (score <13). Addition of cardiac arrest as a variable to the TRS reclassified one patient from group 0 to group 3 (figure 3). The three remaining patients were assessed in detail for patient/procedural factors and healthcare received (table 2). All three PCI procedures were successful and uncomplicated. Review of the case notes demonstrated appropriate healthcare input at all stages and involvement/communication with the patient and family.

Table 2

Patient and procedural details for the three patients in group 0 of the tertiles

Patient number	Risk factors	Diagnosis	Procedure	Echocardiogram	TRS	Additional factors
1	HTNDM	AnteriorSTEMI	PPCILMS	Moderate LVSD	12	DTB: <90 minPreterminal AKI
2	DM	NSTEMI	LADPCI	Flail leaflet with severe MRMild LVSD	9	HAPPreterminal AKI
3	HTNSmokerNew DM	AnteriorSTEMI	PPCILAD	Severe LVSDEF<20%	11	DTB: <90 min HAPPreterminal AKI

AKI, acute kidney injury; DM, diabetes mellitus; DTB, door to balloon time; EF, ejection fraction; HAP, hospital-acquired pneumonia; HTN, hypertension; HTN, hypertension; LAD, left anterior descending; LMS, left main stem; LVSD, left ventricular systolic dysfunction; MR, mitral regurgitation; NSTEMI, non-ST elevation myocardial infarction; PCI, percutaneous coronary intervention; PPCI, primary percutaneous coronary intervention; STEMI, ST elevation myocardial infarction.

About 39.3% (n=11) of patients were treated conservatively (table 3). Mean age of these patients was 78.5 years (range 72.5–89.5) and 36.4% were female (n=4) with 27.3% (n=3) patients with DM. These patients were significantly older (mean age 78.5 years±11.6 years) compared with those who underwent PCI (71.8±11.8 years, p<0.001). They also had a higher Charleson Comorbidity Index (CCMI) (5.81±1.25 vs 5.66±2.26, p<0.0001) and lower 10-year survival percentage (14.5±16.4 vs 26.0±27.68, p=0.01) compared with those who underwent PCI. However, CRP levels and white cell count (WCC) were significantly higher in patients who had PCI (CRP: 134.8±132 vs 102.5±78.3, p<0.0001; WCC: 22.7±8.9 vs 15.2±5.8, p<0.0001) compared with those who were managed conservatively. More than half of the patients managed conservatively (54.5%, n=6) presented late with a STEMI. The the most common reason for conservative management was severe cognitive dysfunction (bedbound patient) in the context of a late-presenting STEMI (45.5%, n=5, table 2). Their in-hospital stay and subsequent demise occurred earlier than those who underwent PCI (days to demise: 3±3.6 days vs 5.3±6.7, p=0.005).

Table 3

Details of patients with type 1 MI managed conservatively

Patient number	Presentation	Reason for conservative management
1	Late-presenting STEMI	Late presentation and severe cognitive dysfunction
2	DKA, sepsis, AKI, severe LVSD, NSTEMI	Previous angiogram no PCI targets
3	Late-presenting STEMI	Late presentation and severe cognitive dysfunction
4	Previous CABG, TIA, severe LVSD. Presented with cardiogenic shock	Patient presented terminally
5	Late-presenting STEMI	Patient presented terminally
6	Critical aortic stenosis and known three vessel disease on angiogram, NSTEMI	Previous surgical turn down, frailty, limited quality of life
7	Heart failure, late-presenting STEMI	Late presentation and severe cognitive dysfunction
8	Generally unwell, severe anaemia, NSTEMI	Severe cognitive dysfunction and severe anaemia
9	Late-presenting STEMI	Late presentation and severe cognitive dysfunction
10	Late-presenting STEMI	Gastrointestinal bleed
11	Ischaemic MR moderate LVSD, NSTEMI	Severe metabolic acidosis, preterminal

AKI, acute kidney injury; CABG, coronary artery bypass graft; DKA, diabetic ketoacidosis; LVSD, left ventricular systolic dysfunction; MR, mitral regurgitation; NSTEMI, non-ST elevation myocardial infarction; PCI, percutaneous coronary intervention; STEMI, ST elevation myocardial infarction; TIA, transient ischaemic attack.

Survival curves demonstrated that most patients died within the first 24–48 hours (figure 4).

Figure 4

Survival curves with TRS quartiles. TRS, Toronto Risk Score.

Discussion

We present the first detailed analysis and reclassification of the SHMI based on the type of MI. Recalculation led to a reduction in the SHMI for MI. Application of an objective prognostic risk score to patients who underwent PCI did not demonstrate deficiency in healthcare or procedural input. In addition, analysis of the patients who were treated conservatively demonstrated appropriateness of conservative management with no deficiency in healthcare.

SHMI for MI was inflated due to the inclusion of type 2 MI and non-MIs. Despite this, the SHMI for MI for the 2017–2018 period at our Trust was still lower than expected. However in other circumstances, this could easily lead to a ‘higher than expected’ SHMI which could be viewed as a ‘smoke alarm’ necessitating further investigation.19 There is no published data on the accuracy of SHMI for MI based on type of MI included. We are one of the first to assess this in detail.

The patients with type 2 MI in our cohort were older with several non-cardiac medical diagnoses and comorbidities. This is in keeping with published literature which has demonstrated that patients with type 2 MI are more likely to be older, female with more comorbidities and poor outcomes despite treatment.20 21 Despite review of the case notes and investigations, in at least five patients of our cohort with type 2 MI, the reason for the diagnosis of MI was unclear. High-sensitivity troponin is widely available in the UK and can detect low levels of troponin released. This, along with lower thresholds for diagnosis of MI have led to increased number of patients being classified as type 2 MI.21 22 Gardezi has demonstrated that troponin is requested frequently on a routine basis, many times as a ‘tick box’ on a blood request form from emergency units, sometimes without clinical justification.23 Provider organisations (hospitals) use the Patient Administration Systems (PAS)24 to input the data for HES to compute the SHMI. This data could be inadequate or inaccurate if information documented in patient notes or in the coding input is insufficient. Hence, inadequate documentation of the diagnosis of MI (sometimes based solely on a troponin level) translates into inaccurate coding of MI with consequent inflation of the SHMI.

There is limited published data on objective assessment of in-hospital mortality post-PCI. This is one of the first studies to apply a prognostic risk score to assess healthcare delivery in patients with MI who did not survive PCI. In the lowest TRS risk group 0, patients did not exhibit any high-risk features (cardiogenic shock, pulmonary oedema, etc). Risk stratification using a prognostic score helped us assess the lowest risk tertile in detail. All three patients underwent uncomplicated PCI but developed hospital-acquired pneumonia and preterminal acute kidney injury. It is difficult to account for these non-quantifiable prognostic contributory factors and comorbidities.25 The high-risk TRS tertiles (groups 1, 2 and 3) included a large proportion of patients with STEMI. These patients had an increased incidence of cardiogenic shock which is known to be the cause of mortality in the first week after STEMI despite PPCI.26

Patients with type 1 MI who were managed conservatively in our patient cohort were much older with multiple comorbidities that precluded coronary intervention. They had a higher CCMI and a lower 10-year survival rate. These patients also had reduced admission to demise time (in days). CCMI is known to be correlated with increased inpatient and 1 year mortality in patients admitted with an acute coronary syndrome.27 The most frequent reason for non-intervention in these patients was severe cognitive impairment. Gharacholou examined outcomes after acute MI in patients with cognitive dysfunction and found less invasive care and worse 1 year survival in those patients with moderate/severe cognitive dysfunction.28 Sloan et al found that patients with dementia were less likely to receive invasive treatment (PCI or coronary artery bypass graft) for MI as compared with patients without dementia.29 Conservative management is often favoured in these patients due to poor short-term and long-term outcomes as intervention makes little or no difference to prognosis.

Limitations

This is a retrospective study over a limited period of 1 year in a small number of patients. Patients from all four quarters from the chosen year were combined to give the annual SHMI. This was performed to ensure adequate patient numbers although we perceive this as one of the major limitations. For definite conclusions, a larger cohort or multicentre analysis is required. Also, the TRS was validated in our population subset and may not apply to other healthcare centres. Mortality modelling with regression-based methods is less accurate than two-factor or three-factor methods.30 Another major limitation is the lack of MPS to quantify the existence of CAD. Future studies which include this quantification will result in a more robust calculation of the number of type 1 and type 2 MIs.

Conclusions

In this analysis of patients with MI, we have demonstrated that SHMI for MI can be inflated due to the inclusion of type 2 MI or non-MI and should be interpreted with caution. Recalculation of this mortality index with inclusion of only type 1 MI probably reflects the correct value. Perhaps future models for healthcare assessment could separate the two types of MI or include only type 1 MI.

Classifying patients with type 1 MI based on invasive or conservative management helps in healthcare assessment. Routine application of a prognostic risk score (not necessarily TRS) to non-survivors of PCI for MI could be initiated on a larger scale for objective assessment of healthcare.