Establishment of a prognostic model based on the Sequential Organ Failure Assessment score for patients with first-time acute myocardial infarction.

OBJECTIVE: This study aimed to identify the prognostic factors of patients with first-time acute myocardial infarction (AMI) and to establish a nomogram for prognostic modeling.
METHODS: We studied 985 patients with first-time AMI using data from the Multi-parameter Intelligent Monitoring for Intensive Care database and extracted their demographic data. Cox proportional hazards regression was used to examine outcome-related variables. We also tested a new predictive model that includes the Sequential Organ Failure Assessment (SOFA) score and compared it with the SOFA-only model.
RESULTS: An older age, higher SOFA score, and higher Acute Physiology III score were risk factors for the prognosis of AMI. The risk of further cardiovascular events was 1.54-fold higher in women than in men. Patients in the cardiac surgery intensive care unit had a better prognosis than those in the coronary heart disease intensive care unit. Pressurized drug use was a protective factor and the risk of further cardiovascular events was 1.36-fold higher in nonusers.
CONCLUSION: The prognosis of AMI is affected by age, the SOFA score, the Acute Physiology III score, sex, admission location, type of care unit, and vasopressin use. Our new predictive model for AMI has better performance than the SOFA model alone.

Introduction

Despite the rate of coronary heart disease significantly declining in most countries over the past few decades, preventing cardiovascular disease is still a matter of great concern.[1] Acute myocardial infarction (AMI) remains the leading cause of death worldwide, and survivors of AMI are at a higher risk of further cardiovascular events.[2] Approximately every 40 s, someone in the United States experiences myocardial infarction. Acute myocardial infarction (AMI) accounts for approximately 80% of patients in cardiogenic shock.[3,4] This is closely related to the health resource services that patients use and the pathological changes caused by changes in trace elements (e.g., melatonin) in their bodies.[5,6] Studies in the United States have shown that the death rate from coronary artery disease has dropped sharply over the past four decades, but this favorable trend does not appear to extend to young people, especially young women. Similarly, the hospitalization rate for AMI in young people has not decreased.[7,8] Therefore, AMI appears to be not only a high-risk disease for elderly people, but also a threat to the health of young people. Fighting a disease requires a focus not only on improving clinical treatment methods, but also an understanding of the many factors that affect the prognosis of the disease.

The Sequential Organ Failure Assessment (SOFA) score was developed in a consensus meeting in 1994. The stated purpose of the assessment was to create a score that reflects the extent of organ dysfunction/failure in the patient population as quantitatively and objectively as possible.[9] Since development of the SOFA score in the early 1990s, it has been integrated into all aspects of intensive care, and is now widely used in daily monitoring of acute onset in intensive care units (ICUs). Moreno et al.[10] found a strong correlation between the SOFA score and mortality. This score performs well and can be used as a discriminant indicator of survival status at discharge from the ICU. In addition to the maximum SOFA score, the change in score or increment in the SOFA score (maximum SOFA score minus the total SOFA score upon admission to the hospital) is also closely related to mortality in the ICU.[10]

The SOFA score is based on scores for functioning of the liver, kidney, and respiratory, cardiovascular, coagulation, and nervous systems.[9,11] In current clinical practice, scoring-based mortality prediction systems, such as the Acute Physiology And Chronic Health Evaluation system, are widely used to determine medications or other treatments for patients admitted to the ICU.[12] However, these scoring systems have substantial limitations, which include the following: (1) usually being limited to a few predictors, (2) poor versatility and only being applicable to subgroups with certain characteristics, and (3) the need for regular recalibration to reflect changes in clinical practice and patients’ demographics.[13]

The present study aimed to identify the factors related to the prognosis of AMI and to establish a predictive model that includes the SOFA score. This prognostic model was designed to be applicable to a wide range of patients and accurate at the individual level. Our model was compared with the SOFA model alone and its performance was verified.

Materials and method

Patients

All patients’ data used in this study were from the Multi-parameter Intelligent Monitoring for Intensive Care (MIMIC) database (https: //mimic.physionet.org/). The MIMIC database is a publicly available dataset developed by the Computational Physiology Laboratory at the Massachusetts Institute of Technology. This database includes health-related data on approximately 60,000 unidentified patients related to ICU visits, such as demographics, vital signs, laboratory tests, and drug information.[14,15]

After completing the web-based training course entitled “Protecting Human Research Participants” of the National Institutes of Health, we were approved to access the MIMIC database (Certificate Number: 38489997). We initially found 2126 records of patients with AMI in the database by searching for the following International Classification of Diseases-9 codes related to AMI: 41000, 41001, 41002, 41010, 41011, 41012, 41020, 41021, 41022, 41030, 41031, 41032, 41040, 41041, 41042, 41050, 41051, 41052, 41080, 41081, 41082, 41090, 41091, and 41092. The exclusion criteria were as follows: (1) not the first diagnosis of AMI (n = 609), (2) missing outcome indicators, or (3) other data of variables were incomplete (n = 532). We found that the minimum age of patients included in the study was 32 years, and therefore, we did not include minors in the exclusion criteria. A flow chart of how the data were obtained is shown in Figure 1.

Figure 1.

Inclusion and exclusion process of the study sample.

MIMIC, Multi-parameter Intelligent Monitoring for Intensive Care; AMI, acute myocardial infarction.

This article does not contain any studies with human participants performed by any of the authors. For this type of study, formal consent is not required. The present study was performed in compliance with the Declaration of Helsinki. Permission was obtained to access the MIMIC program research data. Ethical approval for the study was not required, because for the MIMIC database, analysis is unrestricted once a data use agreement is accepted, and the database was established to ensure the privacy of all patients (http: //www.nature.com/articles/sdata201635). The authors completed the database application and obtained the right to use the database (Record ID 38489997).

Selection and management of variables

We included age, sex, race, marital status, insurance status, admission type, prehospital position, SOFA score, Acute Physiology Score III (APSIII), body mass index (BMI) (calculated from the raw data of the patient’s weight and height), length of stay in the ICU, vasopressin use, and use of mechanical ventilation. Notably, in the MIMIC database, patients older than 89 years are indicated as having an age of 300 years. Therefore, we used 100 years instead of 300 years when processing data because the former age is closer to the actual situation.

Race was divided into white, black, and other, while marital status was divided into married, unmarried, and other. Whether mechanical ventilation was used was set as a binary variable. The survival time was based on the time of hospitalization to the time of death as recorded by the Social Security Bureau. The outcome of this study was death of the patient.

Models and statistical analysis

Multifactor Cox regression analysis was applied to all variables using R software (www.r-project.org). Variables with P < 0.05 were selected for inclusion in the new model and compared with the SOFA model alone. The following indicators were used to judge the prognostic effect of the model: (1) the C-index, which is mainly used to calculate the difference between the predicted value of the Cox model in the survival analysis and the truth, and thus evaluates the predictive ability of the model; (2) the area under the curve (AUC) of receiver operating characteristic (ROC) analysis, which is the standard for determining the pros and cons of a two-class prediction model;[16] (3) a calibration curve for comparing between the actual risk and predicted risk; the closer the curve is to the leading diagonal, the better the actual prediction effect;[17] (4) integrated discrimination improvement (IDI), which represents overall improvement of the model;[18] (5) net reclassification improvement (NRI), which uses quantitative indicators to compare the degree of improvement in diagnostic accuracy of one model compared with another model;[19] and (6) decision-curve analysis (DCA), which is used to judge the clinical net benefits.[20]

The data were divided at a 3:7 ratio into a training set (for estimating the parameters in the model) and a test set (for evaluating the prediction performance of the model), and the model was internally verified. The data were matched using PostgreSQL version 9.6 (IBM Corp., Armonk, NY, USA), and the characteristics of each variable were sorted using Excel version 2019 (Microsoft, Redmond, WA, USA) and IBM SPSS version 25 (IBM Corp.) (Table 1).

Table 1.

Sociodemographic and clinical characteristics of patients in the study.

Variable	Training cohort	Validation cohort
Age (years)	67.17 (28–100)	67.90 (32–100)
SOFA score	3.78 (0–16)	3.53 (0–13)
APSIII	39.94 (7–138)	39.51 (8–116)
Days in the ICU	7.91 (0–100)	7.41 (0–86)
BMI (kg/m2)	28.28 (14.37–62.71)	27.79 (16.65–70.86)
Sex, n (%)
Male	458 (66.5)	194 (65.5)
Female	231 (33.5)	102 (34.5)
Marital status, n (%)
Married	527 (76.5)	225 (76.0)
Unmarried	106 (15.4)	51 (17.2)
Other	56 (8.1)	20 (6.8)
Race, n (%)
White	420 (61.0)	191 (64.5)
Black	27 (3.9)	10 (3.4)
Other	242 (35.1)	95 (32.1)
Insurance, n (%)
Government	20 (2.9)	8 (2.7)
Medicare	375 (54.4)	160 (54.1)
Medicaid	38 (5.5)	16 (5.4)
Private	256 (37.2)	112 (37.8)
Type of admission, n (%)
Elective	6 (0.9)
Emergency	620 (90.0)	267 (90.2)
Urgent	63 (9.1)	29 (9.8)
Location, n (%)
Clinic	80 (11.6)	25 (8.4)
Phys	26 (3.8)	7 (2.4)
Hospital	392 (56.9)	183 (61.8)
Emergency	191 (27.7)	81 (27.4)
Vasopressin use, n (%)
Yes	345 (50.1)	154 (52.0)
No	344 (49.9)	142 (48.0)
Type of care, n (%)
CCU	518 (75.2)	223 (75.3)
CSRU	138 (20.0)	61 (20.6)
MICU	26 (3.8)	10 (3.4)
SICU	4 (0.6)	1 (0.3)
TSICU	3 (0.4)	1 (0.3)
Ventilation use, n (%)
Yes	327 (47.5)	137 (46.3)
No	362 (52.5)	159 (53.7)

SOFA, Sequential Organ Failure Assessment; APSIII, Acute Physiology Score III; ICU, intensive care unit; BMI, body mass index; Phys, physiotherapy referral; CCU, coronary heart disease intensive care unit; CSRU, cardiac surgery intensive care unit; MICU, medical intensive care unit; SICU, stroke intensive care unit; TSICU, surgical intensive care unit.

Age, SOFA score, APSIII, length of stay, and BMI were included as continuous variables, and were statistically analyzed using their mean (range) values. Categorical variables included sex, race, marital status, insurance status, type of admission, place of admission, use of vasopressin, type of inpatient ward, and use of mechanical ventilation. The numbers and proportions of cases in subgroups of the categorical variables were counted.

Results

The final sample size in the study was 985 patients. The sociodemographic and clinical characteristics of patients in the study are shown in Table 1. There were nearly twice as many men as women in the sample, and more married than unmarried patients and those of unknown marital status. White people accounted for a larger proportion than black people and other races. Medicare insurance was the most common type of insurance and there was a high rate of emergency hospital admissions, among which coronary heart disease intensive care unit (CCU) and cardiac surgery intensive care unit (CSRU) admission predominated. Approximately half of the patients were taking pressurized drugs and using mechanical ventilation.

Because AMI is acute and has a rapid onset, studies have investigated the 30- and 90-day readmission rates of AMI.[21] We examined the 30-, 60-, and 90-day survival rates. All of the initially selected variables were incorporated into the model and Cox regression analysis was performed. The patients’ outcomes were significantly affected by age at diagnosis (hazard ratio [HR]=1.03, 95% confidence interval [CI]=1.02–1.04, P < 0.001), SOFA score (HR=1.13, 95% CI=1.07–1.20, P < 0.001), APSIII (HR=1.02, 95% CI=1.01–1.02, P < 0.001), female sex (HR=1.54, 95% CI=1.19–2.01, P=0.001), other race (HR=1.40, 95% CI=1.07–1.82, P=0.013), outpatient referral (HR=0.47, 95% CI=0.29–0.74, P=0.001), vasopressin use (HR=1.36, 95% CI=1.01–1.85, P=0.047), and CSRU admission (HR=0.55, 95% CI=0.39–0.77, P < 0.001) (Table 2).

Table 2.

The results of all factors in Cox regression analysis.

Variable	HR	95% CI	P value
Age	1.03	1.02–1.04	<0.001
SOFA score	1.13	1.07–1.20	<0.001
APSIII	1.02	1.01–1.02	<0.001
Sex
Male	Reference
Female	1.54	1.19–2.01	0.001
Race
White	Reference
Black	0.94	0.47–1.89	0.863
Other	1.4	1.07–1.82	0.013
Location
Room admit	Reference
Transfer	0.87	0.65–1.17	0.354
Normal delivery	1.45	0.78–2.70	0.239
Outpatient referral to ICU	0.47	0.29–0.74	0.001
Vasopressin use
Yes	Reference
No	1.36	1.01–1.85	0.047
ICU type
CCU	Reference
CSRU	0.55	0.39–0.77	<0.001
MICU	1.6	0.98–2.63	0.061
SICU	3.39	0.83–13.92	0.090
TSICU	<0.001	<0.001	0.989

HR, hazard ratio; CI, confidence interval; SOFA, Sequential Organ Failure Assessment; APSIII, Acute Physiology Score III; room admit, emergency room admission; transfer, transfer from hospital; ICU, intensive care unit; CCU, coronary heart disease intensive care unit; CSRU, cardiac surgery intensive care unit; MICU, medical intensive care unit; SICU, stroke intensive care unit; TSICU, surgical intensive care unit.

Based on the results shown above, we constructed a nomogram of a new model that included the SOFA score (Figure 2). This nomogram showed that the prognosis for AMI was worse for older patients, a higher SOFA score, a higher APSIII, female sex, and other races, whereas referrals, vasopressin use, and CSRU admission were protective factors. A higher score in the nomogram indicated a greater risk, with a HR >1 indicating a risk factor and a HR  < 1 indicating a protective factor. The factors that had the largest effect on AMI were age, SOFA score, APSIII, and type of inpatient.

Figure 2.

Nomogram for predicting 30-, 60-, and 90-day probability of survival from acute myocardial infarction.

SOFA, Sequential Organ Failure Assessment; APSIII, Acute Physiology Score III; vaso, vasopressin; CCU, coronary heart disease intensive care unit; CSRU, cardiac surgery intensive care unit; MICU, medical intensive care unit; SICU, stroke intensive care unit; TSICU, surgical intensive care unit.

The C-indices in the training and test sets were 0.781 and 0.761, respectively, for the new model based on the SOFA score. These indices were markedly higher than those of 0.694 and 0.665, respectively, for the SOFA model alone. Figure 3a–c and 3d–f show the ROC curves of the training and test sets, respectively, for the new model combined with the SOFA model. The AUC was larger for the new model than for the SOFA model alone.

Figure 3.

ROC curves. The area under the ROC was used to evaluate the performance of the new nomogram. (a–c) Results of the training cohort. (d–f) Results of the test cohort.

ROC, receiver operating characteristic; AUC, area under the curve; SOFA, Sequential Organ Failure Assessment.

Figure 4a–c and 4d–f show the calibration curves of the training and test sets, respectively. The calibration curves compared the actual and predicted risks, and the calibration curves were close to the leading diagonal in the figure. Moreover, the four tangent points were near the curve. This finding indicated that the real prediction performance of the new model was excellent, and that the new model represented a marked improvement over the SOFA model alone.

Figure 4.

Calibration curves. Calibration curves for 30-, 60-, and 90-day probability of survival from acute myocardial infarction show calibration of each model in terms of the agreement between the predicted probabilities and observed outcomes of the training cohort (a–c) and validation cohort (d–f).

IDI and NRI also indicated good performance of the new model (Table 3). The 30-, 60-, and 90-day IDI values were 0.078, 0.087, and 0.092 for the training set, and 0.091, 0.096, and 0.102 for the test set, respectively. All of the IDI values for the test and training sets were higher than 0 (P < 0.001), which indicated that the newly established model performed better overall than the SOFA model alone. The 30-, 60-, and 90-day NRI values were 0.412, 0.442, and 0.465 for the training set, and 0.683, 0.765, and 0.656 for the test set, respectively. All of the NRI values were also higher than 0, and therefore, had no zero crossing points. This finding indicated that the accuracy of the new model was better than that of the SOFA model alone.

Table 3.

IDI and NRI values of the test and training sets

	IDI				NRI
Time	Training set	P	Test set	P	Training set	Lower–upper	Test set	Lower-upper
30 days	0.078	≤0.001	0.091	≤0.001	0.412	0.231–0.685	0.683	0.331–0.952
60 days	0.087	≤0.001	0.096	≤0.001	0.442	0.201–0.710	0.756	0.294–1.007
90 days	0.092	≤0.001	0.102	≤0.001	0.465	0.273–0.737	0.656	0.271–0.927

IDI, integrated discrimination improvement; NRI, net reclassification improvement.

DCA curves for the new model and the SOFA model are shown in Figure 5. The AUC was larger for the new model than for the SOFA model. This finding indicated that the net clinical benefit of the new model was better than that of the SOFA model alone.

Figure 5.

Decision curves for a nomogram for 30-, 60-, and 90-day prediction of mortality of acute myocardial infarction in the training set (a) and validation set (b). All of the red lines in the figure are above the green lines, and therefore, the area under the curve is larger for the new model than for the Sequential Organ Failure Assessment model.

Discussion

Nearly half of the adults in the United States are estimated to have some form of heart disease by 2035, for which the medical treatment costs will exceed $1.1 trillion.[22] Approximately 720,000 Americans will be hospitalized for the first time owing to AMI or coronary heart disease, and 1 in 7.4 of them will die of AMI. Additionally, 170,000 of approximately 805,000 cases of AMI per year are silent or without classic symptoms, such as chest pain, shortness of breath, and indigestion.[22] Affected people must simultaneously deal with AMI in multiple ways, such as by prevention, clinical treatment, and rehabilitation. The present study focused on the sociodemographic prognostic risk factors for AMI as a first diagnosis to establish a multifactor predictive model that includes the SOFA score and APSIII. We found good performance of our new model.

Predictive models have been increasingly used in hospital settings to assist in risk prediction, prognosis, diagnosis, and treatment planning, with the ultimate aim of producing better health outcomes for patients. Predictive modeling can be used to develop personalized care strategies based on the health characteristics of each patient.[13] The current study examined new prognostic factors for AMI based on the SOFA score, and established a nomogram to visually describe the new model. The performance of the C-index, AUC, calibration curve, IDI, NRI, and DCA in the new model was better than that in the single SOFA model. This finding suggests that this nomogram will be a reliable aid for doctors in making decisions. The nomogram in our study showed that being older was a risk factor for AMI. Using coronary angiography results, Wang et al.[23] found that the most frequent coronary lesion in young patients with AMI was lesions with one branch (62.4%) and secondary injury was limited. These authors also found that older patients with AMI had more multiple branch lesions and calcified lesions, which had a serious effect on cardiac function. Some studies have shown that although the prevalence of AMI is increasing in younger people, older patients still comprise the main affected group.[7,8,24] The SOFA score and APSIII have repeatedly been shown to be related to the death of critically ill patients, with higher scores associated with a higher probability of death.[25-28] The score in the present nomogram similarly increased with the SOFA score and APSIII. In our study, female patients with AMI were at a higher risk than male patients, which may be related to sex-related differences in physiological factors and the ability to resist stress. Female patients presenting with AMI are often older, have higher rates of diabetes mellitus, hypertension, and autoimmune disorders, have a worse Killip class, higher Global Registry of Acute Coronary Events risk scores, and lower weight, baseline hemoglobin levels, and creatinine clearance.[29] Laura Barrett et al.[21] also showed the same findings. Most studies have shown that the short-term and long-term mortality rates after AMI are higher in women than in men.[30,31]

The present study classified race into white, black, and other races, and the model indicated that the other race category was a risk factor. This finding may be related to how patients who are not originally from the United States are treated. A lack of understanding of treatment policies and communication difficulties between patients and doctors will indirectly increase the difficulty of treatment. However, because the population of the United States mainly comprises black and white populations, such results may also be biased by the patients who were selected for inclusion in this study.

With regard to the type of admission, the prognosis of AMI was better for outpatient referrals. Outpatient referrals are already under health supervision, and doctors have a more comprehensive understanding of these conditions. The most appropriate treatments can be adopted to increase the likelihood of a good prognosis. The prognosis of patients with AMI who do not use booster drugs is poor because pre-onset manifestations of AMI are usually not obvious.[22] Therefore, obvious manifestations of certain symptoms, such as increased blood pressure, can alert the doctor to the patient’s condition in a timely manner and facilitate prescribing the correct medicine to avoid a poor prognosis. Our study showed that patients with AMI in the CSRU had a good prognosis, which may have been due to comprehensive monitoring of the heart, early detection of changes in disease, and timely treatment. Age, the SOFA score, the APSIII, and the type of inpatient had the largest effect in our new model.

Obesity might be a risk factor for coronary heart disease and is also related to the prognosis of AMI.[32,33] In our model, BMI was not a prognostic risk factor for AMI, but this “obesity paradox” (inverse relationship between BMI and mortality) can be explained by more aggressive treatments of patients with obesity or confounding factors, such as age and sex.[34] Specifically, a prediction model for mortality estimates the patient’s likelihood of death based on their characteristics, including the severity of the disease and many other risk factors related to death.[35] These are important supplementary tools for assisting clinical decision-making.[36,37]

To the best of our knowledge, the present study represents the first attempt to construct nomograms on the basis of the SOFA score for predicting AMI as the first diagnosis in the general population. The results of the present study might be useful as a reference for doctors in making decisions about the diagnosis and rehabilitation of patients with AMI. We will continue to investigate more comprehensive prognostic factors, including obtaining more laboratory data, to increase the understanding of AMI and thereby improve the outcomes of patients with AMI.

Conclusion

Our study shows that an older age, a higher SOFA score, and a higher APSIII are risk factors for the prognosis of AMI as a first diagnosis. The risk of further cardiovascular events is 1.54-fold higher in women than in men. Races other than black and white are at a higher risk of AMI, and patients in the CSRU have a better prognosis than patients in other ICUs. The use of vasopressin is a protective factor, and the risk of further cardiovascular events patients is 1.36-fold higher in those who do not use pressurized drugs. A nomogram based on these findings in which performance was evaluated using the C-index, AUC, standard curve, IDI value, NRI value, and DCA curve showed excellent performance of the model.

Limitations

This study has several limitations. First, the data used in this study were from the MIMIC database. The majority of the included patients were residents of the United States, which restricts generalizability of the present results. Second, because this database contains numerous variables, completely recording the value of each indicator for every patient was difficult. Therefore, there were missing data for some indicators for many patients, which decreased the sample size. Third, in this study, we extracted the records of the first diagnosis of AMI and the first admission, and some patients had been admitted to hospital multiple times. This resulted in some missing reference values for subsequent measurements. Fourth, the model did not include laboratory data. This is because the original data recorded in the MIMIC database cannot be used directly, these data are difficult to obtain, and the data can be used only after multiple processing and conversion steps. Additionally, this database records patients older than 89 years as 300 years old. Therefore, to avoid decreasing the sample size and to be realistic, we uniformly used the age of 100 years instead for these older patients. Finally, this was a retrospective study, and therefore, some information bias and selection bias were inevitable.