Machine Learning to Predict the Likelihood of Acute Myocardial Infarction.

BACKGROUND: Variations in cardiac troponin concentrations by age, sex and time between samples in patients with suspected myocardial infarction are not currently accounted for in diagnostic approaches. We aimed to combine these variables through machine learning to improve the assessment of risk for individual patients.
METHODS: A machine learning algorithm (myocardial-ischemic-injury-index [MI3]) incorporating age, sex, and paired high-sensitivity cardiac troponin I concentrations, was trained on 3,013 patients and tested on 7,998 patients with suspected myocardial infarction. MI3 uses gradient boosting to compute a value (0-100) reflecting an individual's likelihood of a diagnosis of type 1 myocardial infarction and estimates the sensitivity, negative predictive value (NPV), specificity and positive predictive value (PPV) for that individual. Assessment was by calibration and area under the receiver-operating-characteristic curve (AUC). Secondary analysis evaluated example MI3 thresholds from the training set that identified patients as low-risk (99% sensitivity) and high-risk (75% PPV), and performance at these thresholds was compared in the test set to the 99th percentile and European Society of Cardiology (ESC) rule-out pathways.
RESULTS: Myocardial infarction occurred in 404 (13.4%) patients in the training set and 849 (10.6%) patients in the test set. MI3 was well calibrated with a very high AUC of 0.963 [0.956-0.971] in the test set and similar performance in early and late presenters. Example MI3 thresholds identifying low-risk and high-risk patients in the training set were 1.6 and 49.7 respectively. In the test set, MI3 values were <1.6 in 69.5% with a NPV of 99.7% (99.5%-99.8%) and sensitivity of 97.8% (96.7-98.7%), and were ≥49.7 in 10.6% with a PPV of 71.8% (68.9-75.0%) and specificity of 96.7% (96.3-97.1%). Using these thresholds, MI3 performed better than the ESC 0/3-hour pathway (sensitivity 82.5% [74.5-88.8%], specificity 92.2% [90.7-93.5%]) and the 99th percentile at any time-point (sensitivity 89.6% [87.4-91.6%]), specificity 89.3% [88.6-90.0%]).
CONCLUSIONS: Using machine learning, MI3 provides an individualized and objective assessment of the likelihood of myocardial infarction, which can be used to identify low-risk and high-risk patients who may benefit from earlier clinical decisions. CLINICAL TRIAL REGISTRATION: Unique Identifier: Australian New Zealand Clinical Trials Registry: ACTRN12616001441404. URL: https://www.anzctr.org.au.

In an international collaboration involving 11 011 patients from 9 countries, we used machine learning to train and test a novel algorithm that estimates the probability of myocardial infarction for an individual patient.

The myocardial-ischemic-injury-index (MI3) algorithm generates a value that takes into consideration age, sex, paired cardiac troponin I concentrations, and rate of change in troponin concentration, to estimate the negative and positive predictive value for each patient value associated with these measures.

This represents one of the first effective demonstrations of how machine learning could be used to guide clinical decision making in patients with suspected acute coronary syndrome.

The MI3 algorithm is more versatile than existing algorithms because the former is not dependent on fixed cardiac troponin thresholds, does not require serial testing to be performed at specific time points, and recognizes that different healthcare systems have different priorities and tolerances of risk.

Prospective studies are now required to evaluate patient outcomes and resource use after implementation of the MI3 algorithm into clinical practice.

The use of cardiac troponin testing in clinical practice is evolving rapidly.[1-3] Advances in assay analytical precision now permit quantification of cardiac troponin concentrations in the normal reference range with novel applications for early diagnostics and risk stratification in patients being assessed for possible acute coronary syndromes.[4] In the past, international guidelines have recommended the use of serial cardiac troponin measurements over 6 to 12 hours to define myocardial infarction in patients with a rise or fall in cardiac troponin concentration where at least 1 of the serial measured concentrations is above the 99th percentile.[5,6] However, some studies have challenged this approach suggesting lower thresholds can risk stratify patients to low, intermediate, or high risk of myocardial infarction when using high-sensitivity cardiac troponin assays.[7-15]

These strategies have been incorporated into accelerated diagnostic pathways that advocate earlier troponin measurement at presentation and 1 to 3 hours later to facilitate prompt diagnosis and treatment in those with myocardial infarction or to expedite discharge in those without.[16,17]

The performance of these pathways varies across different populations, reflecting variation in cardiac troponin concentrations with age and sex. [13,18-21] This heterogeneity is not reflected in strategies which advocate the use of fixed thresholds for all patients, which only allow classification of patients as low, intermediate, or high risk and that do not reflect more subtle variations in risk.[6] Machine learning has been advocated as an objective, replicable, approach to integrate multiple quantitative variables to improve diagnostic accuracy.[22,23]

We aimed to test an algorithm, the myocardial-ischemic-injury-index (MI3), which had been trained by machine learning to estimate an individual patient’s likelihood of myocardial infarction.

Methods

Transparency and Openness Promotion

The analysis code for this study is available on request. The algorithm is proprietary and subject to a patent application, but we can share it with researchers who agree to use it only for research purposes with a data sharing agreement.

Study Design

This study was an analysis of prospectively collected data from multiple centers to train and test the MI3 algorithm to predict the diagnosis of type 1 myocardial infarction. The training set comprised data from 2 cohorts[14,24] and the test set comprised data from 7 cohorts of patients attending the emergency department with suspected myocardial infarction.[15,25-29]

Training and testing are the nomenclature of machine learning and are analogous to derivation and validation in studies of new diagnostic biomarkers.

MI3 incorporates age, sex, paired cardiac troponin I concentrations at presentation and at another early, yet flexible, time point, and rate of change of cardiac troponin I concentration. These variables (features) were selected a priori because they are (1) objective and automatically captured from electronic hospital records, (2) include serial measurements as recommended by guidelines, and (3) associated with the diagnosis of myocardial infarction. MI3 computes a value from 0 to 100 (the MI3 value), which reflects the likelihood of a diagnosis of type 1 myocardial infarction for each patient during hospitalization (higher values indicate greater likelihood). The algorithm uses an embedded reference table to report for each individual patient estimates of sensitivity, negative predictive value (NPV), specificity, and positive predictive value (PPV) of the diagnosis for a given MI3 value. MI3 was developed on the training data set by Abbott Diagnostics using a machine learning technique called gradient boosting. This technique iteratively trains a set of sequential weak learners (here decision trees) using the provided features to map onto the outcome (whether the patient was or was not diagnosed with myocardial infarction). For further details regarding the gradient boosting method, see also the online-only Data Supplement.[30] This is analogous to, but more complex than, the β-coefficient weightings of a logistic regression model. The algorithm was provided to an independent statistician, J.P. who evaluated its performance in the test set. J.P. had full access to all the test set data and takes responsibility for its integrity and the analysis.

We report according to relevant sections of the Transparent Reporting of multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) statement.[31] The study was registered on the Australian New Zealand Clinical Trials Registry (URL: http://www.anzctr.org.au. Unique identifier: ACTRN12616001441404).

Participants and Cohorts

Patients presenting with symptoms suggestive of myocardial infarction in whom serial cardiac troponin measurements were obtained were included. Patients with ST-segment elevation myocardial infarction (STEMI) at presentation were excluded. Cohorts were identified for inclusion if they were prospective, had cardiac troponin I concentrations measured with the Abbott ARCHITECT high-sensitivity assay (Abbott Diagnostics, Chicago, IL) at presentation and at a second time point approximately 1 to 3 hours later (details in the online-only Data Supplement), the final diagnosis was adjudicated according to the Universal Definition of Myocardial Infarction,[5] and ethical approval permitted sharing of individual patient-level data (Table I in the online-only Data Supplement). All cohort studies were conducted in accordance with the Declaration of Helsinki and approved by the local research ethics committee or institutional review board. Written informed consent was obtained where this was required. All adjudication was completed before developing the MI3 algorithm.

Outcome Definitions and Adjudication

The primary outcome was the adjudicated diagnosis of type 1 myocardial infarction during the index admission. Although high-sensitivity cardiac troponin I was measured in all patients, other cardiac troponin assays were used for adjudication in some cohorts (Table I in the online-only Data Supplement).

Algorithm Development

A gradient boosting model was developed using predefined features (age, sex, paired cardiac troponin I concentrations at presentation and at another early, yet flexible, time point, and rate of change of cardiac troponin I concentration) to estimate the likelihood of a diagnosis of type 1 myocardial infarction. Once the model was trained, it was used to generate MI3 values for each patient in the test sets.

Primary Analysis

We describe algorithm performance in the test set by (1) visual inspection of a calibration curve to show how accurately MI3 values estimate the likelihood of myocardial infarction and (2) by the area under the receiver operating characteristic curve (AUC) to quantify how well the MI3 values discriminated between those with and without myocardial infarction. In addition, we compared diagnostic metric outputs from the algorithm (sensitivity, NPV, specificity, and PPV) for each patient with the metrics determined in the test set using each individual’s MI3 value as a threshold.

Secondary Analyses

MI3 is designed to be used as a continuous measure. However, we recognize that in this field most tools rely on thresholds to guide clinical decisions. Therefore, as illustrative examples of how an individual hospital may choose to use MI3, we demonstrate its diagnostic performance at 2 exemplar MI3 value thresholds. First, we determined the MI3 values with their 95% CIs that gave a sensitivity ≥99.0% or NPV ≥99.5% in the training set and assessed the accuracy of these example threshold values in the test set. These diagnostic criteria were prespecified and based on an international survey of acceptable risk by emergency department physicians,[32] and prior prospective studies defining risk stratification thresholds for high-sensitivity cardiac troponin.[14] Second, we determined the MI3 values that gave a specificity ≥90% and a PPV ≥75% in the training set based on consensus of the project steering committee, and assessed their performance in the test set. We used 1000 bootstrapped samples to determine these MI3 thresholds and their 95% CIs. All analyses were performed in R (version 3.2.4: The R Foundation for Statistical Computing).

Additional Analyses

Prespecified subgroup analyses were performed by age, sex, comorbidities (coronary artery disease, diabetes mellitus, hypertension, current smoking), time from symptom onset to first sample draw, time between tests, and the presence or absence of myocardial ischemia on the electrocardiogram. Performance of the algorithm was also evaluated for the outcomes of type 1 myocardial infarction within the next 30 days and for type 1 or 2 myocardial infarction on index admission. Last, we compared the performance of MI3 using the example thresholds derived from our training set with the 99th percentile at any time point, and the European Society of Cardiology (ESC) 0/1-hour and 0/3-hour pathways[6] for a diagnosis of type 1 myocardial infarction in our test set.

Results

The training set comprised 3013 patients of whom 404 (13.4%) had a diagnosis of type 1 myocardial infarction. The set was predominantly male (63%), with a mean age of 62.4 years (Table 1). The test set comprised 7998 patients, 62% male, with a mean age of 58.8 years and mean time between samples of 2.5 hours (SD, 1.2 hours). Of these patients, 849 (10.6%) had a diagnosis of type 1 myocardial infarction. There were no missing data for any of the variables used in the training and testing sets. Patients in the testing set were younger, less likely to have known coronary artery disease, but more likely to smoke cigarettes, have diabetes mellitus, dyslipidemia, or a family history of coronary artery disease than those in the training set.

Table 1.

Baseline Characteristics of Training and Testing Sets

Primary Analysis: Calibration and Discrimination

The MI3 algorithm was well calibrated and discriminated between those with and without type 1 myocardial infarction (AUC, 0.963 [95% CI, 0.957–0.968], Figure 1). Compared to the MI3 output estimated metrics, the sensitivity was similar, specificity and NPV were slightly higher, and the PPV marginally lower in the test set (Figure I in the online-only Data Supplement).

Figure 1.

Calibration and discrimination of the myocardial-ischemic-injury-index (MI Calibration of the MI3 algorithm with the observed proportion of patients with type 1 myocardial infarction in the test data set (A). Each point represents 100 patients. The dashed lines represent perfect calibration. Receiver operating characteristic curve showing discrimination of the MI3 algorithm in the test data set (B). Some MI3 values shown for illustrative purposes only.

There was no difference in AUCs for those presenting within 3 hours (0.966 [0.959–0.973]) compared with later presenters (0.965 [0.959–0.972]) and no difference when stratifying by sex (men 0.962 [0.955–0.969] and women 0.962 [0.952–0.973]; Figure II in the online-only Data Supplement). The AUCs were higher in patients with no prior history of coronary artery disease, diabetes mellitus, or hypertension compared with patients with these comorbidities. The AUC was higher in younger compared with older patients and in those with no myocardial ischemia on the ECG compared with those with ischemia.

Secondary Analysis: Example Diagnostic Thresholds

The MI3 threshold values from the training set that corresponded to our prespecified diagnostic performance metrics were 1.6 (0.9–3.0; sensitivity ≥99.0%), 3.1 (1.7–4.7; NPV ≥99.5%), 17.2 (13.8–21; specificity ≥90.0%), and 49.7 (36.6–60.0; PPV ≥75%, Table 2). In the test set, MI3 values of 1.6 and 3.1 gave a sensitivity of 97.8% (96.7%–98.7%) and an NPV of 99.4% (99.2%–99.6%), respectively. MI3 values of 17.2 and 49.7 gave a specificity of 91.7% (91.1%–92.3%) and a PPV of 71.8% (68.9%–75.0%) (Table 2).

Table 2.

Performance in the Testing Set of Example MI3 Threshold Values From the Training Set

If, for example, patients with MI3 values <1.6 were to be classified as low risk of myocardial infarction then this threshold would identify 69.4% (68.4%–70.4%) as low risk of whom 0.5% (0.3%–0.7%) would be false negatives. If patients with MI3 values ≥49.7 were classified as high risk, then this threshold would identify 10.6% (10.0%–11.2%) as high risk of whom 28.1% (25.1%–31%) would be false positives (Table II in the online-only Data Supplement). These 2 exemplar thresholds were used for all subsequent analysis. The MI3 threshold value of 1.6 performed similarly across all subgroups including those who presented within 3 hours of symptom onset (sensitivity 98.7% [97.3%–100%] and NPV 99.8% [99.7%–100%]; Figure III in the online-only Data Supplement). The MI3 threshold value of 49.7 also performed similarly across most groups with the exception of sex and time from symptom onset, where the PPV was lower in women than men, and in those presenting within 3 hours compared with those presenting more than 3 hours from symptom onset (Figure IV in the online-only Data Supplement).

Secondary Analysis: Type 1 Myocardial Infarction Within 30 Days

In addition to the 849 (10.6%) patients with type 1 myocardial infarction during the initial hospitalization there were 23 (2.9%) with myocardial infarction following discharge within the next 30 days. The MI3 value discriminated between those patients with and without type 1 myocardial infarction within the next 30 days with an AUC of 0.957 (0.951–0.963). Threshold values of 1.6 and 49.7 gave a sensitivity of 96.6% (95.3%–97.8%) and PPV of 71.9% (69.0%–74.9%), respectively (Table II in the online-only Data Supplement).

Secondary Analysis: Type 1 or 2 Myocardial Infarction on Presentation

In addition to the 849 (10.6%) patients with type 1 myocardial infarction there were 216 (2.7%) with type 2 myocardial infarction during the initial hospitalization. The MI3 value discriminated between those with and without type 1 or type 2 myocardial infarction with an AUC of 0.963 (0.957–0.968, Figure V in the online-only Data Supplement). The example low-risk MI3 threshold, 1.6, had a sensitivity of 97.4% (96.3%–99.5%), and an NPV of 99.5% (99.3%–99.7%), identifying 69.5% of patients as low risk. The example high-risk threshold, 49.7, had a specificity of 97.7% (97.3%–98.0%), and a PPV of 80.8% (78.1%–83.5%), identifying 10.6% of patients as high risk (Table III in the online-only Data Supplement).

Secondary Analysis: Comparison With Other Recommended Diagnostic Strategies

In all 7998 patients in the test set, the 99th percentile upper reference limit at any time-point identified 6473 (80.9%) patients as low risk (NPV, 98.6% [98.3%–98.9%]; sensitivity, 89.6% [87.4%–91.6%]) and 1525 (19.1%) as high risk (PPV, 49.9% [47.4%–52.4%]; specificity, 89.3% [88.6%–90.0%]). A total of 1652 patients (21%) from the test set were eligible for inclusion in the analysis of the ESC 0/3-hour pathway with ≥2.5 hours between serial samples (Figure 2). This pathway identified 86.7% (1433/1652) of patients as low risk (NPV, 98.5% [97.8%–99.1%]; sensitivity, 82.5% [74.5%–88.8%]) with 21 missed events, and 13.3% (219/1652) of patients as high risk (PPV, 45.2% [38.5%–52.1%]; specificity, 92.2% [90.7%–93.5%]). In the same patient group, MI3 <1.6 or ≥49.7, identified 70.7% (1168/1652) of patients as low risk (NPV, 99.9%; [99.5%–100%]; sensitivity, 99.2% [95.4%–100%]) with 1 missed event, and 9.0% (149/1652) of patients as high risk (specificity, 95.9% [94.8%–96.8%]; PPV, 57.7% [49.4%–65.8%]), respectively. Use of these thresholds identified 20.3% (335/1652) of patients as intermediate risk (MI3, 1.6 to 49.6), of whom 33 patients had an event.

Figure 2.

Performance of the myocardial-ischemic-injury-index (MI Performance of MI3 at example thresholds (A) and the ESC 3-hour algorithm (B) for high-sensitivity cardiac troponin I (hs-cTnI) in 1652 patients with ≥2.5 hours between serial samples in the test set. URL indicates upper reference limit.

Only 336 patients (4%) from the test set were eligible for inclusion in the analysis of the ESC 1-hour pathway with >0.5 hour but ≤1.5 hours between serial samples (Figure 3). This pathway identified 54.3% (183/336) of patients as low risk (NPV, 100% [98.0%–100%]; sensitivity, 100% [93.2%–100%]) with no missed events, and 18.3% (61/336) of patients as high risk (PPV, 75.4% [62.7%–85.5%]; specificity, 94.7% [91.4%–97.0%]), and 27.4% (92/336) of patients as intermediate risk, of whom 6 had an event. Here, MI3 <1.6 or ≥49.7, identified 64.3% (216/336) of patients as low risk (NPV, 100% [98.3%–100%]; sensitivity, 100% [93.2%–100%]) with no missed events, and 14.9% (50/336) of patients as high risk (specificity, 97.2% [94.5%–98.8%]; PPV, 84.0 [70.9%–92.8%]), respectively. Use of these thresholds identified 20.8% (70/336) of patients as intermediate risk (MI3, 1.6–49.6), of whom 10 patients had an event.

Figure 3.

Performance of the myocardial-ischemic-injury-index (MI Performance of MI3 at sample thresholds (A) and the ESC 1-hour algorithm (B) for high-sensitivity cardiac troponin I (hs-cTnI) in 336 patients with >0.5 hour but ≤1.5 hours between serial samples in the test set.

Discussion

In a large, international, multicenter study of more than 11 000 patients with suspected myocardial infarction, we used machine learning to train and test a novel decision tool that incorporates simple, objective variables known to be associated with the diagnosis of myocardial infarction to accurately predict the likelihood of a diagnosis of myocardial infarction. The algorithm was well calibrated, and the overall diagnostic performance was identical in both training and test data sets. This study has several unique and important characteristics.

First, this technique provides an individualised and precise assessment of risk by using age, sex, and paired high-sensitivity cardiac troponin I concentrations and allows for the complex and nonlinear ways in which these variables may interact. This contrasts with contemporary algorithms in clinical use which are based on fixed time-points for sampling, fixed troponin thresholds, and do not account for any interaction between input variables. One exception is the Troponin only-Manchester Acute Coronary Syndrome rule which uses a logistic regression model to provide risk estimates for myocardial infarction, incorporating age, sex, multiple clinical variables, and a single high-sensitivity troponin T measurement.[33,34]

They reported an AUC of 0.90. This model has several strengths but does not take into account dynamic interaction between variables and has to date only been assessed using thresholds allocating patients into 1 of 4 risk categories.[35,36]

Second, MI3 recognises the importance of both the magnitude and rate of change in cardiac troponin concentration for the diagnosis of myocardial infarction, without applying a fixed absolute or percentage change in concentration or mandating specific time-points for serial testing. MI3 performed well across a range of time intervals between sampling. This enhances its transferability because variation in the sample timing is commonplace in busy emergency departments. It is important to note that there was no difference in performance when stratified by time from symptom onset, which means that unlike some other approaches, MI3 can be applied in those patients presenting early (ie, within 3 hours of symptom onset). This is important because early presenters are a sizable subset of patients (34% of patients in the test cohort) and because the time of symptom onset is often uncertain.

Third, the large cohort size and number of patients with type 1 myocardial infarction allowed for a robust analysis of subgroups.

Fourth, at the example thresholds MI3 was better than the 99th percentile alone or the ESC 0/3-hour pathway at identifying low- and high-risk patients. Consistent with previous reports,[18,20,21] both of these approaches gave a low diagnostic sensitivity and poor positive predictive value despite being widely used in clinical practice.[37] MI3 compared well to the ESC 0/1-hour pathway, with the primary advantage of MI3 being flexibility in the timing of serial testing, and the simplicity of using probabilities rather than multiple thresholds to stratify risk in individual patients.

Fifth, the algorithm performed well even when type 2 myocardial infarction was included as an outcome event.

Last, MI3 provides guidance on high-risk patients reporting the PPV and specificity for type 1 myocardial infarction that could be used to initiate earlier treatment or expedite cardiology consultation. Previous attempts to optimise specificity for myocardial infarction have used absolute or relative changes in troponin concentrations to differentiate from chronic myocardial injury or have recommended thresholds well above the 99th percentile.[16,17,19,38]

MI3 avoids the need to decide on the use of relative or absolute changes or on a threshold for change a priori. The developed machine-learning model uses the rate of concentration change, patient age, and sex to decide on the weighting of relative and absolute troponin concentration changes.

These features are presented to the gradient boosting algorithm which outputs the MI3 value which can provide a clinical decision support tool for the assessment of all patients with suspected myocardial infarction. The clinical decision support tool also reports the diagnostic parameters associated with the calculated MI3 value. These diagnostic parameters cannot be derived for an individual patient, and therefore the tool uses an embedded reference table to report estimates of sensitivity, specificity and negative and positive predictive values from our training set alongside the calculated MI3 value. This tool is easy to implement in practice, is objective because it does not rely on potentially inconsistent assessment of symptoms or patient history, and provides accurate estimates of the likelihood of myocardial infarction for each individual patient to aid clinical decision making.

We are aware of 2 attempts to use machine learning in this field, both using an artificial neural network. In 2005, Green et al developed a model for predicting acute coronary syndrome based on demographics, prior history, symptom duration, and diastolic blood pressure.[39] This model had an AUC of 0.778. In 2007, a model based on serial measurements of myoglobin and contemporary troponin I concentrations, either alone or in combination with their rate of change was evaluated in 310 patients.[40] The output of this algorithm was dichotomised for diagnostic purposes, such that it had a sensitivity of 99%. These pioneering efforts were undertaken before the availability of high-sensitivity troponins and prior to the availability of multiple high-quality cohorts that we have been able to draw on.

Most studies that have developed or assessed strategies to risk stratify patients with suspected myocardial infarction have enrolled a limited number of patients with a small number of events resulting in limited precision and therefore limited external generalizability. In contrast, MI3 has been trained and tested in a population that includes patients from 9 studies across multiple geographic regions with more than 1250 events, and significant variation in the prevalence of disease, suggesting this approach is generalizable and could be used in any healthcare setting worldwide. Furthermore, rather than being an inflexible diagnostic strategy with multiple set thresholds, the MI3 algorithm is a dynamic tool that in the future could be retrained for individual health care settings depending on disease prevalence and diagnostic priorities to facilitate healthcare delivery.

From a clinical perspective, each patient will have an MI3 value that takes into consideration their own age and sex and measured cardiac troponin concentrations. This approach is distinct from previous algorithms. Whilst early diagnostic pathways, such as the ESC 0/1-hour pathway, identify groups of patients with high negative and moderate positive predictive value, they do not report these metrics or derive them for individual patients. This is a particularly important limitation for the one third of patients triaged to the observe zone, for whom the 0/1-hour pathway provides no guidance. The MI3 algorithm is also more versatile because it does not require serial testing to be performed at specific timepoints, and recognises that different healthcare systems have different priorities and tolerances of risk. The MI3 algorithm allows implementation to be tailored accordingly. For example, in some more conservative institutions triage to out-patient investigation might be acceptable only where the NPV is >99.8% (false negative rate of 1 in 1000; see Figure 4). Similarly, a cardiology consultation in the emergency department might be triggered if the PPV of myocardial infarction is >60%, but direct transfer to the cardiac cath lab only considered where the PPV is >80%. Prospective studies are now required to evaluate patient outcomes and resource utilisation following implementation of this algorithm into clinical practice.

Figure 4.

Myocardial-ischemic-injury-index (MI The figure shows a mockup of how the MI3 algorithm may be presented to physicians and patients. The top row illustrates a low-risk patient, the middle row illustrates an intermediate-risk patient, and the bottom row illustrates a high-risk patient, using the sample MI3 values of 0.9, 8.2, and 68.2, respectively. The screens on the left are for data input and return the MI3 value and estimated diagnostic metrics for an individual patient. A screen swipe presents the data in a natural frequency number and graphical format for the patient.

Limitations

Not all sites used the high-sensitivity cardiac troponin I assay to adjudicate the diagnosis of myocardial infarction, with several sites using either contemporary assays or high-sensitivity cardiac troponin T assays. This should reduce performance of the algorithm, yet we observed little heterogeneity across the cohorts. The choice of the high-risk example threshold was prespecified based on the PPV which is dependent on prevalence and may explain some of the heterogeneity between cohorts. Institutions could choose their own threshold based on the local prevalence of myocardial infarction or clinical priorities. Factors such as acceptability of risk, availability of inpatient beds or outpatient services, or the need to transfer patients for coronary angiography may influence the adoption of local thresholds. In contrast with the ESC 0/1- and 0/3-hour pathways, the primary purpose of MI3 is to assess risk in an individual patient. There are some limitations to our comparison with both pathways due to the requirement for precise sample intervals, which reduced the proportion of patients eligible for inclusion to 21% and 4% of our test cohort respectively. Prospective studies are needed to better understand the advantages and disadvantages of pathways based on individual probabilities rather than fixed thresholds. It is conventional in machine learning to use a larger data set to train and smaller data set to test out of practical considerations to develop a robust algorithmic model. We have used the smaller data set to train because it was available before the other data sets and because it was already a large data set. Machine learning also has the capability to use many features which could include aspects of patient history and clinical symptoms. These variables were not included because our intention was to develop a tool that only uses variables that are objective and always available to ensure our algorithm can be applied widely in clinical practice. We focus on the training and testing of the MI3 algorithm and do not report a comparison with other linear regression or machine learning methods here.

Conclusions

The MI3 clinical decision support tool incorporates simple and objective variables including age, sex and serial cardiac troponin I concentrations measured using a high-sensitivity assay to rapidly estimate risk of myocardial infarction. It can be used to individualise the risk assessment of patients with suspected myocardial infarction or to categorize patients into low- or high-risk groups.

Sources of Funding

M.P. Than is supported by a Clinical Practitioners Research Fellowship by the Health Research Council of New Zealand. Dr Pickering is supported by a Senior Research Fellowship in Acute Care from the Emergency Care Foundation, the Canterbury Medical Research Foundation, and the Canterbury District Health Board. Drs Shah and Mills are supported by an Intermediate Clinical Research Fellowship (FS/19/17/34172) and the Butler Senior Clinical Research Fellowship (FS/16/14/32023) from the British Heart Foundation, respectively. Dr Neumann was supported by a grant from the German Heart Foundation/German Foundation of Heart Research and the Else Kroöner Fresenius Stiftung. The BACC study was supported by German Center of Cardiovascular Research (DZHK) and an unrestricted grant by Abbott Diagnostics (no influence on the study). The study was also supported by a Research Excellence Award from the British Heart Foundation (RE/18/5/34216) and by Health Data Research UK which receives its funding from HDR UK Ltd (HDR-5012) funded by the UK Medical Research Council, Engineering and Physical Sciences Research Council, Economic and Social Research Council, Department of Health and Social Care (England), Chief Scientist Office of the Scottish Government Health and Social Care Directorates, Health and Social Care Research and Development Division (Welsh Government), Public Health Agency (Northern Ireland), British Heart Foundation and the Wellcome Trust. The funders had no role in the study and the decision to submit this work to be considered for publication.

Disclosures

Abbott Diagnostics applied the supervised machine learning technique “boosting” to train the MI3 algorithm (A.B., S.D., J.H.) and provided a description of the training process, which is supplied in the supplement. Abbott Diagnostics has filed for an international patent on the use of the MI3 algorithm (International Publication Number WO2017/173353A1). Abbott Diagnostics did not participate in data collection in any of the contributing studies or in the testing of algorithm performance. Abbott Diagnostics provided some funding to support two group meetings and data extraction. The Abbott Diagnostics employees listed (AB, SD, JH) contributed as coauthors to the drafting of the article. M.P. Than has consulted for Abbott Diagnostics, Beckman Coulter and Roche Diagnostics, and has received research funding from Abbott Diagnostics, Beckman Coulter and Roche Diagnostics. Dr Pickering has served as a consultant statistician for Abbott Diagnostics. Dr Sandoval has participated in advisory boards for Abbott Diagnostics and Roche Diagnostics without personal financial compensation. Dr Shah has received speaker fees from Abbott Diagnostics. Dr Cullen has consulted for Siemens Healthineers, Beckman Coulter and Abbott Diagnostics, and has received research funding from Beckman Coulter, Radiometer, Abbott Diagnostics and Siemens. Dr Neumann has received honoraria from Abbott Diagnostics and Siemens Healthineers. Dr Mills has served as a consultant for Abbott Diagnostics, LumiraDx, Siemens Healthineers, and Singulex. The other authors have nothing to disclose.