Performance of a clinical/proteomic panel to predict obstructive peripheral artery disease in patients with and without diabetes mellitus.

Background: Patients with diabetes mellitus (DM) are at substantial risk of developing peripheral artery disease (PAD). We recently developed a clinical/proteomic panel to predict obstructive PAD. In this study, we compare the accuracy of this panel for the diagnosis of PAD in patients with and without DM. Methods and results: The HART PAD panel consists of one clinical variable (history of hypertension) and concentrations of six biomarkers (midkine, kidney injury molecule-1, interleukin-23, follicle-stimulating hormone, angiopoietin-1 and eotaxin-1). In a prospective cohort of 354 patients undergoing peripheral and/or coronary angiography, performance of this diagnostic panel to detect ≥50% stenosis in at least one peripheral vessel was assessed in patients with (n=94) and without DM (n=260). The model had an area under the receiver operating characteristic curve (AUC) of 0.85 for obstructive PAD. At optimal cut-off, the model had 84% sensitivity, 75% specificity, positive predictive value (PPV) of 84% and negative predictive value (NPV) of 75% for detection of PAD among patients with DM, similar as in those without DM. In those with DM, partitioning the model into five levels resulted in a PPV of 95% and NPV of 100% in the highest and lowest levels, respectively. Abnormal scores were associated with a shorter time to revascularisation during 4.3 years of follow-up.
Conclusion: A clinical/biomarker model can predict with high accuracy the presence of PAD among patients with DM. Trial registration number: NCT00842868.

The ankle-brachial index (ABI) is most commonly used to diagnose lower extremity peripheral artery disease (PAD); however, its diagnostic accuracy is limited in patients with stiff, calcified arteries which is common among patients with diabetes mellitus (DM).

We recently developed a clinical/proteomic panel (HART PAD) using machine learning, capable of diagnosing obstructive PAD with high accuracy; however, the utility of this score in patients with DM is uncertain.

The HART PAD panel predicted with high accuracy the presence of PAD among patients with DM.

Furthermore, the HART PAD panel was predictive of revascularisation among patients with DM.

The HART PAD panel offers an attractive alternative to ABI for diagnosing PAD among patients with DM.

The panel could act as a gatekeeper to imaging or invasive testing, thereby reducing costs, and exposures to intravenous contrast and/or ionising radiation by avoiding expensive imaging modalities when unwarranted.

Furthermore, the panel could be used for prognostic purposes to guide more intensification of medical therapies.

Introduction

Diabetes mellitus (DM) is a global health problem; it is estimated, by 2030, approximately 366 million people worldwide will suffer from the disease.1 Patients with DM are at substantial risk for developing both microvascular and macrovascular complications.2 One notable macrovascular complication of DM is peripheral artery disease (PAD) which is prevalent in approximately 20%–30% of patients.3 4 PAD is associated with a considerable increase in the risk of fatal and non-fatal cardiovascular and cerebrovascular events,5 and event rates are higher among patients with DM.6

Symptoms of PAD are variable, especially in patients with DM who may suffer from concomitant peripheral neuropathy, thus it is often undiagnosed until its advanced stages. As a result, patients with DM and PAD often receive suboptimal management that may prevent progression of disease.4 The ankle-brachial index (ABI) is the most common non-invasive diagnostic modality used to detect the presence of lower extremity PAD; however, its accuracy is reduced in patients with stiff, calcified arteries. Approximately 60% of patients with DM have calcified lower extremity peripheral arteries, and expectantly, ABI has correlated poorly with angiographic PAD in this population.7 Imaging modalities are also used to diagnose PAD but imaging is expensive, has variable availability and requires intravenous contrast and/or ionising radiation. For these reasons, we recently developed a clinical/proteomic panel (HART PAD) using machine learning, capable of diagnosing obstructive PAD with high accuracy.8 In this study, we compare the accuracy of this panel for the diagnosis of obstructive PAD in patients with and without DM a population at high risk for PAD that is particularly challenging to evaluate and manage.

Methods

Study population

The Catheter Sampled Blood Archive in Cardiovascular Diseases study was a prospective, single-centre, observational cohort study that was undertaken at the Massachusetts General Hospital in Boston, Massachusetts, between 2008 and 2011. The investigators enrolled 1251 subjects undergoing coronary and peripheral angiography with or without intervention over the study period.9

For the purpose of this study, we included 354 patients who underwent peripheral angiography only (n=140), peripheral and coronary angiography but without significant coronary artery disease (CAD) (n=11) and those who underwent coronary angiography alone without significant CAD and no history of PAD (n=203). The latter group were incorporated to increase cohort size and were assumed to have an absence of PAD, based on their medical history. The indications for peripheral angiography included claudication (n=96), carotid artery stenosis with/without stroke (n=11), hypertension (n=21) and other PAD without claudication (n=25). The peripheral angiograms included: lower extremity (n=129), renal arteries (n=59) and carotid/subclavian (n=18). All study procedures were approved by the Partners HealthCare Institutional Review Board and carried out in accordance with the Declaration of Helsinki.

Data acquisition

After obtaining informed consent and at the time of the procedure, clinical and historical variables and reason for referral for angiography were recorded. Results of the peripheral +/−coronary angiography (based on visual estimation at the time of the procedure) were recorded. For the purposes of this analysis, ≥50% luminal obstruction in at least one peripheral vessel was considered obstructive PAD.

HART PAD model

The derivation of the HART PAD panel has been previously described.8 Briefly, using the same 354 patients included in this study, we used machine learning, a subset of artificial intelligence, to identify predictors of significant PAD. We examined 109 biomarkers and more than 50 clinical variables. Candidate panels of proteins and clinical features were generated via least angle regression. In this method, factors were included in the model one at a time, with their coefficients determined by their correlation with the outcome. This was repeated until all factors were included in the model, and the step at which the performance plateaued resulted in our initial panel of interest. With this panel of interest, predictive analyses were run on the training set using least absolute shrinkage and selection operator with logistic regression, predicting the outcome of obstructive PAD using only the variables in the panel of interest. If the contribution of the least performing variable was not statistically significant, then that variable was removed and the analysis rerun. This process was repeated until the predictive contribution of all variables in the model was statistically significant. This model development process was done via Monte Carlo cross-validation, using 400 iterations with an 80:20 (training:test) split. The final panel was used to create a final model with the entire sample, and this model was then evaluated to predict obstructive PAD.

After the machine learning model building process, the final panel consisted of one clinical variable (systemic hypertension) and six biomarkers: follicle-stimulating hormone, angiopoietin-1, kidney injury molecule-1 (KIM-1), midkine, interleukin-23 and eotaxin-1.

Biomarker testing

Using a centrally placed vascular access sheath, 15 mL of blood was taken immediately prior to angiography. Blood samples were then stored in a 4°C refrigerator until centrifuging was undertaken. After a single freeze-thaw cycle, 200 µL of plasma was used for biomarker analysis on a Luminex 100/200 xMAP technology platform. The biomarkers were obtained from a commercially available kit, Myriad RBM MAP; this technology uses multiplexed, microsphere-based assays in a single reaction vessel whereby an assay-specific capture antibody on each microsphere binds to a protein of interest. Comparable to a flow cytometer, as each individual microsphere passes through a series of excitation beams, it is analysed for size, encoded fluorescence signature and the amount of fluorescence generated is proportionate to the protein concentration.

Follow-up

A review of patients’ medical records from time of enrolment to the end of follow-up was undertaken. For identification of clinical endpoints, in addition to a review of medical records, telephone follow-up was performed with patients and/or their managing physicians. Study investigators adjudicating angiographic severity of PAD or events during follow-up were blinded to results of all biomarker testing.

Statistical analysis

Baseline characteristics between those with and without DM were compared; dichotomous variables were compared using two-sided Fisher’s exact test, while continuous variables were compared using two-sided two-sample t-test. The biomarkers compared were tested with the Wilcoxon rank-sum test, as their concentrations were not normally distributed. A complete case analysis was performed; two patients were missing at least one of the concentration read-outs for the six proteins in the final panel, so they were excluded, leaving 352 samples available for analysis. For any biomarker result that was unmeasurable, we used a standard approach of imputing concentrations 50% below the limit of detection.

The HART PAD panel was originally developed to predict obstructive PAD using Monte Carlo cross-validation, and the final model was evaluated using in-sample validation. In this study, we used the final model trained to diagnose PAD in all patients, and evaluated it on all patients with DM. We generated a receiver operating characteristic curve and determined an optimal diagnostic cut-off using the optimal Youden’s index. With this cut-off we calculated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV). The predictive score generated by the diagnostic model was rescaled to a range of 0–10 to facilitate interpretation. The score was then partitioned into five different risk levels, corresponding to multiple levels of PAD risk. Time to revascularisation as a function of elevated PAD score was calculated from 7 days after index angiography over a mean of 4.3 years’ follow-up period and displayed as a Kaplan-Meier survival curve. Lastly, after excluding patients with non-lower extremity PAD (eg, carotid and renal artery disease), we then assessed time to revascularisation or amputation in patients with lower extremity PAD as a function of the continuous HART PAD score (adjusting for age) over a mean of 4.3 years’ follow-up period and displayed this as a Kaplan-Meier survival curve.

All statistics were performed by using R software, V.3.4 (R Foundation for Statistical Computing, Vienna, Austria). P values are two sided, with a value <0.05 considered significant.

Results

Baseline characteristics

Of the 354 patients included in this study, 132 had obstructive PAD (online supplementary table 1). Baseline characteristics of study subjects, dichotomised as a function of DM, are detailed in table 1. Patients with DM were older, more likely to be male and had a higher prevalence of hypertension, CAD, prior myocardial infarction and chronic kidney disease. Notably, of the biomarkers measured, those with DM had higher concentrations of KIM-1 and midkine.

Table 1

Baseline characteristics of patients with and without diabetes mellitus

Characteristics	Subjects with DM(n=94)	Subjects without DM(n=260)	P value
Demographics
Age (years)	68±11.3	63±8.4	0.002
Male sex	74.5%	56.9%	0.003
Caucasian	92.6%	92.3%	1.00
Medical history
Smoker	15.2%	15.5%	1.00
Atrial fibrillation/flutter	25.5%	21.2%	0.39
Hypertension	91.5	69.6%	<0.001
Coronary artery disease	58.5%	28.46%	<0.001
Prior MI	14.9%	13.5%	<0.001
Heart failure	30.9%	19.6%	0.24
COPD	21.2%	20.2%	0.89
CVA/TIA	22.3%	8.5%	<0.001
CKD	27.7%	4.2%	<0.001
Prior CABG	29.8%	9.2%	<0.001
Prior percutaneous coronary intervention	40.4%	22.3%	0.001
Medications
ACE-I/ARB	70.2%	47.3%	<0.001
Beta blocker	74.5%	56.8%	0.003
Aldosterone antagonist	4.3%	3.5%	0.75
Loop diuretics	31.9%	16.6%	0.003
Nitrates	17.0%	10.8%	0.14
Statin	81.9%	61.4%	<0.001
Aspirin	78.7%	68.0%	0.06
Warfarin	21.3%	18.6%	0.65
Clopidogrel	27.7%	14.7%	0.008
Biomarkers
Angiopoietin-1 (ng/mL), median (25th–75th percentiles)	6.4 (4.3, 8.58)	6.8 (5, 11)	0.05
Eotaxin-1 (pg/mL), median (25th–75th percentiles)	108 (42.5, 150.5)	97 (42.5, 144)	0.10
Follicle-stimulating hormone (mIU/mL), median (25th–75th percentiles)	8.7 (4.55, 29.25)	9.7 (4.15, 41.5)	0.38
Interleukin-23 (ng/mL), median (25th–75th percentiles)	2.8 (2.23, 3.38)	2.5 (2, 3.2)	0.06
Kidney injury molecule-1 (ng/mL), median (25th–75th percentiles)	0.07 (0.04, 0.12)	0.01 (0.01, 0.05)	<0.001
Midkine (ng/mL), median (25th–75th percentiles)	18 (13.25, 26)	13 (9.9, 19)	<0.001

All continuous variables are displayed as mean±SD, unless otherwise specified.

ACE-I/ARB, ACE inhibitor/angiotensin receptor blocker; CABG, coronary artery bypass graft;CKD, chronic kidney disease;COPD, chronic obstructive pulmonary disease;CVA/TIA, cerebrovascular accident/transient ischaemic attack;DM, diabetes mellitus; MI, myocardial infarction.

HART PAD diagnostic performance

With the data represented in the 0–10 scale, the optimal cut-off for the panel was determined to be 5.607 using the optimal Youden’s index, which identifies the optimal balance of sensitivity and specificity. In receiver operating characteristic testing, the model generated an in-sample AUC of 0.848 for patients with DM (figure 1); slightly higher than the performance for patients without DM (AUC 0.83).

Figure 1

Receiver operating characteristic curve for the HART PAD score to predict obstructive peripheral arterial disease in patients with diabetes mellitus. The score had a very robust area under the receiver operating characteristic curve (AUC).

At its optimal cut-off to diagnose PAD, for patients with DM, we observed a sensitivity of 84%, specificity of 75%, NPV of 75% and PPV of 84%.

HART PAD five-level score

Partitioning the score into five categories yielded a PPV of 95% and NPV of 100% in the highest and lowest scores, respectively, for patients with DM (table 2). We found a higher prevalence of obstructive PAD in those with higher scores and lower prevalence among those with lower scores (figure 2). When the score was divided into five categories of predicted risk, an increasing score correlated with an increasing degree of mean PAD stenosis in both patients with and without DM (figure 3).

Table 2

Predictive performance as a five-level score

Score	Positive predictive value	Negative predictive value
5	0.95	–
4	0.66	–
3	0.31	0.69
2	–	0.86
1	–	1.00

Figure 2

Distribution of score among patients with diabetes (positive) and without diabetes (negative) in a histogram.

Figure 3

Correlation between peripheral artery disease (PAD) score and mean degree of arterial stenosis in patients with and without diabetes mellitus (DM).

Performance in various vascular territories

The score had a similar in-sample performance in each individual vascular territory using the optimal cut-off among patients with DM. In the diagnosis of obstructive lower extremity artery disease, the score had a sensitivity of 83%, specificity of 62%, PPV of 70% and NPV of 78%. For diagnosing obstructive carotid and renal artery stenosis, the score appeared most useful for its NPV. The score had a sensitivity of 80%, specificity of 58%, PPV of 10% and NPV of 98% for obstructive carotid arterial disease. While for the renal arteries, the corresponding sensitivity was 88%, specificity 59%, PPV 17% and NPV 98%.

Predicting the need for revascularisation

In adjusted Cox proportional hazards models, from 7 days after index angiography to the end of follow-up, those patients with DM who had a dichotomously elevated score had higher risk for revascularisation, compared with patients with DM who had a lower PAD score (HR: 2.88; 95% CI 1.16 to 7.19, p=0.02); those with higher scores also had shorter time to first revascularisation event (figure 4). After excluding patients with obstructive PAD in non-lower extremity territories (figure 5), the score predicted time to revascularisation or amputation as a continuous score among patients with DM and lower extremity PAD (HR: 1.3; 95% CI 1.02 to 1.66, p=0.04).

Figure 4

Kaplan-Meier survival curves depicting time to revascularisation as a function of peripheral artery disease (PAD) score. Patients in the positive group had a score greater than or equal to the optimal cut-off for the score, which was determined to be 5.607 using the optimal Youden's index (with the model’s output rescaled to the range of 0–10). Patients in the negative group had a score below 5.607. DM, diabetes mellitus.

Figure 5

Kaplan-Meier survival curves depicting time to revascularisation as a function of continuous HART PAD score in patients with lower extremity peripheral artery disease (PAD). Patients in the positive group had a score greater than or equal to the optimal cut-off for the score, which was determined to be 5.607 using the optimal Youden’s index (with the model’s output rescaled to the range of 0–10). Patients in the negative group had a score below 5.607. DM, diabetes mellitus.

Discussion

Though prevalent in patients with DM, PAD is a challenge to identify and manage; tools typically used for diagnosis of PAD are often less accurate in those with DM. Accordingly, we wished to verify the performance of a biomarker-leveraged scoring system derived using machine learning recently found to predict angiographically significant PAD.8 We demonstrate excellent performance of the model in patients with DM. When analysed as a five-level score, the highest and lowest scores yielded a PPV of 95% and NPV of 100%, respectively, for diagnosing obstructive PAD among patients with DM.

As discussed, the biomarkers used in this model all have credible biological links to atherosclerotic PAD.10–19 We believe the HART PAD panel could be useful in those with DM. As 60% of patients with DM have calcified lower extremity peripheral arteries, ABI, the most commonly used non-invasive diagnostic tool, has correlated poorly with angiographic PAD in this population.7 The HART PAD panel offers an attractive alternative to ABI with high NPV and PPV in patients with DM. Given a range of score values that provides both strong PPV and NPV, the panel could potentially avoid the need for imaging or invasive testing, thereby reducing costs, and exposures to intravenous contrast and/or ionising radiation. Of note, the model performed similarly well in other vascular regions (renal and carotid arteries), in particular for its NPV. Undoubtedly, the panel should serve as an adjunct to a thorough history and physical examination.

Furthermore, we demonstrate that the HART PAD panel has prognostic utility, with a shorter time to revascularisation in those with an elevated score. Consequently, the scoring system may also be used to predict patients at risk for vascular complications, which could then be used to guide therapeutic intervention. Aspirin and statins are already standard therapies in PAD, but other medications such as novel oral anticoagulants (NOAC) and proprotein convertase subtilisin/kexin type 9 (PCSK9) inhibitors have recently demonstrated promise in this population.20 21 Cost (PCSK9 inhibitors) and side effect profile (NOACs) may limit their utility and, thus, appropriate risk stratification will be important to guide therapy.

Lastly, the HART PAD panel may play a role in clinical trials to enrich for PAD-related events or to identify patients at risk for adverse effects of drug therapies. For example, although the sodium-glucose co-transporter-2 inhibitor canagliflozin significantly reduced the risk of cardiovascular events by 14%, it doubled the risk of amputation in patients with type 2 DM. Our panel could be useful in predicting underlying PAD or the need for revascularisation in patients who are being considered for these agents.22

Despite being novel, our study has limitations. Notably, the number of patients from which we derived our findings was relatively small and included patients who only underwent coronary angiography as negative controls; though such patients were low risk for PAD, unexpected disease might have been present. As such, the results of our study should serve as preliminary evidence that requires confirmation in larger and ad hoc cohorts. The study participants were predominantly Caucasian which limits to the external validity of our panel to African-American patients. Patients with critical limb ischaemia were not included in our study and the utility of the panel in this cohort is uncertain. The diagnostic model was not compared with other non-invasive modalities such as ABI or ultrasonography; a comparison which will necessitate investigation in future studies. Our results need further validation and should not be extrapolated to the general population; the patients in our study were referred with clinical suspicion for significant PAD. Lastly, using the clinical/biomarker panel alone will not be sufficient to differentiate the exact territory of PAD and thus clinical correlation with history and physical examination will be an important component of assessment.

Conclusion

We describe a clinical/biomarker panel with high accuracy for predicting the presence of PAD in patients with DM.