Comparison of the Framingham risk and SCORE models in predicting the presence and severity of coronary artery disease considering SYNTAX score.

OBJECTIVE: Although various risk stratification models are available and currently being used, the performance of these models in different populations is still controversial. We aimed to investigate the relation between the Framingham and SCORE models and the presence and severity of coronary artery disease, which is detected using the SYNTAX score.
METHODS: The observational cross-sectional study population consisted of 227 patients with a mean age of 63.3±9.2 years. The patients were classified into low- and high-risk groups in the Framingham and SCORE models separately. Following coronary angiography, the patients were classified into SYNTAX=0 (SYNTAX score 0), low SYNTAX (SYNTAX score 1-22), and high SYNTAX (SYNTAX score>22) groups. The relation between the risk models and SYNTAX score was evaluated by student t test, Mann-Whitney U test or Kruskal-Wallis test and Receiver operating characteristic analysis were used to detect the discrimination ability in the prediction of SYNTAX score>0 and a high SYNTAX score.
RESULTS: Both the Framingham and SCORE models were found to be effective in predicting the presence of coronary artery disease, and neither of the two models had superiority over each other [AUC=0.819 (0.767, 0.871) vs. 0.811 (0.757, 0.861), p=0.881]. Furthermore, both models were also effective in predicting the extent and severity of coronary artery disease [AUC=0.724 (0.656, 0.798) vs. 0.730 (0.662, 0.802), p=0.224]. When the subgroups were analyzed, the SCORE model was found to be better in predicting coronary artery disease extent and severity in subgroups of men and diabetics {[AUC=0.737 (0.668, 0.844) vs. 0.665 (0.560, 0.790), p=0.019], [AUC=0.733 (0.684, 0.798) vs. 0.680 (0.654, 0.750) p=0.029], respectively).
CONCLUSION: In addition to their role in predicting cardiovascular events, the use of the Framingham and SCORE models may also have utility in predicting the extent and severity of coronary artery disease. The SCORE risk model has a slightly better performance than the Framingham risk model.

Introduction

Coronary artery disease is the leading cause of morbidity and mortality worldwide. As well being the most common reason for disability and deaths, it also results in a significant portion of all health expenditures. Furthermore, if coronary artery disease results in myocardial infarction, the patients are candidates for a more challenging process such as heart failure. Therefore, the prediction of coronary artery disease is crucial to prevent the disease and to reduce healthcare costs. Although various risk stratification models are available and currently being used, a consensus on the reliability and suitability of them has not yet been reached.

The oldest and probably the most frequently used model is the Framingham system. The Framingham risk score uses information on age, gender, total and HDL cholesterol levels, systolic blood pressure, smoking status, diabetes mellitus (DM) existence, and being under treatment for hypertension (HT). The Framingham risk score estimates the 10-year risk of developing coronary heart disease (1). The SCORE model is another risk stratification model commonly used in the estimation of the cardiovascular risk in the European population. Gender, age, systolic blood pressure, total cholesterol level, and smoking status are the parameters used to estimate the total cardiovascular risk in the SCORE model. The SCORE risk model estimates the 10-year risk of fatal cardiovascular disease (2). However, there is uncertainty about the performance of these models to predict the absolute risk for the first coronary events when applied to different populations.

The SYNTAX score is an angiographic lesion-based scoring system that quantifies coronary anatomic complexity (3, 4). Furthermore, the role of the SYNTAX score in the short- and long-term risk stratification of patients undergoing percutaneous coronary interventions has also been shown by various clinical trials (5-9). The correlation between the SYNTAX score and these non-interventional clinical risk stratification models is unclear. We aimed to evaluate the roles of the Framingham and SCORE models in predicting the complexity of coronary artery disease considering the SYNTAX score and to find out the most valuable model for estimating total cardiovascular risk.

Methods

Study design

This is an observational cross-sectional study.

Study population

The study population consisted of 227 patients with a mean age of 63.3±9.2 who underwent coronary angiography for the presence of chest pain with objective signs of ischemia (treadmill exercise or myocardial single-photon emission computed tomography) between March and September 2014 at the cardiology department of Ordu University in Ordu, Turkey. Patients with severe valve disease, heart failure (ejection fraction<50%), acute coronary syndrome, hematologic disorders, chronic kidney disease, previous myocardial infarction or any revascularization procedures (whether percutaneous transluminal coronary angioplasty or coronary artery bypass grafting), congenital heart disease, digoxin therapy, and those who refused to undergo coronary angiography were excluded. Initially, the risk score of each patient was calculated using each risk stratification model before coronary angiography. The patients were then classified into low- and high-risk groups in the Framingham and SCORE models separately. Following coronary angiography, the patients were classified into SYNTAX score=0, low SYNTAX (SYNTAX score 1–22), and high SYNTAX (SYNTAX score>22) groups. The correlation between the non-interventional risk scoring systems and SYNTAX score was evaluated. All participants gave informed consent, and the study protocol was approved by the local Ethics Committee.

Assessment of risk scores

The two most frequently used risk stratification models were assessed in the study. We used the originally developed and validated predictors and coefficients for both the Framingham and SCORE models to calculate the predicted cardiovascular risk. The Framingham risk model is applicable to subjects between the ages of 30 and 79 years. Age, gender, total and HDL cholesterol levels, systolic blood pressure, smoking status, DM existence, and being under treatment for HT are used in calculating the Framingham score [http://cvdrisk.nhlbi.nih.gov/(National Heart, Lung, and Blood Institute, USA] (1). The SCORE model is applicable to subjects between the ages of 40 and 65 years. The parameters of the SCORE model are gender, age, systolic blood pressure, total cholesterol level, and smoking status. The original SCORE model disregards diabetes status; however, the addition of the risk of diabetes to the total cardiovascular risk is strictly recommended (http://www.escardio.org/communities/EACPR/toolbox/healthprofessionals/Pages/SCORE-Risk-Charts.aspx) (2). The reason for this is the presence of a higher cardiovascular risk in diabetics when compared with that in non-diabetics almost in parallel for every conventional cardiovascular risk factor (2). Considering the current estimated impact of diabetes on cardiovascular risk, the calculated risk in diabetics was multiplied by three in men and by five in women in our study (10). A high SCORE model was used in our study because it is more likely to perform better in Turkey, which is neighboring to the countries that are recommended to use the high-risk score chart. Individuals who had a value of ≥20% in the Framingham model and ≥5% in the SCORE model were accepted as high-risk patients.

Coronary angiography

The 227 patients recruited in the study underwent coronary angiography [Siemens AXIOM-Artis (Siemens AG 2001 Muenchen-Germany)] at the cardiology department of Ordu University. Coronary angiographies were performed in our clinic using the standard Judkins method with iohexol (Omnipaque, Nycomed Ireland Ltd, Cork, Ireland). During each injection, 6–10 mL contrast agent was manually delivered. Coronary angiograms were independently assessed by two invasive cardiologists who were blinded to the clinical findings. In the case of disagreement, an opinion was obtained from the third cardiologist, and the final decision was made by consensus.

SYNTAX score

The SYNTAX score is an angiographic tool used in grading the complexity of coronary artery disease. Each coronary lesion ≥50% in diameter in each vessel ≥1.5 mm is scored. The latest updated version online was used in the calculation of the SYNTAX scores (www.syntaxscore.com) (11). After receiving basic training from the SYNTAX score website, the cardiologists calculated the SYNTAX score. Both numeric values and tertiles (0, 1–22, >22) of the score were used. SYNTAX score=0 represents the absence of coronary artery disease or the presence of a <50% lesion in any coronary artery. SYNTAX>0 represents the presence of ≥50% lesion in any coronary artery disease.

Definitions

Chest pain was defined as discomfort in the chest, jaw, shoulder, back, or arms typically elicited by exertion or emotional stress and relieved by rest or nitroglycerin. HTN was defined as systolic blood pressure >140 mm Hg and/or a diastolic blood pressure >90 mm Hg, or use of antihypertensive medications. DM was defined as fasting glucose 126 mg/dL or use of insulin or oral hypoglycemic medications (1). A positive family history for coronary artery disease was defined as evidence of coronary artery disease, such as a history of myocardial infarction, coronary artery bypass surgery, angina pectoris, or pathological exercise tolerance test diagnostic of ischemia, in a parent or sibling before the age 60 years (1). BMI was calculated by dividing the weight (kg) of an individual by the square of his or her height. A patient with a BMI value >25 kg/m2 was defined as overweight. Current smokers were defined as having a history of smoking for a certain period within the past year.

Statistical analysis

Baseline characteristics of the patients were compared between the SYNTAX score=0, low-, and high-SYNTAX groups. We then calculated what percentage of the patients with a high risk in a non-interventional risk stratification model also had a high risk in the SYNTAX score to assess the correlation between risk scoring systems and the extent of coronary artery disease. Continuous variables were expressed as mean±SD or median (interquartile range). Categorical variables were expressed as percentages. An analysis of the normality of the continuous variables was performed with the Kolmogorov–Smirnov test. The comparison of parametric values between the three groups was performed by means of an independent samples t-test. The comparisons of nonparametric values between the three groups were performed by Mann–Whitney U test or Kruskal–Wallis test. The chi-square test was used to compare the categorical variables. Receiver operating characteristic (ROC) analyses were used to detect the discrimination ability in the prediction of SYNTAX score>0 and a high SYNTAX score. Two-tailed p-values <0.05 were considered as statistically significant. All statistical studies were conducted with the SPSS program version 20.0 for Windows (IBM Corp, Armonk, NY). Area under the curves (AUC) was compared using MedCalc for Windows, version 12.5 (MedCalc Software, Ostend, Belgium).

Results

Evaluation of risk factors

The comparison of clinical and demographical characteristics of the SYNTAX score=0, low-, and high-SYNTAX groups is given in Table 1. There was not a statistically significant difference among the three groups in terms of age, male gender, triglyceride and blood glucose levels, BMI, and smoking status (p>0.05). On the other hand, there was a statistically significant difference in terms of low density lipoprotein (LDL) cholesterol, total cholesterol, and HDL cholesterol levels among the three groups (p=0.034, p=0.021, and p=0.030, respectively). The patients in the high SYNTAX group also had higher rates of hypertension and family history (p<0.001, p=0.008, respectively). Similar findings were found when the SYNTAX score=0 group was compared with the low- and high-SYNTAX groups. Additionally, the prevalence of DM was significantly lower in the SYNTAX score=0 group when compared with that in the low- and high-SYNTAX groups (p<0.032).

Table 1

Comparison of baseline characteristics between SYNTAX tertiles (SYNTAX=0, Low SYNTAX (1–22), High SYNTAX (22<)

Variables	SYNTAX Score=0 n=74	Low SYNTAX (1–22) n=83	High SYNTAX (>22) n=70	P *
Age, years; %	52.2 ±9.6	53.4±8.9	55.1±9.1	0.264
Male gender, n (%)	45 (62)	53 (65)	45 (64)	0.367
Body mass index, kg/m2	25.8±3.4	26.8±3.7	27.3±4.4	0.305
Smoking, n (%)	14 (19)	20 (23)	17 (24)	0.517
Diabetes mellitus, n (%)	12 (16)	45 (22)	16 (23)	0.032[a]
Family history, n (%)	13 (18)	31 (37)	29 (42)	0.008[b]
Hypertension, n (%)	19 (25)	35 (42)	39 (55)	<0.001[c]
LDL, mg/dL	132.4±31.1	139.7±33.0	147.3±31.5	0.034[d]
HDL, mg/dL	48.9±9.7	44.5±10.2	41.5±10.7	0.021[e]
Triglyceride, mg/dL	185.9±64.1	183.2±63.4	184.9±75.7	0.778
Total cholesterol, mg/dL	201.6±29.2	206.4±40.1	216.2±31.4	0.030[f]
Blood glucose	96.3±21.1	106.7±30.8	115.7±47.9	0.157
Previous medication, %
Aspirin, n (%)	8 (11)	14 (17)	13 (19)	0.216
Statin, n (%)	6 (8)	17 (20)	16 (23)	0.097[g]
ACEI/ARB, n (%)	10 (14)	18 (22)	23 (33)	0.031[h]
CCB, n (%)	4 (5)	6 (7)	6 (9)	0.498
B-blocker, n (%)	4 (5)	7 (9)	7 (10)	0.612
Framingham risk score
>20%, n (%)	10 (13)	25 (30)	35 (50)	<0.001[¥]
10–20%, n (%)	31 (42)	40 (48)	20 (29)	<0.001[¥]
<10%, n (%)	33 (45)	18 (22)	15 (21)	<0.001[¥]
SCORE
≥5%, n (%)	18 (24)	40 (48)	49 (70)	<0.001[¥]
<5%, n (%)	56 (76)	43 (52)	21 (30)	<0.001[¥]

Data are presented as number (percentage) and mean±standard deviation.

ANOVA, Kruskal–Wallis, Pearson chi-square tests SS=0 (SYNTAX score=0), LS-low SYNTAX (1–22), HS-high SYNTAX (>22)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P=0.134, P<0.001, and P=0.064, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P=0.893, P=0.015, and P=0.051, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P=0.051, P<0.001, and P=0.001, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P<0.001, P<0.001, and P<0.001, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P<0.001, P<0.001, and P<0.001, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P<0.001, P<0.001, and P<0.001, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P=0.583, P=0.004, and P=0.013, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P=0.014, P<0.001, and P=0.057, respectively)

(SS=0 vs. LS, SS=0 vs. HS, and LS vs. HS P<0.001, P<0.001, and P<0.001, respectively) HDL - high-density lipoprotein; LDL - low-density lipoprotein; ACEI/ARB - Angiotensin converting enzyme inhibitors/angiotensin receptor blockers; CCB - calcium channel blockers

Assessment of risk stratification models

When the overall population was evaluated, both the Framingham >20% and SCORE ≥5% models were found to be effective in predicting the presence of coronary artery disease, and neither of the two models had superiority over each other [AUC=0.819 (0.767, 0.871) vs. 0.811 (0757, 0.861), p=0.881; Fig. 1, Table 2]. Furthermore, both models were also effective in predicting the extent and severity of coronary artery disease as both had a favorable discriminative ability for patients with a high SYNTAX score. Again, there was not a significant difference between the discriminative abilities [AUC=0.724 (0.656, 0.798) vs. 0.730 (0.662, 0.802), p=0.224; Fig. 2, Table 3]. When the subgroups were analyzed, the SCORE model showed a more favorable discriminative ability in the subgroups of men and diabetics {[AUC=0.737 (0.668, 0.844) vs. 0.665 (0.560, 0.790), p=0.019, [AUC=0.733 (0.684, 0.798) vs. 0.680 (0.654, 0.750) p=0.029], respectively; Table 4). Furthermore, the discriminative ability of the Framingham model could not reach statistical significance in these subgroups (AUC<0.7). On the other hand, the Framingham model was better in predicting the patients with a high SYNTAX score in a hypertensive population [AUC=0.745 (0.662, 0.822) vs. 0.713 (0.623, 0.844), p=0.026], in an overweight population [0.713 (0.637, 0.798) vs. 0.657 (0.576, 0.734), p=0.041], and in patients with a family history [0.732 (0.614, 0.820) vs. 0.673 (0.566, 0.766), p=0.038]. The discriminative ability of the SCORE model was insufficient in an overweight population and in patients with a family history (AUC<0.7). Interestingly, both models were inadequate for women in terms of predicting high SYNTAX patients [0.570 (0.334, 0.806) vs. 0.591 (0.370, 0.828), p=0.307]. Finally, in smokers, the Framingham and SCORE models showed a good discriminative ability for high SYNTAX patients, with a slightly better performance in the Framingham model [0.717 (0.618, 0.836) vs. 0.719 (0.670, 0.860), p=0.903].

Figure 1

Receiver operating characteristic (ROC) curves for the Framingham and SCORE models in the prediction of SYNTAX Score>0

Table 2

Sensitivity, specificity, and discriminative ability of the Framingham and SCORE risk models for SYNTAX=0 and SYNTAX>0 in the overall population

	Cut-off	Sensitivity	Specificity	AUC (95% Cl)
Framingham Risk Score	>20%	70.9	79.9	0.819 (0.767, 0.871)
SCORE	>5%	72.3	74.4	0.811 (0.757, 0.861) P=0.881

Figure 2

Receiver operating characteristic (ROC) curves for the Framingham and SCORE models in the prediction of high SYNTAX (SYNTAX Score>22)

Table 3

Sensitivity, specificity, and discriminative ability of the Framingham and SCORE risk models for SYNTAX≤22 and SYNTAX>22 in the overall population

	Cut-off	Sensitivity	Specificity	AUC (95% Cl)
Framingham Risk Score	>20%	57.7	82.6	0.724 (0.656, 0.798)
SCORE	>5%	70.8	73.3	0.730 (0.662, 0.802) P=0.224

Table 4

Sensitivity, specificity, and discriminative ability of the Framingham and SCORE risk models for SYNTAX<22 and SYNTAX>22 in the prediction of cardiovascular mortality in subgroups considering gender, family history, weight, smoking status, and diabetes and hypertension existence

	Cut-off	Sensitivity	Specificity	AUC (95% Cl)
Men
Framingham Risk Score	>20%	62.7	67.7	0.665 (0.560, 0.790)
SCORE	>5%	70.2	51.4	0.737 (0.668, 0.844) P=0.019
Women
Framingham Risk Score	>20%	42.8	71.9	0.570 (0.334, 0.806)
SCORE	>5%	53.1	61.5	0.591 (0.370, 0.828) P=0.307
Family history (+)
Framingham Risk Score	>20%	71.4	72.3	0.732 (0.614, 0.820)
SCORE	>5%	62.5	52.8	0.673 (0.566, 0.766) P=0.038
Overweight
Framingham Risk Score	>20%	63.2	80.4	0.713 (0.637, 0.798)
SCORE	>5%	72.4	57.6	0.657 (0.576, 0.734) P=0.041
Smokers
Framingham Risk Score	>20%	70.1	67.7	0.717 (0.618, 0.836)
SCORE	>5%	72.1	66.5	0.719 (0.670, 0.860) P=0.903
Diabetics
Framingham Risk Score	>20%	49.5	83.5	0.680 (0.654, 0.750)
SCORE	>5%	74.1	63.2	0.733 (0.684, 0.798) P=0.029
Hypertensive patients
Framingham Risk Score	>20%	75.6	79.3	0.745 (0.662, 0.822)
SCORE	>5%	69.1	72.6	0.713 (0.623, 0.844) P=0.026

Discussion

Although fewer parameters our study results demonstrated that both the Framingham and SCORE risk stratification models were powerful in predicting both the presence and severity of coronary artery disease, with a slightly better performance in the SCORE model, in our population. As expected, the power of the models was higher when discrimination was performed between the SYNTAX score=0 group and SYNTAX score>0 (SYNTAX score=0–22 and SYNTAX score>22) patients. While still sufficient, a slight reduction was observed in the discriminative ability of the models when performed in the SYNTAX score=0–22 and SYNTAX score>22 groups.

We also compared the baseline characteristics of the patients that were thought to be conventional risk factors for coronary artery disease between the SYNTAX score=0, low-, and high-SYNTAX groups. Family history, hypertension, and LDL, HDL, and total cholesterol levels were the parameters that differed between the SYNTAX score=0, low-, and high-SYNTAX groups. This may be a sign on the necessity of keeping these parameters more prominent in risk stratification models. The Framingham model includes three of these five parameters (hypertension and total and HDL cholesterol levels), whereas the SCORE model uses information on only two (hypertension and total cholesterol level). None of the models include information on family history and LDL cholesterol levels.

Although fewer parameters are evaluated in the SCORE model than in the Framingham model, the former was as effective as the latter in assessing the extent and severity of coronary artery disease. Questioning of being under treatment and HDL levels generally had no impact in model performance except for hypertensive and overweight populations and for the patients with a family history as the Framingham model was found to be better in these subgroups.

Beside similar AUC values, the two models also had similar sensitivity and specificity for the discrimination of the SYNTAX=0 and SYNTAX>0 groups. Nevertheless, although similar AUC values were detected for the discrimination of SYNTAX≤22 and SYNTAX>22 in the Framingham and SCORE models, the sensitivity of Framingham model was notably lower when compared with that of the SCORE model (57.7% vs. 70.8%). On the other hand, a high specificity of the Framingham model in this population should be kept in mind. As the burden of cardiovascular morbidity and mortality is disproportionately higher in developing countries such as ours, the misclassification of a high cardiovascular risk at the crucial period is unacceptable. Categorization of most people into the low cardiovascular risk group incorrectly will result in the under-identification of high-risk individuals leading to higher rates of under treatment and subsequently more complications. This will also cause utilization of resources on screening rather than treatment. In our study, both models had enough discriminative ability to not cause underestimation.

Accurate risk stratification is essential for the effective management of cardiovascular risk. Hence, the applicability of these prediction models has been evaluated in several populations, and the comparison of risk stratification models has also been the subject of research. As it is a model developed to predict cardiovascular events in the United States, the Framingham model generally indicates an overestimation of coronary risk in European countries (12-18). On the other hand, the applicability of the SCORE model has also been evaluated in some European countries. Although developed for Europe, it has also been observed that the SCORE model may lead to overestimation or underestimation in some European countries (19-21). Thus, the efficacy of the risk stratification models needs further evaluation considering ethnicity and genetic variance of different populations. Scheltens et al. (22) compared the efficacy of the two models in people under the age of 60 years in Netherlands and demonstrated that both the SCORE and Framingham models have a good discriminative ability (22). Similar findings were observed in a study comparing the risk score models in a Malaysian population (23). Nevertheless, Marchant et al. (24) suggested that the SCORE model should be preferred to the Framingham model to predict the risk cardiovascular death in a French population. Erkan et al. (25) evaluated the association between peripheral artery disease and coronary artery disease considering two interventional score models such as the SYNTAX score and Trans-Atlantic Inter-Society Consensus II classification. In the current study, we evaluated the reliability and applicability of the two risk stratification models in deference to the SYNTAX score. The significance of the present study is that it confirmed the applicability of the two cardiovascular risk models in a Turkish population. Furthermore, our results also reveal that these two models may provide an estimation of the patients who will require surgery because both are favorable in assessment of patients with a high SYNTAX score.

Study limitations

HDL cholesterol levels were not taken into account in the calculation of the SCORE model. This can be considered as a limitation of our study. However, herein, we aimed to compare the original versions of both the Framingham and SCORE models.

Nevertheless, the present study cannot conclude which risk function is the most accurate for the Turkish population. This will need to be evaluated by a longitudinal study on the development of cardiovascular events in a cohort of healthy subjects.

Conclusion

Both the Framingham and SCORE models were found to be applicable and reliable in the estimation of both the presence and severity of coronary artery disease in addition to their role in predicting cardiovascular events. The use of cardiovascular risk scores in clinical practice may also have utility for the prediction of the extent and severity of coronary artery disease.