Comparison of Framingham, PROCAM, SCORE, and Diamond Forrester to predict coronary atherosclerosis and cardiovascular events.

BACKGROUND: Cardiologists are often confronted with patients presenting with chest pain, in whom clinical risk profiling is required. We studied four frequently used risk scores in their ability to predict for coronary artery disease (CAD) and major adverse cardiovascular events in patients presenting with stable chest pain at the cardiology outpatient clinic. METHODS AND
RESULTS: We enrolled 1,296 stable chest pain patients, who underwent cardiac computed tomographic angiography (CCTA) to assess CAD (any, significant: stenosis ≥50%). Framingham (FRS), PROCAM, SCORE risk score, and Diamond Forrester pre-test probability were calculated. All patients were followed up for a mean 19 ± 9 months for all cardiovascular events (mortality, acute coronary syndrome, revascularization >90 days after CCTA). In ROC-analysis for prediction of significant CAD, the areas under the curve for FRS; 0.68 (95% confidence interval: 0.64-0.72) and for SCORE; 0.69 (95% confidence interval: 0.65-0.72) were significantly higher than for PROCAM; 0.64 (95% confidence interval: 0.61-0.68; P ≤ .001), as well as marginally higher than for Diamond Forrester; 0.65 (95% confidence interval: 0.61-0.68; P ≤ .05). Low FRS category showed the lowest number of patients with significant CAD, compared to patients with low risk using PROCAM, SCORE or Diamond Forrester (P < .001). Also, low FRS category showed less events (compared to PROCAM and SCORE; P < .001, for Diamond Forrester; P = .14).
CONCLUSION: Our data show that in a stable chest pain population, the ability of FRS and SCORE to predict for CAD was similar and better compared to PROCAM and Diamond Forrester. The number of low risk patients showing significant CAD or events was lower using FRS. Consequently, risk categorization using FRS seems to be safest to stratify stable chest pain patients prior to CCTA.

Introduction

Cardiovascular disease (CVD) is the leading cause of death for both men and women in the United States.1 The CVD epidemic is expected to rise even further and it is forecasted that by 2030, 40.5% of the US population will have some form of CVD.2 Globally, the atherosclerotic burden is increasing in an even more rapid pace, with more expected CVD deaths in India or China than in all developed countries added together by 2030.3

In clinical practice, physicians are often confronted with patients presenting with chest pain, in whom clinical risk profiling is required.4,5 Recently, cardiac computed tomographic angiography (CCTA) has been introduced in the clinical workup of chest pain patients. The excellent negative predictive value of this technique makes it most suitable to rule out coronary artery disease (CAD).6 Although initial reports on diagnostic accuracy and prognostic value of CCTA are very promising, there are disadvantages such as the substantial radiation dose and administration of contrast agent. Therefore, most guidelines and appropriateness criteria recommend use of CCTA for the intermediate risk category.7,8 To assess the pre-test probability of having CAD prior to CCTA, clinical risk profiling is often used. Several algorithms to assess clinical risk have been developed, which predict for cardiovascular events in a different way, and are calibrated in different geographical regions and in diverse patient and population cohorts. Framingham risk score (FRS) predicts for 10-year risk of having any cardiovascular event, whereas PROCAM predicts for risk of myocardial infarction only. The SCORE algorithm predicts for death due to myocardial infarction and not for disease. Also, none of these scores were developed for use in symptomatic patients. The Diamond Forrester pre-test probability model was developed to assess probability of having significant CAD in symptomatic patients.9 Yet, it is not known which algorithm is most suitable to predict the likelihood of having CAD. The identification of the most precise algorithm to assess the likelihood of CAD is essential for effective alignment of healthcare resources and safety of the patient. We studied the most commonly used risk profiling algorithms in their ability to predict for (1) CAD on CCTA, and (2) for major adverse cardiovascular events, in patients presenting with chest pain at the cardiology outpatient clinic.

Methods

Study Population

Between December 2007 and June 2010, 1,891 patients presenting with chest pain at our outpatient clinic were enrolled in this study. Inclusion criteria were a recent history of cardiac (a)typical chest pain, a diagnostic CCTA scan, defined as seven or more interpretable coronary segments. Exclusion criteria were unstable angina, previous myocardial infarction, previous revascularization, hemodynamic instability, contrast allergy, pregnancy, and renal failure. In 1,518 patients, a complete lipid spectrum was collected, Diamond Forrester pre-test probability, Framingham, PROCAM, and SCORE risk score were assessed and a CCTA scan was made to assess the extent of CAD. In 222 subjects, CCTA was excluded because of artifact formation, or previous revascularization. Eventually, 1,296 subjects were analyzed in this study. Institutional Review Board and ethics committee at the Maastricht University Medical Center approved the study and all patients signed informed consent.

Clinical Definitions

Cardiac risk factors were gathered at the outpatient clinic. Patients were classified as active smoker if they had smoked in the previous 12 weeks. Patients treated with a hypoglycemic agent or with a fasting plasma glucose ≥126 mg/dL were classified as diabetic. A family history of CAD was defined as having a first-degree relative with a history of myocardial infarction or sudden cardiac death before the age of sixty.

Diamond Forrester score

The probability of having significant CAD was calculated using the Diamond Forrester model. This model takes into account age, sex, and type of chest pain, which was classified as typical, atypical or non-anginal.9 The commonly used classification cut-offs of 30% and 70% were used.10 Consequently, a score below 30% was considered low, 30%-70% intermediate and >70% high risk of having significant CAD.

Framingham risk score

The Framingham risk score is a multivariable risk function that predicts 10-year risk of developing cardiovascular disease events (coronary heart disease, stroke, peripheral artery disease or heart failure). The sex-specific scores incorporate age, total and high-density lipoprotein cholesterol, systolic blood pressure, treatment for hypertension, smoking, and diabetic status. A score below 10% is considered low, 10%-20% intermediate, and >20% high 10-year risk of cardiovascular events.11

PROCAM risk score

PROCAM participants were followed up for acute coronary events (myocardial infarction, sudden cardiac death) for 10 years. The calibrated risk score included; age, LDL cholesterol, smoking, HDL cholesterol, systolic blood pressure, family history of premature myocardial infarction, diabetes mellitus, and triglycerides.12 A score below 10% is considered low, 10%-20% intermediate, and >20% high 10-year risk of coronary events.

SCORE risk score

The SCORE predicts 10-year risk on fatal cardiovascular disease resulted in a model which included gender, age, systolic blood pressure, total cholesterol, and smoking. A score of 0%-4% was considered low, 5%-9% intermediate, and ≥10% high risk of cardiovascular death in 10 years.13

CCTA Acquisition

CCTA was performed using a 64-slice CT scanner (Brilliance 64; Philips Healthcare) with a 64 × 0.625 mm slice collimation, a gantry rotation time of 420 ms and a tube voltage of 80-120 kV depending on the patient’s height and weight. Patients received 5-20 mg of Metoprolol intravenously to lower the heart rate (HR) <65 bpm as well as sublingual nitroglycerin spray. HR and ECG were monitored during CCTA. CCTA was performed using 85-110 mL of contrast agent (Xenetix 350; Guerbet), which was injected in the antecubital vein at a rate of 6.0 mL/s, directly followed by 40 mL intravenous saline (6.0 mL/s). In patients with HR <65 bpm, a prospective-gated “Step and shoot” protocol was used.14 In patients with HR >65 bpm, a retrospective-gated “Helical” protocol with dose modulation was used to obtain the best image quality at minimal radiation dose.15

CCTA Coronary Plaque Assessment

All CCTA scans were independently analyzed by two experienced cardiologists, both blinded for patient details. In case of disagreement, consensus was reached by discussion. To indicate the location of coronary atherosclerosis, the classification of the American Heart Association (AHA) in 16 segments was used.16 The coronary artery tree was assessed using the source images on the Cardiac Comprehensive Analysis software (Philips Healthcare). Coronary plaques were defined as visible structures within or adjacent to the coronary artery lumen, clearly distinguished from the vessel lumen and the surrounding pericardial tissue. The degree of stenosis of atherosclerotic lesions was evaluated visually and classified as insignificant (no lesions, or one or more lesions with luminal stenosis of <50%), or significant (one or more lesions with luminal stenosis of ≥50%).17

Statistics

Data were analyzed using SPSS 17.0 (SPSS Inc). Continuous variables were reported as means and SDs and proportions (%) were used for categorical values. Receiver operating curves (ROC) were produced and area under the curve (AUC) was reported for different risk scores in relation to significant CCTA lesions. Mc Nemar test was used to assess significance of difference between different risk score categories. All P values were 2-sided, and a value below 0.05 was considered significant.

Follow-up

Electronic patient records were monitored for all-cause mortality and acute coronary syndrome (ACS), including myocardial infarction and unstable angina requiring hospitalization. Additionally, the national mortality records were checked. ACS was defined as typical angina pectoris and troponin T elevation (>0.01 μg/L) and ST-segment elevation/depression of ≥1 mm, or at least two of these symptoms together with invasive angiographic conformation of a culprit lesion.18 Secondary endpoints included percutaneous coronary intervention (PCI) and coronary artery bypass graft (CABG) surgery. Revascularization procedures <90 days after CCTA were excluded as an event, as these would predominantly be CCTA driven. Further, we censored follow-up after the first endpoint, so that the recorded ACS was not a complication of revascularization therapy. Patients were seen by their cardiologist on a regular basis, and all visits were recorded in the electronic patient records.

Results

Patient Characteristics

Baseline characteristics are reported in Table 1. Of 1,296 patients, any CAD was found in 62%, significant CAD in 25%. Mean follow-up period was 19 ± 9 months, and a total of 47 events were reported. Events consisted of 7 deaths, 18 cases of ACS (of which 5 were diagnosed as myocardial infarction), and 22 cases of revascularization (15 PCIs and 7 CABGs).

Table 1

Baseline characteristics

Age (years)	56 ± 11
Female gender	606 (46.8)
BMI (kg/m2)	27 ± 5
Active smoking	316 (24.4)
Diabetes mellitus	102 (7.9)
Positive family history	522 (40.3)
Systolic blood pressure (mmHg)	142 ± 19
Typical chest pain	169 (13)
Cholesterol (mg/dL)
Total	209 ± 46
LDL	128 ± 42
HDL	51 ± 29
Triglycerides (mg/dL)	153 ± 100
Glucose (mg/dL)	104 ± 24
Creatinin (mg/dL)	1.1 ± 0.2
Clinical risk scores
Framingham	21 ± 16
PROCAM	12 ± 13
SCORE	4 ± 4
Diamond Forrester	42 ± 26
CAD on CCTA
No CAD	490 (37.8)
Insignificant CAD (<50% stenosis)	489 (37.7)
Significant CAD (≥50% stenosis)	317 (24.5)

BMI, Body mass index; CAD, coronary artery disease; CCTA, cardiac computed tomographic angiography; LDL, low-density lipoprotein; HDL, high-density lipoprotein.

Overall Prediction of CAD per Risk Score

In ROC-analysis for prediction of any coronary lesion, the areas under the curve for FRS; 0.74 (95% confidence interval: 0.72-0.77) and for SCORE; 0.72 (95% confidence interval: 0.70-0.75) were significantly higher than for PROCAM; 0.70 (95% confidence interval: 0.67-0.73; P ≤ .03), which was significantly higher than for Diamond Forrester; 0.65 (95% confidence interval: 0.62-0.68; P < .01), Figure 1. Moreover, the areas under the curve for predicting significant CAD stenosis for FRS; 0.68 (95% confidence interval: 0.64-0.72) and for SCORE; 0.69 (95% confidence interval: 0.65-0.72) were significantly higher than for PROCAM; 0.64 (95% confidence interval: 0.61-0.68; P ≤ .001), as well as marginally higher than for Diamond Forrester; 0.65 (95% confidence interval: 0.61-0.68; P = .05), Figure 2.

Figure 1

ROC analysis of Framingham, PROCAM, SCORE, and Diamond Forrester (DF) score in predicting any lesion on CCTA, area under the curve is 0.74, 0.72, 0.70, and 0.65, respectively

Figure 2

ROC curves for Framingham, PROCAM, SCORE, and Diamond Forrester (DF) score in predicting ≥50% lesion on CCTA, area under the curve is 0.68, 0.69, 0.64, and 0.65, respectively

Risk Categorization per Risk Score

Table 2 displays the number (%) of patients per risk category according to FRS, PROCAM, SCORE, and Diamond Forrester in relation to CAD and number of events during follow-up. All scores were highly predictive for CAD (P < .001). Low FRS category showed the lowest number of patients with significant CAD (45/374 = 12%), compared to patients with low risk using PROCAM, SCORE or Diamond Forrester (P < .001). Also, low FRS category showed significantly lower number of events (6/47 = 13%), compared to PROCAM and SCORE (P < .001). When comparing to Diamond Forrester, however, the number of events was not significantly lower (P = .14). As a consequence, FRS high risk category included less patients with significant CAD and events, compared to patients with high risk using PROCAM, SCORE or Diamond Forrester (P ≤ .02). In addition, in the low FRS category, 62% showed no CAD, compared to 48%, 47%, and 47% for low PROCAM, SCORE, and Diamond Forrester (P ≤ .04). Figure 3 provides a visual overview of proportions of CAD in different risk categories, according to the different scoring models.

Table 2

Risk categories in FRS, PROCAM, SCORE, and Diamond Forrester (DF) in relation to CAD outcome and events

	Low risk				Intermediate risk				High risk
	FRS	PROCAM	SCORE	DF	FRS	PROCAM	SCORE	DF	FRS	PROCAM	SCORE	DF
	0–10	0–10	0–4	0–30	10–20	10–20	4–9	30–70	>20	>20	≥10	>70
No CAD	233 (62)	360 (48)	426 (47)	261 (47)	166 (42)	93 (31)	57 (21)	188 (31)	91 (17)	37 (15)	7 (6)	41 (28)
Insignificant CAD	96 (26)	252 (34)	318 (35)	196 (36)	155 (39)	122 (41)	114 (41)	236 (40)	238 (46)	115 (46)	57 (48)	57 (38)
Significant CAD	45 (12)	136 (18)	159 (18)	95 (17)	77 (19)	85 (28)	104 (38)	171 (29)	195 (37)	96 (39)	54 (46)	51 (34)
Total number	374	748	903	552	398	300	275	595	524	248	118	149
Events	6 (2)	17 (2)	25 (3)	13 (2)	16 (4)	13 (4)	13 (5)	28 (5)	25 (5)	17 (7)	9 (8)	6 (4)

Number (percentage) of patients with no CAD, insignificant CAD (<50% lesion) or significant CAD (≥50% lesion) within low-, intermediate-, and high-risk according to Framingham, PROCAM, SCORE, and Diamond Forrester. Percentage of total number within group is given. FRS, Framingham risk score; CAD, coronary artery disease.

Figure 3

Proportions of significant, insignificant and no CAD for low-, intermediate-, and high-risk categories by FRS, PROCAM, SCORE, and Diamond Forrester (DF)

Discussion

Our data show that the ability of FRS and SCORE to predict for CAD was similar and significantly better compared to PROCAM and marginally better than Diamond Forrester. The number of low risk patients showing significant CAD or events was lower using FRS, compared to PROCAM, SCORE, and Diamond Forrester, using the indicated cut-off points for low-, intermediate- or high-risk for the different algorithms. Consequently, risk categorization using FRS is safest, but comes with a cost as more individuals with high risk do not have significant CAD or events, compared to PROCAM, SCORE, and Diamond Forrester.

The cardiovascular disease epidemic produces a heavy burden on medical care. As the population ages, the cardiovascular disease burden will increase and costs are expected to rise substantially.2 Consequently, this is of great concern for physicians confronted with patients presenting with stable chest pain. The first question clinicians confronted with these patients want answered, concerns the chance of having CAD, and secondly the risk of having a cardiovascular event in the future. Clinical risk profiling remains the advised starting point for evaluation of these patients. Using the most effective clinical risk profiling algorithm is essential, and will make a substantial difference in risk stratification on a large scale. Above all, effective initial risk stratification will facilitate more efficient use of further diagnostics, all with their accompanying costs and disadvantages.

With the rapidly expanding epidemic of CAD in many developing countries, the identification of the most effective algorithm to predict for presence of CAD is an important step in allocation of appropriate care. Especially since in most developing countries the availability of diagnostics is limited and insurance programs are lacking. The fact that cost-effective use of health care resources also becomes more and more a critical issue in Western countries, makes it a global matter. An additional reason to use the most precise prediction model for presence of CAD is the issue of radiation dose in follow-up diagnostics. Concerns have been raised about radiation dose and cancer risk implied by cardiac imaging techniques.19 Therefore, although recent innovations in CT scanners have substantially lowered effective dose, CCTA in stable chest pain patients is still not generally advised.20 Most guidelines find CCTA appropriate in stable chest pain patients with intermediate risk profiles.7,8 In patients with a low risk profile, the benefits of CCTA do not seem to outweigh its costs and radiation dose. We showed that when applying these appropriateness criteria, FRS was safest, as less patients with any CAD, significant CAD or events were found in the low FRS group, as compared to the low PROCAM, SCORE group. Also, significantly less patients with any CAD or significant CAD were found in the low FRS group, as compared to the low Diamond Forrester group.

When using cardiovascular risk profiling algorithms to risk stratify stable chest pain patients prior to CCTA, one must realize that these scores were developed to predict coronary heart disease and not coronary atherosclerosis. However, there seems to be a strong relation between coronary atherosclerotic burden and risk of future coronary heart disease events.21 We acknowledge that all scores were based on different risk factors, and were calibrated on different geographical regions and populations. Above all, we investigated a symptomatic population. In contrast to the clinical risk algorithms, Diamond Forrester pre-test probability score might be more appropriate for prediction of CAD, but was never developed to predict for events. Surprisingly, the FRS, which was calibrated for an American population, performed better than the European calibrated PROCAM risk score. The major difference between the two algorithms is that FRS includes blood pressure treatment, further differences must be explained by calibration factors. For that matter, our findings seem concurrent with previous reports investigating FRS in the PROCAM population. Hense et al reported FRS to overestimate actual risk in the PROCAM population.22 For the use of risk stratification prior to CCTA, this overestimation might result in more safety, as we reported the low FRS category to have less CAD and events in this study. In SCORE, no glucose or diabetic status is regarded in the risk score algorithm. Possibly, this may be a disadvantage in countries were DM is on the rise. In this study, the performance of Diamond Forrester seemed to somewhat disappoint. There are several explanations for this observation. First, the Diamond Forrester score was developed in 1970s, for populations undergoing conventional angiography. In contrast, the current population undergoing CCTA is a relatively low risk group, in which the majority presented with atypical chest pain. Eventually, only 25% showed significant coronary stenosis. In addition, there is evidence that conventional angiography and CCTA seem to assess the severity of stenosis differently, especially in non-circular geometry.23 Furthermore, although symptoms are incorporated in the Diamond Forrester score, FRS includes more clinical risk factors. Given our data, this apparently compensates for the lack of information about symptoms.

In patients categorized as low risk according to FRS, still 12% had significant lesions in our study. Also, 13% of all events occurred in the low FRS group. This is in concurrence with previous reports, and questions the relative safety of using clinical risk profiling as a screening tool in patients with stable chest pain.24,25 Performance of calcium scoring in addition to clinical risk profiling has shown an increment in predictive value for CAD and cardiovascular events, and certainly makes a safer yet more expensive screening tool compared to clinical risk profiling only.26 There is ample evidence that calcium score is a good predictor for cardiovascular events.26,27 The role of CCTA as a risk predictor is still more uncertain, although CCTA is reported to have an excellent negative predictive value,6 as well as providing additive value in identifying the patient at risk.28 In addition, there is some evidence that CCTA might outperform clinical risk profiling as well as calcium scoring in predicting events.29 Question remains what will be the most cost-effective strategy on the long term. It is plausible that more extensive use of imaging techniques for early detection of patients at risk for CAD can increment preventive therapy.30 In the end, prevention of disease will be more cost-effective compared to treatment of manifest CAD, and innovations in imaging techniques such as CCTA will make its application more justifiable, probably even for low risk patients. However, for now the initial use of clinical risk profiling is a cornerstone in the assessment of patients to be assessed for CAD and determines the appropriateness of further CCTA studies. We hereby show that the use of FRS is safer to engage this role as compared to PROCAM, SCORE, and Diamond Forrester score.

Study Limitations

The analysis was strengthened by the relatively large sample size. In general, the study population consisted of patients of European descent, presenting with both typical and atypical stable chest pain, in the Netherlands. Therefore, caution should be taken in interpreting these study results on other populations, and similar evaluations in different populations and regions worldwide seem valuable. We used: (1) clinical risk profiling algorithms that were developed to predict different cardiovascular events within a 10-year period and (2) Diamond Forrester score which was developed to predict pre-test probability of significant CAD prior to conventional angiography, to predict for CAD as assessed by CCTA. Although these scores were not developed for this purpose, there is no designated pre-test probability score for CCTA yet, resulting in the widespread alternative use of these scores. The relative short follow-up time and small number of events made the data on cardiovascular events preliminary. We acknowledge the possibility that events could have presented elsewhere, resulting in an incomplete follow-up. However, as all patients were followed up in our clinic, we expect the completeness of the follow-up to be high.

Conclusion

Our data show that in a stable chest pain population referred for CCTA, the ability of FRS and SCORE to predict for CAD was similar and significantly better compared to PROCAM and marginally better than Diamond Forrester. The number of low risk patients showing significant CAD or events was lower using FRS, compared to PROCAM, SCORE, and Diamond Forrester, using the indicated cut-off points for low-, intermediate- or high-risk for the different risk profiling algorithms. Consequently, risk categorization using FRS seems to be safest in stable chest pain patients, but comes with a cost as more individuals with high risk will not have significant CAD or events, compared to PROCAM, SCORE, and Diamond Forrester.