Construction and Validation of a 14-Year Cardiovascular Risk Score for Use in the General Population: The Puras-GEVA Chart.

UNLABELLED: The current cardiovascular risk tables are based on a 10-year period and therefore, do not allow for predictions in the short or medium term. Thus, we are unable to take more aggressive therapeutic decisions when this risk is very high.To develop and validate a predictive model of cardiovascular disease (CVD), to enable calculation of risk in the short, medium and long term in the general population.Cohort study with 14 years of follow-up (1992-2006) was obtained through random sampling of 342,667 inhabitants in a Spanish region. MAIN OUTCOME: time-to-CVD. The sample was randomly divided into 2 parts [823 (80%), construction; 227 (20%), validation]. A stepwise Cox model was constructed to determine which variables at baseline (age, sex, blood pressure, etc) were associated with CVD. The model was adapted to a points system and risk groups based on epidemiological criteria (sensitivity and specificity) were established. The risk associated with each score was calculated every 2 years up to a maximum of 14. The estimated model was validated by calculating the C-statistic and comparison between observed and expected events.In the construction sample, 76 patients experienced a CVD during the follow-up (82 cases per 10,000 person-years). Factors in the model included sex, diabetes, left ventricular hypertrophy, occupational physical activity, age, systolic blood pressure × heart rate, number of cigarettes, and total cholesterol. Validation yielded a C-statistic of 0.886 and the comparison between expected and observed events was not significant (P: 0.49-0.75).We constructed and validated a scoring system able to determine, with a very high discriminating power, which patients will develop a CVD in the short, medium, and long term (maximum 14 years). Validation studies are needed for the model constructed.

INTRODUCTION

Cardiovascular diseases (CVD) with underlying atherosclerosis are the leading cause of mortality in the world.[1] As a result, predictive models of CVD have been obtained to determine which patients are more likely to develop a disease of this nature and in turn to take action on modifiable factors to decrease the likelihood of CVD.[2] Because these models come from complex mathematical expressions, they, however, have been transformed into points systems so that health professionals can calculate the probability of CVD by performing simple sums to determine the benefits of a possible intervention on cardiovascular risk factors.[3]

As a reference, the Systematic COronary Risk Evaluation risk chart (SCORE) is used in Europe whereas the United States uses the Framingham risk table.[2] These risk charts are based on a follow-up of 10 years and thus do not allow for accurate predictions in periods shorter (short or medium term) or longer than 10 years (long term). This is a clinically important question, because if we know that a patient has a high probability of developing CVD in the short term (eg, 2 years), we should give more intense drug and nonpharmacological therapy, because failing to do so may result in the patient experiencing a CVD. Given that we are unable to obtain short- to medium-term probabilities from the current points systems, we conducted a study in a Spanish region in which we constructed and validated a predictive model for CVD of the proposed features using a population-based cohort followed up for a period of 14 years. The results of this work provide a tool to help us make treatment decisions in the short, medium, and long term to reduce the incidence of CVD in the general population.

METHODS

Study Population

The study population comprised inhabitants of the province of Albacete (Spain) who were at least 18 years of age (adults). In 1991, this province consisted of 342,667 inhabitants, equivalent to 0.89% of the whole country.[4]

Study Design and Participants

This was a population-based cohort study with a maximum follow-up of 14 years. The cohort was recruited through a 3-stage sampling design of all the inhabitants in the province of Albacete registered in the 1991 census. The first phase consisted of a stratified sample based on groups according to the size of the population of the nucleus of residence (capital, 39.7%; >10,000, 23.3%; 2001–10,000, 20.8%; 501–2000, 14.5%; and <501, 1.7%); the second phase was based on a cluster sampling of the municipalities contained in the above groups, and the final phase was a simple random sampling. The sample size in each municipality was proportional to the size of its population. This process (baseline) was conducted between February 1, 1992 and September 5, 1994. Between July 13, 2002 and December 2, 2006, a second visit to the study participants (follow-up) was performed. For both processes, the selected patients were contacted by mail (up to 2 times) and by telephoning those who did not respond. Patients in secondary cardiovascular prevention were excluded from this study.

Variables and Measurements

The primary study variable was the time to the first occurrence of CVD (time-to-event data). This (date of event occurrence) was obtained from the patient's clinical documents and was defined as presenting at least one of the following conditions: angina of any kind, myocardial infarction, stroke, peripheral arterial disease of the lower limbs, or death from CVD. Mortality (date and cause) was obtained through the patient's death certificate. Secondary variables were sex; personal history of hypertension, diabetes, and dyslipidemia; family history of coronary heart disease, left ventricular hypertrophy; occupational physical activity (heavy or moderate activity; light activity); age (years); body mass index (BMI) (kg/m2); systolic (SBP) and diastolic blood pressure (mm Hg); cigarettes per day; fasting blood glucose (mmol/L); total cholesterol (TC) (mmol/L); high-density lipoprotein cholesterol (mmol/L); triglycerides (mmol/L); fibrinogen (μmol/L); heart rate (bpm); and ankle brachial index (ABI). Moreover, the product of the SBP and heart rate was used.[5]

A patient was considered to have a personal history of hypertension, dyslipidemia, or diabetes when there was daily drug treatment or the patient responded affirmatively to the question: Have you been told by your doctor that you have diabetes, hypertension, or hypercholesterolemia? Sex, date of birth, number of cigarettes smoked (as in other studies,[2] this variable had a value of 0 when the patient was either a nonsmoker or a former smoker), and occupational physical activity were obtained through interviews with the patient. The latter was measured according to the Food and Agriculture Organization of the United Nations and the World Health Organization: light activity: activity associated with sitting at a desk or behind a counter with automated instruments; moderate activity: continuous light physical activity, such as light work in industry or in agriculture out of season; intense activity: heavy work and at times, energetic (agricultural production, mining, or steel work). If a person was not working, they were classified as performing light activity.[6] In addition, when a relative (parents, children, and/or siblings) had suffered an event before 56 years of age this was considered to be a family history of ischemic heart disease. This variable was collected by interview with the patient.

Left ventricular hypertrophy and heart rate were determined through an electrocardiogram. The variables relating to blood tests were obtained after a minimum 12-hour fast. The BMI was calculated by measuring the weight and height of the patient with calibrated equipment and with the patients in underwear and barefoot. Blood pressure was measured using a sphygmomanometer and stethoscope following the procedures regulated by the guidelines recommended in the consensus to control hypertension in Spain published in 1990 by the Ministry of Health and Consumer Affairs.[7] Finally, the ABI was measured through Doppler ultrasound equipment and a sphygmomanometer.[8]

Sample Size

The original recruitment of patients was intended to estimate the prevalence of peripheral arterial disease.[8] With this sample size, we determined how many patients had to be recruited in each stratum (see Study design and participants). The study estimating that prevalence finished and the patients were followed in a new study to determine prognostic factors for CVD. Because the original sample size was not calculated for this purpose, the accuracy of the sample was estimated with the new objectives (to construct and validate a new diagnostic test): construction: A total of 823 people, of whom 76 had a CVD (9.23%), were included. Expecting to find a specificity of 75% and establishing a confidence level of 95%, the accuracy in estimating the specificity was 3.11%. Validation: in this sample, 26 of the 227 had a CVD. Assuming an area under the receiver operating characteristic curve (AUC) of 0.90 and establishing a confidence level of 95% to contrast an AUC different from 0.5, a power of contrast close to 100% was obtained.[9]

Statistical Methods

General: absolute and relative frequencies were used to describe the qualitative variables, whereas for quantitative variables means and standard deviations were used. All analyses were performed with α = 5% and for each relevant parameter, its associated confidence interval (CI) was calculated. All analyses were performed using IBM SPSS Statistics 19 (IBM, Armonk, NY), Epidat 3.1 (Junta de Galicia, Galicia, Spain), Microsoft Office Excel 2007 (Microsoft, Redmond, Washington DC), and R 2.13.2.

Comparison of patients who completed the study and lost patients: a comparison between these 2 groups was performed using χ2 tests (Pearson or Fisher) and Student t test, according to the type of each variable. Comparison of patients in the construction and validation samples: the patients who participated in the study (no losses) were randomly divided into 2 groups: the construction sample (80% of the sample) and the validation sample (20%). To verify that there were no differences between the 2 groups, the same tests as performed on the comparison of the losses were used.

Construction of the model: in the construction sample (80%), a multivariate Cox regression model was performed to identify which variables were associated with CVD. For this, a forward stepwise algorithm based on the likelihood ratio test to determine those variables that may better predict CVD was used. The goodness-of-fit of the model was assessed with the likelihood ratio test and verification of the proportional risks was evaluated using the Schoenfeld residuals method. After estimating the model parameters (β coefficients), through the methodology of the Framingham study,[10] a points system was constructed taking into account the specific weight of each variable in the development of CVD (β coefficients). In other words, the β coefficients were adapted to a system that can be used by health professionals systematically and without requiring the use of electronic devices to perform the calculations, because categorizations of cardiovascular risk factors are made to which a score is associated, according to the β coefficient of said factor obtained in the multivariate model.[10] The points and their associated risk were calculated every 2 years up to a maximum of 14 years. The following cutoff points were chosen: optimal point: the score that minimized the square root of (1-sensitivity)2 + (1-specificity)2; confirmation point: the minimum score value that had a positive likelihood ratio greater than or equal to 10; discard point: the maximum score that had a negative likelihood ratio of less than 0.1. Points 2 and 3 were chosen because, according to the School of Evidence Based Medicine, they are those that permit conclusive confirmation or discarding of positivity and negativity, respectively, of a diagnostic test. Furthermore, CVD risk groups were formed using the points obtained previously: low risk (below Discard point), medium risk (from the discard point to the optimal point), high risk (from the optimal point to the confirmation point), and very high risk (greater than or equal to the confirmation point). This methodology has been used in similar studies.[11,12]

Validation: the points system was implemented in the validation sample (20%) and the AUC and the C-statistic were calculated. In addition, the survival of the risk groups was compared using the log-rank test and the survival curves were plotted using the Kaplan–Meier technique. Finally, the observed and expected events were plotted and compared every 2 years (up to a maximum of 14 years) using the χ2 test.

Ethical Issues

The study was approved by the Ethics Committee of the General University Hospital of Albacete and carried out in accordance with the ethical standards set forth in the Declaration of Helsinki of 1964 and its subsequent amendments. All patients gave written informed consent before inclusion in the study.

RESULTS

Of 1322 patients, 80 were excluded for having a history of CVD, leaving a total of 1242 individuals who were followed until the development of a cardiovascular event. During the study period, no information was obtained from 192 patients, thus they were excluded. Table 1 shows the characteristics of those who remained in the study and those who dropped out, highlighting some significant differences between the 2 samples (dyslipidemia, age, BMI, SBP, diastolic blood pressure , TC, and ABI).

TABLE 1

Descriptive Analysis of Patients Who Completed or Withdrew From the Study in Albacete (Spain), 1992 to 1994 Data

Once the sample was divided into 2 groups (80% and 20%); no statistically significant differences were seen (P-values between 0.054 and 0.868). In these samples, the majority was women, there were patients with cardiovascular risk factors, a majority of the patients carried out moderate or intense activity at work, and the average age was approximately 47 years (see Table 2).

TABLE 2

Descriptive Analysis for Construction and Validation Samples in Albacete (Spain), 1992 to 1994 Data

The construction sample consisted of 823 individuals, of whom 76 had a CVD during the follow-up (33 fatal), equivalent to an incidence density of 82 cases per 10,000 person-years (95% CI: 65–103; 36 fatal, 95% CI: 25–50). Regarding the multivariate model (Table 3), the following profile of prognostic factors for CVD was obtained: male sex, diabetes, left ventricular hypertrophy, light occupational physical activity, older age, higher value of SBP × heart rate, higher number of cigarettes per day, and higher values of TC. The overall model was highly significant (P < 0.001), that is, our model explained the development of CVD better than the null model (model with no explanatory variables), and provided the following cutoff points: optimal, 10 (square root = 0.343); discard, 7 (negative likelihood ratio = 0.07); and confirmation, 14 (positive likelihood ratio = 10.56). The probabilities of each of the possible scores, along with its associated risk every 2 years (up to a maximum of 14 years) are reflected in Table 4. The scoring system constructed through the multivariate model is shown in Figure 1.

TABLE 3

Multivariable-Adjusted Cox Proportional Hazards Regression Coefficients for 14-Year Risk of Cardiovascular Disease in Albacete (Spain), 1992 to 1994 Data

TABLE 4

Likelihood (%) of Having a Cardiovascular Disease by Score and Follow-Up Time in Albacete (Spain), 1992 to 1994 Data

FIGURE 1

Predictive model to determine which patients will suffer a cardiovascular disease within a maximum period of 14 years. Definition of occupational physical activity: light activity: activity associated with sitting at a desk or behind a counter with automated instruments; moderate activity: continuous light physical activity, such as light work in industry or in agriculture out of season; intense activity: heavy work and, at times, energetic (agricultural production, mining, or steel work). If a person was not working, he or she was classified as performing light activity. LVH = left ventricular hypertrophy, PA = physical activity, SBP = systolic blood pressure.

As regards the validation sample, a C-statistic value of 0.886 (standard error = 0.061) and an AUC with a very similar value were obtained (Fig. 2). When comparing survival between the different risk groups (Fig. 3), we observed that as the risk category increased, the probability of experiencing a CVD increased very significantly (P < 0.001). Finally, comparison between the expected and observed events in all risk groups every 2 years (Fig. 4) showed no statistically significant differences (P-values between 0.49 and 0.75).

FIGURE 2

Area under the receiver operating characteristic curve of the points system in the validation sample. AUC = area under the receiver operating characteristic curve, CI = confidence interval.

FIGURE 3

Differences between the different risk groups constructed in the validation sample.

FIGURE 4

Comparison between observed and expected events in the validation sample.

DISCUSSION

Summary

This study constructed and validated a cardiovascular risk table with primary data to determine the risk every 2 years, up to a maximum of 14 years. The results enable us to make decisions in the short, medium, and long term to prevent CVD in a new patient.

Strengths and Limitations of the Study

The main strength of this study is the methodology followed in constructing this cardiovascular risk scale because, unlike other risk tables,[2] it enables us to make decisions in the short and medium term. Thus, if a new patient has a high probability of developing CVD in the short term, we should carry out more intensive treatment of the factors in our points system on which it is possible to act (TC, SBP, and smoking). Moreover, we have included new factors that do not appear in the cardiovascular risk tables used.[2] Finally, regarding statistical issues, a very high discriminatory power was obtained.

To minimize possible selection bias, we used a random sample design that considered the whole population in the province we were analyzing. Information bias was minimized because calibrated and validated equipment was used, and great care was taken when measuring all parameters. In addition, all the medical records and hospital reports of each participant were comprehensively reviewed. Finally, although we collected our data in the early 1990s (baseline) and early 2000s (second visit), we must also consider that the data collection for both the Framingham Heart Study and the SCORE project was performed before our study and these tables are still being used today in daily clinical practice around the world.[2]

Regarding statistical issues, the sample size used was smaller than in other risk scales.[2] We, however, must bear in mind that this size was sufficient for the proposed objectives (validation power close to 100%), which, added to a sample design of the defined characteristics, provided great validity to our findings. The constructed model obtained factors that were nonsignificant independently (each factor separately). When constructing the model, the choice of variables, however, was performed using a forward stepwise algorithm based on the likelihood ratio test; thus, when these factors were entered into the model they did reach statistical significance (P < 0.05). We must also highlight that we are considering the totality of the constructed model,[13] that is, its discriminatory capacity and the obtaining of results similar to reality (observed–expected comparison), which as a whole was very satisfactory (C-statistic and AUC close to 90%, and no differences found between observed and expected events).

Comparison With the Existing Literature

Our methodology has differences when compared with the models mainly used in Europe and United States (SCORE and Framingham). First, we must take into account that these models only allow for long-term predictions, whereas ours enables decision making in the short, medium, and long term. The type of sampling used in this study was completely random, whereas the Framingham study participants were volunteers and in SCORE, there were some patients from working population cohorts. Finally, the age range of the Framingham and SCORE studies is more restricted, whereas our study allows the constructed scale to be applied to all adults (≥18 years). The Framingham and SCORE studies, however, have a minimum and maximum patient age for their use.[2]

The factors found in our model are consistent with the current literature[2] except that the product of the heart rate and SBP is used in a novel way in our model. As this variable has shown its weight in predicting CVD,[5] its weight is logical and expected when determining which patients will develop CVD. Secondly, work activity other than sitting at a desk or behind a counter with automated instruments had a protective nature in our outcome. Given that other studies have found that exercise has this protective character and adherence to exercise is not poor (because otherwise the patient could not perform their work properly), it makes sense to find this factor in our risk scale as well.[14] Finally, quantifying smoking with the number of cigarettes enables setting targets in patients who are smokers so that a partial reduction of the habit could lead to a decrease in cardiovascular risk. As this factor has only been assessed in a binary form in the SCORE and Framingham studies, a partial reduction in the number of cigarettes in the patient who smokes does not produce a decreased risk.[2]

Assessing the discriminating power of the model in determining which patients will suffer a CVD gave an AUC of 0.90 and a C-statistic very similar to this value. The other scales obtained a maximum value of 0.82 in their internal validation.[2] Hence, if we apply our model in other geographical areas and obtain a value similar to that found in our validation sample, our risk scale could become an alternative to the scales of previous studies. Nonetheless, because heart rate and work activity must be included, these variables should be taken into account in the validation of our predictive model on data from cohort studies conducted in other geographical areas.

Implications for Research and Practice

The preparation and internal validation of this new cardiovascular risk scale with a higher number of discriminating power than those currently known indicates that, if similar results are obtained in other populations, our predictive model could become a reference when calculating the cardiovascular risk in the general population. We must be cautious though, we only applied this scale in the province of Albacete. The authors propose the validation of this predictive model in other populations. If this validation achieves results similar to ours, we can make decisions in the short, medium, and long term, agreeing upon realistic targets with smokers (partial reduction), controlling heart rate, and keeping in mind the work activity of the participant, in addition to other known and treated risk factors in preventing CVD.

CONCLUSIONS

This study developed and validated a points system able to determine, with a very high discriminating power, which patients will develop a CVD within a maximum period of 14 years. This discriminating power was higher than in the known scales. Therefore, if these results are maintained in validation studies in other geographical areas, the cardiovascular risk scale prepared in this study may be proposed as a tool for use in clinical practice to reduce the incidence of CVD in the general population.