Development and validation of a 2-year new-onset stroke risk prediction model for people over age 45 in China.

Multiple factors, including increasing incidence, poor knowledge of stroke and lack of effective, noninvasive and convenient stroke risk prediction tools, make it more difficult for precautions against stroke in China. Effective prediction models for stroke may assist to establish better risk awareness and management, healthier lifestyle, and lower stroke incidence for people.The China Health and Retirement Longitudinal Survey was the development cohort. Logistic regression was applied to model's development, in which the candidate variables with statistically significant coefficient were included in the prediction model. The area under receiver operating characteristic curve (AUC) and 10-times cross-validation were used for internal validation. Cutoff point of high-risk group was measured by Youden index. The China Health and Nutrition Survey was the validation cohort.The development cohort and the validation cohort included 16557 and 5065 participants, and the incidence density was 358.207/100,000 person-year and 350.701/100,000 person-year, respectively. The model for 2-year new-onset stroke risk prediction included age, hypertension, diabetes, heart disease, and smoking. The AUC and cross-validation AUC were 0.707 (95% confidence interval[CI]: 0.664, 0.750) and the 0.710 (95% CI: 0.650, 0.736). The sensitivity, specificity and accuracy of the cutoff point were 0.774, 0.545, and 0.319. The AUC and cross-validation AUC were 0.800 (95% CI: 0.744, 0.856) and 0.811(95% CI:0.714, 0.847), and the sensitivity, specificity and accuracy of cutoff point being 0.857,0.569, and 0.426 in external validation.A simple prediction tool using 5 noninvasive and easily accessible factors can assist in 2-year new-onset stroke risk prediction in Chinese people over 45 years old, which is believed to be applicable in identifying high-risk individuals and health management in China.

Introduction

The incidence of stroke has been declining worldwide, but it is still increasing by 5.4% per year in China.[ By 2016, China ranked the first in terms of age-specific stroke incidence and lifetime stroke risk[ around the world and stroke became the first cause of death[ and disability[ in China. The incidence of stroke increases significantly with age, and has a much higher morbidity among people over age 45.[ However, 45% of people at this age are unaware of any risk factor.[ About 50% of the residents in China have limited knowledge about the risk factors of stroke.[ Thus, multiple tasks make stroke prevention undoubtedly more intensified.[ Both the annually increasing incidence of stroke and the lack of relevant knowledge among high-risk individuals cause the dilemma of stroke prevention in China.

Prediction model for individuals can be used for risk alarming and lifestyle planning.[ As a result, stroke risk assessment, and management are of great significance for middle-aged and elderly people, who would be more aware of risk factors, and make behavioral adjustments to finally reduce the incidence.

Recently, stroke risk prediction model development has attracted much attention. The Framingham Stroke Risk Score (FSRS) involves gender, age, blood pressure, heart diseases, and diabetes mellitus to assess the risk in the next 10 years.[ With AUC of 0.588,[ FSRS poorly predicts the risk of stroke in Chinese population, and may overestimate the risk in Asian populations.[ Subsequently, FSRS was modified particularly for Chinese population, but AUCs of the modified model for the male and female were only 0.726 and 0.656 respectively.[ The Ischemic Cardiovascular Disease model (ICVD) is specially developed for the Chinese population and widely used, involving age, systolic blood pressure, body mass index (BMI), smoking, diabetes, and total cholesterol for risk prediction.[ However, not only stroke, but Ischemic heart disease could be predicted by ICVD, so its ability to predict the single outcome of stroke is unclear. Additionally, the 2 models above are outdated (FSRS in 1994, ICVD in 2006) to evaluate the changing risk factors and effects on stroke.[ Meanwhile, the target population in these 2 models fails to cover all middle-aged and elderly people (FSRS and ICVD cover age groups 55–85 and 35–59 respectively). Other stroke risk prediction tools for the Chinese population are mostly developed for special patient groups (e.g., diabetes mellitus, atrial fibrillation) rather than for the general population.[

Therefore, a new stroke risk prediction tool for Chinese people over age 45 was developed with larger sample size and relatively newer data in this study.

Methods

Study population

Development cohort

The development cohort was the China Health and Retirement Longitudinal Survey (Charls), which is conducted among families and individuals aged 45 and above in China every 2 years since 2011 and covers 28 provinces. The whole survey consists of 3 parts:

face-to-face electronic questionnaire: used to collect demographic information, health status and lifestyle from the respondents by professionals;

physical examination: including height, weight, waist circumference, blood pressure and other information;

laboratory examination (blood sample): conducted by the Chinese Center for Disease Control and Prevention. More information such as sampling methods, quality control, and laboratory inspection methods of Charls were published.[

Validation cohort

The validation cohort was the China Health and Nutrition Survey (CHNS), which is conducted by the Carolina Population Center at the University of North Carolina, the National Institute for Nutrition and Health and Chinese Center for Disease Control and Prevention since 1989, covering 9 provinces in China. Two waves of data were included in this study in 2009 and 2011. Face-to-face questionnaires and physical examinations were conducted by professionals, and blood samples were added in 2009. The samples were tested by the local community health service center and China-Japan Friendship Hospital. More information on CHNS were published.[

Figure 1 shows the derivation of 2 cohorts. Ethical approval is not necessary in this research.

Figure 1

Derivation of the study population.

Candidate predictors and outcome indicators

Based on previous studies, common understanding of epidemiology and the characteristics of development and validation cohorts, a total of 12 candidate predictive factors were included[: age, education, marital status, hypertension, diabetes, dyslipidemia, heart disease, sleep duration, smoking, alcohol consumption, BMI, and waist-to-height ratio (WHtR).

Hypertension status was determined based on the Hypertension Diagnostic Criteria of World Health Organization (WHO) and the Primary Prevention Guideline for Cerebrovascular Diseases of Chinese Population.[ It was determined according to both self-reported result and physical examination because middle-aged and elderly people in China have limited awareness of hypertension.[ Hypertension status was divided into 4 categories, including no hypertension, unknown hypertension, well-controlled hypertension, and poorly-controlled hypertension. If the measured blood pressure of a self-reported patient with no hypertension reached the WHO diagnostic criteria, this patient was classified as unknown hypertension. According to the guideline for primary prevention of stroke in China,[ patients with self-reported hypertension, under age 65, with systolic blood pressure below 140 mm Hg and diastolic blood pressure below 90 mm Hg was considered as well-controlled hypertension, and otherwise, as poorly-controlled hypertension. For any patient aged 65 and above and reporting hypertension, the cutoff point of good systolic blood pressure control is 150 mmHg.[ The mean blood pressure averaged from 3 times of physical examination was taken as the blood pressure of the individual.

The laboratory diagnosis of diabetes was based on the glucose metabolism status issued by WHO in 1999[ and the Glycosylated Hemoglobin Standard by WHO in 2011.[ The individual who met any of the diagnostic criteria for diabetes (fasting blood glucose ≥7.0 mmol/L, postprandial blood glucose ≥11.1 mmol/L or hemoglobin a1c ≥6.5%) or with reported diabetes was considered as diabetic.

Dyslipidemia was defined as the condition of self-report or meeting any of the following criteria: total cholesterol ≥240 mg/dL, triglyceride ≥200 mg/dL, high-density lipoprotein < 40 mg/dL, and low-density lipoprotein > 160 mg/dL.[

Drinking was defined as drinking any kind of alcohol more than once a month. BMI was calculated by kg (weight)/ m2(height). In China, BMI ≥24 kg/m2 and BMI ≥28 kg/m2 are considered as overweight and obesity respectively.[ WHtR was calculated as the waist circumference (cm) divided by the height (cm). WHtR ≥ 0.5 is regarded to be abdominal obesity.[

The definition of heart disease in the development and validation cohorts is different. Heart disease includes heart attack, coronary heart disease, angina, congestive heart failure, or other heart problems in the development cohort, but only includes myocardial infarction in the validation cohort.

The outcome indicator is the 2-year new-set intracranial hemorrhage or ischemic stroke during the 2-year follow-up period. In both cohorts, the outcome was clarified by face-to-face questionnaire, by asking “Have you been diagnosed with stroke by a doctor in the last 2 years?”.

Statistics methods

T test and Chi-squared test was applied in the comparison of baseline characteristics and outcome between Charls and CHNS. Logistic regression was used to select the model variables. Heteroscedasticity is often found in generalized linear regression, so the heteroscedasticity robust standard error method was used for model correction.[ In consideration of the confounding effect and convenience, a total of 12 candidate factors were included to develop the model by logistic regression, while the variables with statistically significant coefficient were included in the prediction system. The total risk score of each individual was calculated first by multiplying the logistic regression coefficient by 10 as the risk score for each predictor and then by summing the risk scores.

The reliability of the scores was examined by the Hosmer-Lemeshow goodness of fit test. Receiver operating characteristic curve (ROC) was plotted and the area under the curve (AUC) was calculated to evaluate the validity of scores. The Youden index was adopted to calculate the cutoff point of high risk, which was used to divide the high-risk group and the low-risk group. The sensitivity, specificity and accuracy of the cutoff point were calculated.

Internal validation of the model was performed with 10-fold cross-validation to calculate the cross-validation AUC (cvAUC), with 1995 as the random seed. The 95% confidence interval (CI) of cvAUC was computed by the Bootstrap method with 1000 times. The mean cvAUC of 10 times of 10-fold cross-validation was also calculated (random seed: 1986–1995). External validation of the model was conducted in the validation cohort. In addition to cross-validation, the sensitivity, specificity and accuracy of the cutoff point were also verified. Fisher exact test was applied to compare the difference in incidence density between low-risk group and high-risk group in both cohorts.

Sensitivity analysis was applied to compare the AUC and cvAUC when all 12 risk factors were taken as the predictors (Situation A) and when the excluded covariates were taken as the predictors (Situation B).

Stata (Stata13 Corp, College Station, TX) was used for statistics analysis and the level of 2-sided significance was < 0.05.

Results

Baseline characteristics and incidence

Table 1 showed that the proportions of males, education level at high school or above, heart diseases, smokers, and WHtR ≥0.5 were all higher in Charls than those in CHNS, while the proportions of dyslipidemia, drinkers, and overweight or obesity were all less than those in CHNS. A total of 137 and 42 patients suffered stroke during the 2-year follow-up in Charls and CHNS, with incidence density of 358.207/ 100,000 person-year and 350.701/100000 person-year respectively.

Table 1

Baseline characteristics and outcome of the study populations.

Risk factor selection and model construction

The development cohort indicated that heart disease, hypertension status, age, diabetes, and smoking were risk factors for stroke (Table 2). Compared with those without hypertension, hypertension patient of any state were more likely to suffer stroke. Compared with those aged 45 to 55 years old, the older ones suffered higher risk of stroke. Regression coefficients and risk score for each factor were showed in Table 2.

Table 2

Multivariable logistic model and risk score associated with each risk factor for 2-year risk of new-onset stroke for participants in the China Health And Retirement Longitudinal Survey study.

Model prediction effect and internal validation

The risk of stroke got higher when there was a higher total risk score, with the regression coefficient being 0.104 (Odds Ratio: 1.109, 95%CI: 1.084, 1.134). In the Hosmer-Lemeshow goodness-of-fit test for the prediction model, P value was .178. AUC was 0.707 (95%CI: 0.664, 0.750) (Fig. 2).

Figure 2

Cross-validation ROC curves of 2-year new-onset stroke risk prediction model in the China Health and Retirement Longitudinal Survey study and the China Health and Nutrition Survey study: cross-validation area under receiver operating characteristic curve stands for cross-validation AUC.

The cvAUC was 0.710(95%CI: 0.650, 0.736) (Fig. 2). The cvAUC of 10 times 10-fold cross-validation ranged from 0.703 to 0.714, with the average of 0.707. The ROC line and cross-validation ROC line in Charls were close.

When the Youden index was the largest, the total risk score was 9.485, which was taken as the cutoff point of high-risk. The high-risk group involved 7581 individuals, 106 of whom suffered stroke in 2 years; the low-risk group involved 8976 individuals, 31 of whom suffered stroke. The incidence was shown in Figure 3. The sensitivity, specificity and accuracy of this cutoff point were 0.774 (95%CI: 0.704, 0.844), 0.545 (95%CI: 0.537, 0.552), and 0.547 (95%CI: 0.539, 0.554), respectively.

Figure 3

Different incidence rate between low risk group and high risk group in the China Health and Retirement Longitudinal Survey and he China Health and Nutrition Survey: ∗∗∗indicates P < .001 (Fisher exact test between low risk group and high risk group).

For sensitivity analysis, no risk factor was excluded, so all 12 risk factors were taken as the predictors. In the development cohort, the Hosmer-Lemeshow goodness-of-fit test showed P value was .337. AUC for the new total risk score was 0.720, and cvAUC was 0.724. The sensitivity, specificity and accuracy of new cutoff point were 0.745, 0.595, and 0.596, respectively.

External validation

In the validation cohort, the regression coefficient of total risk score was 0.151, and OR was 1.163 (95%CI: 1.121, 1.206). The Hosmer-Lemeshow goodness-of-fit test indicated the model fitted well (P = .260), and AUC was 0.800 (95%CI: 0.744, 0.856) (Fig. 2).

The cvAUC was 0.811 (95%CI: 0.714, 0.847) (Fig. 2). The cvAUC of 10-fold cross-validation for 10 times was between 0.789 and 0.822, with an average of 0.802. The cross-validation ROC line in CHNS was higher than the ROC line, and these 2 lines in CHNS were not as close as in Charls.

In the validation cohort, the high-risk group involved 2,199 individuals, 36 of whom suffered stroke; the low-risk group involved 2866 patients, 6 of whom suffered stroke in 2 years. Figure 3 shows the incidence of high-risk and low-risk groups. The sensitivity, specificity and accuracy of this cutoff point were 0.857 (95%CI:0.751, 0.963), 0.569 (95%CI:0.556, 0.583), and 0.572 (95%CI:0.558, 0.585) respectively.

Sensitivity analysis

In the development cohort, in Situation A, when all 12 risk factors were taken as the predictors, the AUC for the new total risk score was 0.720, and cvAUC was 0.724. In Situation B, when the excluded covariates, as education, marital status, dyslipidemia, sleep duration, alcohol consumption, BMI, and WHtR were taken as the predictors, the AUC for covariates was 0.610, and cvAUC for covariates was 0.614.

We did not include sleep duration into sensitivity analysis because there were too many missing in sleep duration data in CHNS cohort. The AUC was 0.819 and cvAUC was 0.815 in Situation A and the AUC for covariates and cvAUC for covariates were 0.614 and 0.599 in Situation B.

Discussion

In this study, a 2-year risk prediction model for new-onset stroke in Chinese people over 45 years old has been developed. The model showed high predictive value and discrimination ability in external validation and is believed to be applicable in health management to identify individuals with higher risk of 2-year new-onset stroke in Chinese people over 45 years old.

The prediction tool fitted well, and a higher risk score indicated a higher risk of the disease. In the development cohort, stroke risk had a tendency of increasing 10.9% when the total risk score increased by 1 point, and the trend was also observed in the validation cohort. The model showed good prediction performance in both development cohort and validation cohort. The cvAUC was 0.710 in development cohort, and 0.811in validation cohort. The average AUC in this study was higher than that in FSRS among Chinese and in modified FSRS, indicating that this model might perform better in predicting risk among Chinese. The sensitivity of high-risk cutoff point exceeded 0.75 in both development and validation cohort, while the specificity did not reach 0.6 in either cohort, and only reached 0.612 among the female in the CHNS cohort. Despite the relatively low specificity, the incidence among the screened high-risk groups was far higher than that in the low-risk group, indicating the model could be well applied in screening high-risk groups. Though there were some differences in prediction performance in 2 cohorts, this model was believed to distinguish the high-risk group of new-onset stroke in Chinese population.

The classification of hypertension status in this study is quite different from other stroke prediction tools. In consideration of the difference in the risk of stroke among people with different awareness of hypertension and blood pressure control,[ we divided hypertension status into 4 types of no, unknown, well-controlled, and poorly-controlled hypertension, so the model could be used to predict the risk of stroke in different states of hypertensive patients. The risks of stroke in patients with unknown, well-controlled and poorly-controlled hypertension were 2.202, 2.286, and 2.642 times higher than those without hypertension, respectively. A meta-analysis has argued that antihypertensive therapy could effectively reduce the risk of stroke in the elderly.[ Those patients with undiagnosed hypertension, unreasonable treatment and poor blood pressure control were at much higher risk of stroke than those with normal blood pressure.[ Every 10 mm Hg reduction in systolic blood pressure significantly reduced the risk of stroke, with relative risk being 0.73.[ Other factors including diabetes, heart diseases, age, and smoking are all clearly correlated with stroke.[

The sensitivity and AUC of the model in CHNS were higher, but the ROC line and cross-validation ROC line were closer in Charls, and the gap between AUC and cvAUC was larger in CHNS, which might be correlated with some differences between the 2 cohorts. The heart diseases in Charls involved a number of different types of heart problems, while only heart attack/myocardial infarction were included in CHNS. This suggested myocardial infarction had greater impact on the new-onset stroke than other heart diseases. As reported, the risk of Ischemic stroke increases due to residual ischemic risk within 4 years after the onset of myocardial infarction.[ In addition, the samples in Charls were more representative due to the coverage of 28 provinces of China, while the samples in CHNS only coverd 9 provinces, and the sample size of Charls was larger than CHNS's, which might lead to the different prediction performance in 2 cohorts. China is vast and the morbidity of stroke is higher in north China and relatively lower in south China.[ As a consequence, the multilevel model may be applied to develop a disease risk prediction model in the future.

Logistic regression rather than Cox proportional hazard regression was used here because the follow-up interval was 2 years and the minimum time gap was year (in the questionnaire “When (in which year) was the stroke first diagnosed or known by yourself?”). Some individuals did not know the new-onset age or the disease-free time.

Only factors with statistically significant coefficient were selected to construct the prediction model, which contributed to the simplified and convenient application of the model. Sensitivity analysis shows the AUC and cvAUC were only enlarged by 0.013 and 0.014 in Charls when taking all candidate factors in the model construction. In CHNS, the increase of AUC and cvAUC were only enlarged by and 0.019 and 0.004, respectively. Although the prediction performance improved, the improvement is fairly limited in the consideration of the inconvenience in application with 7 additional factors. The AUC and cvAUC for covariates were lower than the AUC and cvAUC of this prediction tool in both Charls and CHNS, which meant these covariates poorly predicted stroke.

Although the model has good prediction and discrimination performances, there are still some limitations. For example, indispensable risk factors such as family history of stroke, precise diagnosis of heart disease like atrial fibrillation and antiplatelet drugs were not included owing to the limitations of Charls and CHNS data. In addition, only 50% of the respondents in the Charls survey answered the items of physical exercise, so this factor was ignored. The diagnosis of stroke was based on self-report in both Charls and CHNS cohorts and both cohorts failed to distinguish intracranial hemorrhage or Ischemic stroke, which might lead to potential bias. Due to lack of onset time, Cox proportional hazard regression cannot be used for model development.

The novel model is available to all people over 45 years of age in China and divides the hypertensive population into 4 categories and considers the impact of blood pressure control of hypertensive patients on the stroke risk, which is rarely involved in previous risk prediction models. This model can predict new-onset 2-year stroke risk, so it is expected to be beneficial for people to be aware of their risk, improving living habits or seek medical and health services so as to enhance the primary prevention of stroke.

Acknowledgments

We thank Dr Fengdi Cao (DDs & PhD) for his sincerely help to this study and invaluable guidance to author Qiang Yao. We also thank the investigators, staff, and participants of the Charls study and CHNS study for their valuable contributions. A full list of Charls and CHNS study investigators/institutions can be found at http://charls.pku.edu.cn/ and https://www.cpc.unc.edu/projects/china/.

Author contributions

Conceptualization: Qiang Yao.

Data curation: Qiang Yao.

Formal analysis: Qiang Yao, Yanling Yang.

Funding acquisition: Cairong Zhu.

Methodology: Yawen Li, Lu Zhang, Chenyao Wu.

Project administration: Cairong Zhu.

Resources: Jing Zhang, Ke Yan, Qianwen Zheng.

Software: Cairong Zhu.

Supervision: Muke Zhou, Cairong Zhu.

Validation: Qiang Yao.

Visualization: Qiang Yao, Jing Zhang, Ke Yan.

Writing – original draft: Qiang Yao.

Writing – review & editing: Jing Zhang, Ke Yan, Qianwen Zheng.