Development and Internal Validation of a Model Predicting Premature Cardiovascular Disease Among Women With Hypertensive Disorders of Pregnancy: A Population-Based Study in Quebec, Canada.

Background Hypertensive disorders of pregnancy (HDP) are associated with an increased risk of premature cardiovascular disease (CVD), but existing cardiovascular prediction models do not adequately capture risks in young women. We developed a model to predict the 10-year risk of premature CVD and mortality among women who have HDP. Methods and Results Using a population-based cohort of women with HDP who delivered between April 1989 and March 2017 in Quebec, Canada, we developed a 10-year CVD risk model using Cox proportional hazards regression. Women aged 18 to 45 years were followed from their first HDP-complicated delivery until March 2018. We assessed performance of the model based on discrimination, calibration, and risk stratification ability. Internal validity was assessed using the bootstrap method. The cohort included 95 537 women who contributed 1 401 084 person-years follow-up. In total, 4024 (4.2%) of women were hospitalized for CVD, of which 1585 events (1.6%) occurred within 10 years of follow-up. The final model had modest discriminatory performance (area under the receiver operating characteristic curve, 0.66; 95% CI, 0.65-0.67) and good calibration with slope of 0.95 and intercept of -0.19. There was moderate classification accuracy (likelihood ratio+: 5.90; 95% CI, 5.01-6.95) in the highest-risk group upon risk stratification. Conclusions Overall, our model had modest performance in predicting the 10-year risk of premature CVD for women with HDP. We recommend the addition of clinical variables, and external validation, before consideration for clinical use.

hypertensive disorders of pregnancy

Clinical Perspective

What Is New?

We developed a 10‐year cardiovascular risk prediction model for women with hypertension in pregnancy.

Our model, developed using large administrative data, performed moderately (likelihood ratio+ >5 in the highest‐risk group).

What Are the Clinical Implications?

Our model has the potential to identify young women at the highest risk of cardiovascular disease.

However, the model must be improved and externally validated before clinical use.

Early identification of high‐risk women may aid in cardiovascular disease prevention through lifestyle and pharmaceutical interventions.

Cardiovascular disease (CVD) is a leading cause of morbidity and mortality among women worldwide, , and more women die of CVD than men at all ages. Improving CVD prediction is a priority since the majority of CVD is preventable. Several clinical prediction models are routinely used for predicting CVD risk in the general population, such as the Framingham risk score model, QRISK score, and the Adult Treatment Panel model. However, most of these models underestimate the risk of CVD in young women. , Professional bodies such as the American Heart Association and Society of Obstetricians and Gynaecologists of Canada have highlighted the need to identify female‐specific factors that can aid in predicting CVD for women.

Pregnancy involves substantial cardiovascular changes, and a woman's response to pregnancy can be indicative of future cardiovascular health. Numerous studies have consistently reported a 2‐fold higher risk of developing premature CVD among women who experience hypertensive disorders of pregnancy (HDP). , , , The postpartum period is now widely regarded as a possible window of opportunity for CVD prevention in women. , , However, there is a lack of guidance on risk factor management in this population.

There have been attempts to improve the prediction of CVD in women by adding pregnancy complications, especially HDP, as potential variables to existing predictive models. , , , , , , , However, these studies failed to improve the performance of models. For example, a study by Markovitz et al reported an improvement of only 0.004 in the c‐index upon the addition of pregnancy complications to NORRISK 2, an existing prediction model. These studies used a population of older women (>40 years of age), possibly diluting the importance of pregnancy factors and limiting generalizability to younger women. Since the absolute risk of CVD is low in young women, merely adding pregnancy complications to an existing model is likely insufficient to permit discrimination between women with and without CVD.

In response to these gaps in clinical prediction, we developed and internally validated a 10‐year model to predict premature CVD and all‐cause mortality before 60 years of age among a population of young women with HDP, a target population with an established higher baseline risk of CVD. This time frame was chosen based on previous recommendation as a clinically important period for CVD prevention. We hypothesized that easily measured clinical and pregnancy/reproductive factors available in administrative data would be sufficient to predict CVD in this high‐risk population.

Methods

Data and Study Population

Requests to access the data set from qualified researchers trained in human subject confidentiality protocols may be sent to Institut de la statistique du Québec (https://www.stat.gouv.qc.ca/resea​rch/#/accueil). We used a longitudinal pregnancy cohort, constructed from the Maintenance and Use of Data for the Study of Hospital Clientele registry, consisting of all hospital‐based deliveries in Quebec, Canada from April 1989 until March 2016. This cohort includes >99% of all deliveries in Quebec, a province that comprises more than one quarter of the Canadian population.

We selected women aged 18 to 45 years who had HDP in at least 1 of their pregnancies during the study period. HDP was defined as preexisting hypertension in pregnancy, gestational hypertension, preeclampsia/eclampsia/hemolysis, elevated liver enzymes, and low platelet count syndrome, superimposed pre‐eclampsia, or unspecified hypertension during pregnancy. , HDP was identified using diagnostic codes from the International Classification of Diseases, Ninth and Tenth Revisions (ICD‐9, ICD‐10). A validation study reported high sensitivity and specificity (>80% each) for identifying HDP using ICD codes.

We excluded women with preexisting CVD before study entry (n=659), and those who died in their first affected pregnancy (n=46) (Figure 1).

Figure 1

Development of study cohort.

 

Outcomes

The primary outcome was a composite of premature (before the age of 60 years) CVD events and/or premature all‐cause in‐hospital mortality. CVD events included hospitalization for heart disease (heart failure, ischemic heart disease, myocardial infarction, angina, cardiac arrest, inflammatory heart disease, conduction defects, valve disorders, cardiomyopathy, pulmonary heart disease), cerebrovascular disease (ischemic, hemorrhagic), atherosclerotic disease, aortic aneurysm or dissection, other aneurysm, arterial embolism, cardiovascular interventions (coronary angioplasty, coronary artery bypass graft, pacemaker, valve surgery, cardiac transplant), and admission to a coronary care unit. CVD diagnoses and procedures were identified using ICD‐9 and ‐10, Canadian Classification of Diagnostic, Therapeutic, and Surgical Procedures, and Canadian Classification of Health Intervention codes. These codes have been previously used for the identification of CVD outcomes.

Follow‐up

Women were followed from the delivery hospitalization for their first HDP‐complicated pregnancy until 10 years thereafter, until they reached 60 years of age, or the end of the study period (March 31, 2018),whichever came first, to identify subsequent hospitalization for CVD or death.

Sample Size

To avoid model overfitting, we required at least 20 events per degree of freedom. Based on this and the total of 4024 events (1585 within 10 years of follow‐up), we had sufficient sample size to consider up to 75 degrees of freedom for candidate predictors.

Candidate Predictors, Variable Selection, and Coding

Demographic, clinical, and reproductive factors known to be associated with an increased risk of CVD were considered candidate variables. These included maternal age, smoking, obesity, socioeconomic deprivation, and preexisting medical conditions such as diabetes mellitus; pregnancy factors such as parity, multifetal pregnancy; and complications during pregnancy such as type of HDP, gestational diabetes mellitus, severe maternal morbidity, stillbirth, preterm delivery, low birth weight, and admission into neonatal intensive care unit and adult intensive care unit. Socioeconomic deprivation was measured using a composite score of neighborhood income, education, and employment. Severe maternal morbidity was defined according to the published Canadian Perinatal System Surveillance criteria. All candidate predictors were measured at the time of the index delivery.

Variable selection was done in different stages to include only candidate predictors that are routinely and reliably collected. Variables that were clinically related but with very low incidence were combined. For example, tobacco, alcohol, and illicit drug use were combined as substance use, while any previous pregnancy complications (previous stillbirth, preterm delivery, low birthweight, admission to neonatal intensive care unit, neonatal death, or severe maternal morbidity) were combined as a composite variable for previous history of obstetric complications. Variables were assessed for collinearity, and where collinear (r>0.5), the most clinically relevant or commonly reported variable was selected. Previous history of obstetric complications was combined with parity coded in 3 levels as these variables were collinear; categories were previous obstetric complication (multiparous), no previous obstetric complication (multiparous), and no previous obstetric complication (primiparous).

The final candidate predictors were the following: prenatal substance use, obesity, socioeconomic deprivation, diabetes mellitus (preexisting diabetes mellitus/gestational diabetes mellitus/no diabetes mellitus), HDP subtype (preexisting hypertension, gestational hypertension, preeclampsia/hemolysis, elevated liver enzymes, and low platelet count syndrome/eclampsia, superimposed preeclampsia, or unspecified hypertension), cesarean delivery, severe maternal morbidity, stillbirth, neonatal death, admission to neonatal intensive care unit; and previous pregnancy complications (yes/multiparous, no/multiparous, primiparous). Continuous candidate predictor variables included maternal age, gestational age at delivery, and length of stay in intensive care unit, modeled using restricted cubic splines with 3 clinically relevant knot locations.

Possible interactions between variables were assessed and included if statistically significant at α=0.10. The final variable selection was done using Least Absolute Selection and Shrinkage Operator regression. , A Cox proportional hazards regression model was fit using the final variables to predict the onset of premature cardiovascular outcomes and all‐cause mortality.

Statistical Analysis

Cox proportional hazards regression models were developed according to the steps outlined by Steyerberg and Vergouwe, and Harrell, to predict outcomes. We reported the model development process based on the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis guidelines (Table S1).

Model Performance and Internal Validation

The 10‐year risk of developing the outcome was estimated using the final selected predictors, and the prediction performance of the model was assessed based on discriminatory, calibration, and risk stratification abilities. Discrimination measures the ability of the model to accurately distinguish between women who developed an adverse outcome and those who did not. This was measured by the c‐statistic, which is analogous to the area under the receiver operating characteristic curve for a binary outcome but taking censoring into account. A c‐statistic of 0.5 was considered not useful, 0.5 to 0.6 poor, 0.6 to <0.7 modest, and ≥0.7 good. Calibration measures the agreement between the observed and predicted risks. Calibration performance was examined by plotting the mean observed events versus the mean predicted risks by decile of the predicted risk. Calibration slopes were interpreted as noninformative (slope ≤0.5), poor (0.5 < slope ≤0.7), or good (slope >0.7).

The model's ability to stratify the population into low‐ and high‐risk categories was examined using a risk classification table. We divided the population into the top 2% of the population with the highest calculated risk; the next >2% to 5%, >5% to 10%, and the rest of the population were grouped according to their calculated risk of disease occurrence. These cut‐offs corresponded to the distribution of events in the population. Likelihood ratios (LR) were computed for each group using Deeks and Altman`s method for multiple categories, to assess classification accuracy within each group. For clinical use, positive LRs of >5 or >10 were respectively interpreted as moderate or good “rule‐in” tests (a value reflecting a high probability of disease occurrence). For negative LRs, <0.2 was considered a moderate rule‐out test (ie, high probability that the disease will not occur), and <0.1 was considered a good rule‐out test.

The model was assessed for internal validity using the Efron bootstrap method with 200 iterations and the over‐optimism (ie, the degree to which a model is overfit to the sample data) was reported.

Secondary Analyses

For secondary analyses, we predicted the 15‐year risk of developing the outcome using the same final selected predictors. Similar to the primary analyses, women were followed from their first HDP‐complicated pregnancy until 15 years thereafter, 60 years of age, or the study period end, whichever came first, to identify study outcomes. The model was re‐assessed based on discriminatory performance, as described above.

Sensitivity Analyses

Recognizing that pregnancy complications could occur up to 42 days after delivery and an outcome in this time window could be related to the pregnancy itself, we ran the models after excluding women with outcomes within this time period. We also ran the primary model excluding women with preexisting hypertension and diabetes mellitus.

All analyses were conducted using R version 3.5.1 (The R Project for Statistical Computing).

Ethics review for this study was waived by the institutional review board of the University of Montreal Hospital Centre, because the data were de‐identified and informed consent was not required.

Results

Participants

Of the 1 229 263 women in the original cohort, HDP occurred in 7.9% of the population, with the majority (81.6%) occurring in the first pregnancy. Steps in the selection of the final cohort are shown in Figure 1. The final cohort consisted of 95 537 women between the ages of 18 and 45 years who contributed a total length of follow‐up of 1 401 084 person‐years. In total, 4024 (4.2%) women experienced premature CVD or death, of which 1585 (1.6%) and 2435 (2.6%) occurred within 10 and 15 years of follow‐up, respectively.

Table 1 (and Table S2) show the characteristics of women in the total population according to the occurrence of CVD/mortality. Women who developed these outcomes were more likely to be older, socioeconomically deprived, use substances (alcohol/tobacco/illicit drugs), have preexisting diabetes mellitus or hypertension, and have preterm delivery, at the time of their complicated pregnancy compared with women who did not develop an outcome.

Table 1

Characteristics of Women With Hypertensive Disorders of Pregnancy According to Outcome (Premature Cardiovascular Events or Death), Quebec, 1989 to 2018, N=95 537

	No. Women (%)
	No Outcome (N=4024)	Outcome (N=91 513)
Maternal age, y
<20	96 (2.4)	2890 (3.2)
20–24	590 (14.6)	17 546 (19.2)
25–29	1222 (30.4)	31 438 (34.4)
30–34	1165 (29.0)	24 695 (27.0)
35–39	767 (19.0)	11 916 (13.0)
≥40	184 (4.6)	3028 (3.2)
Multiple pregnancy
Yes	104 (2.6)	3228 (3.5)
No	3920 (97.4)	88 285 (96.5)
Time period at delivery
1989–1995	2093 (52.0)	22 448 (24.5)
1996–2002	1097 (27.3)	20 219 (22.1)
2003–2009	596 (14.8)	23 610 (25.8)
2010–2016	237 (5.9)	25 230 (27.6)
Socioeconomic disadvantage
Yes	964 (24.0)	18 694 (20.4)
No	2722 (67.6)	67 731 (74.0)
Unknown	338 (8.4)	5088 (5.6)
Substance use*
Yes	116 (2.9)	1397 (1.5)
No	3908 (97.1)	90 116 (98.5)
Morbid obesity
Yes	198 (4.9)	4047 (4.4)
No	3826 (95.1)	87 466 (95.6)
Preexisting diabetes mellitus
Yes	375 (9.3)	2569 (2.8)
No	3649 (90.7)	88 944 (97.2)
Primiparity at complicated pregnancy
Yes	3310 (82.3)	74 688 (81.6)
No	714 (17.7)	16 825 (18.4)
Hypertensive disorder of pregnancy subgroups
Preexisting hypertension	688 (17.1)	8583 (9.4)
Gestational hypertension	715 (17.7)	29 300 (32.0)
Preeclampsia/HELLP syndrome	1766 (43.9)	38 208 (41.8)
Superimposed preeclampsia	143 (3.6)	1813 (2.0)
Unspecified hypertension	712 (17.7)	13 609 (14.8)
Gestational diabetes mellitus
Yes	420 (10.4)	8387 (9.2)
No	3604 (89.6)	83 126 (90.8)
Cesarean section
Yes	1541 (38.3)	30 104 (32.9)
No	2483 (61.7)	61 409 (67.1)
Severe maternal morbidity
Yes	799 (19.9)	18 638 (20.4)
No	3225 (80.1)	72 875 (79.6)
Preterm delivery (GA <37 wk)
Yes	952 (23.7)	15 733 (17.2)
No	3072 (76.3)	75 780 (82.8)
Stillbirth
Yes	50 (1.2)	532 (0.6)
No	3974 (98.8)	90 981 (99.4)
Low birth weight (<2500 g)
Yes	910 (22.6)	15 301 (16.7)
No	3112 (77.4)	76 204 (83.3)
Admission to NICU
Yes	260 (6.5)	7266 (8.0)
No	3728 (93.5)	83 853 (92.0)
Neonatal death
Yes	31 (0.8)	290 (0.3)
No	3957 (99.2)	90 829 (99.7)
Previous pregnancy complication a , b (among multiparous women)
Yes	163 (4.1)	3298 (3.6)
No	551 (13.7)	16 825 (14.8)

GA indicates gestational age; HELLP, hemolysis, elevated liver enzymes, and low platelet count; and NICU, neonatal intensive care unit.

Substance use includes tobacco, alcohol, and illicit drug use.

Previous complications include any previous stillbirth, preterm delivery, low birth weight, admission to NICU, neonatal death, or severe maternal morbidity.

The types and incidence rates of CVD/mortality are presented in Table 2. The incidence of the outcome was 28.72 per 10 000 person‐years and the most frequent outcomes were ischemic heart disease (9.49/10 000 person‐years), conduction disorders/arrhythmia (8.84/10 000 person‐years), and atherosclerotic disease (7.59/10 000 person‐years).

Table 2

Incidence Rates of Hospitalization for Different Cardiovascular Diseases for Women With Hypertensive Disorders of Pregnancy, Quebec, 1989 to 2018

	No. Cardiovascular Hospitalizations	Incidence Rate Per 10 000 Person‐Years (95% CI)
Heart failure	471	3.36 (3.07–3.68)
Ischemic heart disease	1329	9.49 (8.99–10.01)
Myocardial infarction	754	5.38 (5.01–5.78)
Angina	417	2.98 (2.70–3.28)
Cardiac arrest	105	0.75 (0.62–0.81)
Inflammatory heart disease	105	0.75 (0.62–0.81)
Conduction defects	1239	8.84 (8.36–9.35)
Valve disorders	386	2.76 (2.49–3.04)
Cardiomyopathy	260	1.86 (1.64–2.10)
Pulmonary heart disease	165	1.18 (1.01–1.37)
Ischemic cerebrovascular disease	567	4.05 (3.72–4.39)
Cerebrovascular hemorrhage	266	1.90 (1.68–2.14)
Atherosclerotic disease	1063	7.59 (7.14–8.06)
Aortic aneurysm or dissection	44	0.31 (0.23–0.42)
Other aneurysm	158	1.13 (0.97–1.32)
Coronary angioplasty	532	3.80 (3.49–4.13)
Coronary artery bypass graft	149	1.06 (0.91–1.25)
Pacemaker	142	1.01 (0.86–1.20)
Valve surgery	118	0.84 (0.70–1.01)
Cardiac transplant	<5	0.03 (0.01–0.08)
Coronary care unit admission	621	4.43 (4.10–4.80)
Arterial embolism	124	0.89 (0.74–1.06)
Death	626	4.47 (4.13–4.83)
Outcome (cardiovascular disease or death)	4024	28.72 (27.85–29.62)

N=95 537.

Model Development and Performance

The final prediction model contained 10 variables with 15 degrees of freedom and the model equation is presented in Table 3. The model included maternal age, socioeconomic deprivation, substance use, gestational age at delivery, obesity, diabetes mellitus, cesarean delivery, previous complications, type of HDP, and number of days in the intensive care unit. The model had moderate discriminatory performance with a c‐statistic of 0.66 (95% CI, 0.65–0.67), albeit minimal overfitting (optimism of 0.008) upon internal validation (Figure 2).

Table 3

Final Model With Coefficients for Prediction of Cardiovascular Risk Among Women With Hypertensive Disorders of Pregnancy, Quebec, 1989 to 2018

Variables	Coefficients
Maternal age*
Socioeconomically deprived
No	Reference
Yes	0.252
Substance use
No	Reference
Yes	0.694
Gestational age at delivery*
Obesity
No	Reference
Yes	0.260
Diabetes mellitus
None	Reference
GDM	0.027
Preexisting diabetes mellitus	0.590
Cesarean delivery
No	Reference
Yes	0.031
Previous complications
No, multiparous	Reference
Yes, multiparous	−0.047
Primiparous	−0.206
Type of HDP
Preexisting hypertension	Reference
Gestational hypertension	−0.538
Preeclampsia/HELLP	−0.567
Preeclampsia superimposed on preexisting hypertension	0.022
Unspecified hypertension	−0.424
NICU admission days*
GDM*cesarean section	−0.004
Preexisting diabetes mellitus*cesarean section	0.446

GDM indicates gestational diabetes mellitus; HELLP, hemolysis, elevated liver enzymes, and low platelet count; HDP, hypertensive disorders of pregnancy; and NICU, neonatal intensive care unit.

Variables fit as splines, therefore no coefficients.

Figure 2

Area under the receiver operating characteristic curve (AUROC) showing model discrimination (0.66 95% CI: 0.65‐0.66) for 10‐year prediction of cardiovascular disease or death.

 

The calibration slope was 0.95 and the intercept was −0.19, indicating a good agreement between the predicted and observed rates of outcomes within each decile of calculated predicted probabilities (Figure 3).

Figure 3

Calibration plot of observed vs predicted 10‐year risk of outcome using deciles of predicted probability.

ROC indicates receiver operating characteristic.

Risk stratification and classification accuracy are presented in Table 4. In the 2% of the population with the highest risk, the calculated risk probability was ≥0.07%. In this highest‐risk category consisting of 1601 (1.7%) women, the model was able to moderately rule‐in the likelihood of outcome occurrence with LR+ of 5.90 (5.01–6.95), showing good risk classification accuracy. However, among the women in this high‐risk group, only 9.1% had the outcome, indicating poor risk stratification.

Table 4

Risk Classification and Stratification Table (N=95 537)

Calculated Risk Probability	No. Women in Risk Group (%)	No. Women With Outcomes (%)	No. Women Without Outcomes (%)	Likelihood Ratio (95% CI)
<0.035	86 546 (90.6)	1176 (1.4)	85 370 (98.6)	0.82 (0.73–0.92)
≥0.035 to <0.045	4027 (4.2)	126 (3.1)	3901 (96.9)	1.91 (1.61–2.27)
≥0.045 to <0.07	3363 (3.5)	138 (4.1)	3225 (95.9)	2.54 (2.15–3.00)
≥0.07	1601 (1.7)	145 (9.1)	1456 (90.9)	5.90 (5.01–6.95)
Total	95 537	4024	91 513

The 15‐year prediction model showed similar discriminatory performance with a c‐statistic of 0.65 (95% CI, 0.63–0.66), although the calibration performance decreased with a slope of 0.89 and intercept of −0.38 for calibration.

Sensitivity Analysis

After excluding women with outcomes occurring within 42 days of delivery (n=108), the total number of women in the analysis was 95 429, with 1477 women having premature CVD/mortality within 10 years of follow‐up. There was a slight decrease in the discriminatory performance of the model: area under the receiver operating characteristic curve of 0.64 (95% CI, 0.63–0.66) for 10‐year risk prediction.

Exclusion of women with preexisting hypertension and diabetes mellitus (n=13 322) resulted in a population of 82 215 women, of which 1133 had an outcome within 10 years of follow‐up. This resulted in decreased discriminatory performance of the model with area under the receiver operating characteristic curve of 0.62 (95% CI, 0.61–0.64) for 10‐year risk prediction.

Discussion

Summary

In this study, we assessed whether the risk of premature CVD and mortality for women with a history of HDP can be predicted using easily measured factors at the time of delivery. Overall, our model had modest discrimination and calibration, and marginal overfitting, for the 10‐year prediction of study outcomes among high‐risk women with HDP. The resulting LR was sufficient to accurately rule‐in adverse outcomes for the highest‐risk category. However, only 3.6% of the total outcomes were captured in this group. Although our model showed strong potential for predicting CVD among women with HDP, its performance, particularly discriminatory and risk stratification abilities, needs to be improved before adoption in clinical practice.

Comparison With Existing Literature

To our knowledge, this is the first study to develop a prediction model using a population‐based cohort of only high‐risk young women with HDP. In studies that added HDP to preexisting models, the change in discriminatory performance ranged from no increase to only 0.003. The population in these studies differed from ours. In the study by Markovitz et al, women below the age of 40 years at the time of predictor measurements were excluded, while the studies by Timpka et al and Stuart et al assessed models in a combined cohort of women with and without pregnancy complications. To be clinically applicable, a model has to be developed and tested in a sample of women with characteristics similar to those of the population of interest. Therefore, our model development approach better addresses CVD prediction among young women and enhances the understanding of risk assessment and management in the postpartum period for women at high risk of CVD.

Strengths

Our study has important methodological strengths that can help guide the further refinement of sex‐specific CVD prediction efforts. First, we developed the model using a large population‐based cohort of women representative of Canadians. This allowed for sufficient sample size and power for model development. Quebec is made up of diverse ethnic populations, thus increasing the generalizability of our findings. Secondly, we preselected variables based on clinical knowledge that could be associated with CVD and final variable selections were based on techniques recommended by methodological experts (data reduction and Least Absolute Selection and Shrinkage Operator), rather than the use of forward and backward selection procedures known to produce biased estimates. We assessed interactions between variables based on clinical importance such as age and type of pregnancy complications. Thirdly, our data had adequate follow‐up, which enabled us to identify women who developed the outcome—an important feature because of the young age at cohort entry. We chose 10 and 15 years for prediction periods because these have been suggested as important time points for interventions to prevent CVD. , Finally, we followed the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis statement to ensure transparency in modeling and the reporting process.

Limitations

A limitation in our study is the reliance on ICD codes for defining predictors, which may not be sensitive for some medical conditions. We tried to minimize this influence by dropping variables known to be inconsistently recorded with ICD codes. However, we cannot ignore the possibility of coding errors and potential effect on model performance.

We were also limited by the number of clinically relevant continuous variables available in the data set; for example, we did not have blood pressure or lipid levels or highly sensitive C‐reactive protein, which are commonly used in cardiovascular clinical prediction models. The inclusion of continuous data would increase heterogeneity between women, allowing for better model discrimination. In addition, we lacked information on ethnicity/race and family history of CVD, which may also add incremental value.

Implications

The resulting LR+ in the highest‐risk group (5.90) suggests that the model would have good clinical utility if risk stratification is improved. If the prediction accuracy of our model is substantially improved (for example, area under the receiver operating characteristic curve ≥0.7 and >50% of total outcome within the highest‐risk group), then the model may be used immediately after external validation, that is, after testing in a different data set. Ideally, the model would be applied immediately after delivery, before hospital discharge, to identify which women are at the highest risk of CVD and who might benefit from targeted screening and risk reduction. The ability to capture these women at delivery hospitalization provides a unique opportunity for positive health interventions, including those focused on physical activity, diet, and smoking cessation, as well as pharmacological approaches, following recommended guidelines.

Although the use of administrative data offers many advantages, it also poses several limitations as discussed above. We acknowledge the possibility that measurements available in administrative health data may not be sufficient to predict CVD in this population of women with HDP. Further studies are required to test whether the addition of paraclinical variables, such as blood pressure, would increase model performance. If such clinical variables would add incremental value to our model, we believe that a useful clinical prediction tool based on easily established variables is a real possibility in postpartum women.

Of note, there was a slight decrease in model discriminatory performance when women with preexisting hypertension and diabetes mellitus were excluded. This suggests that women with preexisting diseases are easier to distinguish from women without preexisting conditions, and supports the notion that disease risk is higher in women with premorbid conditions. Therefore, preexisting conditions are important to include in CVD prediction models.

Conclusions

Our clinical prediction model developed using administrative health data alone can reasonably predict the presence of premature CVD or mortality among women with a history of HDP. However, further model refinement is needed to reliably stratify women into low and high risk of developing CVD. Upon model improvement and external validation, the goal is to ultimately develop a risk calculator for use in clinical practice to guide cardiovascular risk reduction in women with complicated pregnancies.

Sources of Funding

This study was supported by the Canadian Institutes of Health Research Foundation grant (FDN‐143297) and the Heart & Stroke Foundation of Canada (G‐18‐0021776). U. Vivian Ukah is supported by the Fonds de recherche du Québec‐Santé (FRQS) postdoctoral award. Natalie Dayan (Junior Track 1) and Nathalie Auger (Junior Track 2) acknowledge Clinical Research Scholar career awards from FRQS.

Disclosures

None.

Tables S1–S2

Click here for additional data file.