Assessing the validity of QRISK3 in predicting cardiovascular events in systemic lupus erythematosus.

OBJECTIVES: Traditional cardiovascular risk calculators such as the Framingham Risk Score (FRS) have been shown to underestimate risk in patients with SLE. The QRISK3 calculator is unique in including SLE and corticosteroid use as risk factors. This study aims to assess the validity of QRISK3 compared with other cardiovascular risk models in a cohort of patients with SLE in the USA.
METHODS: We studied a prospective cohort of 366 adult patients with SLE without history of any cardiovascular event and followed them for 10 years. We compared the diagnostic performance of QRISK3 with FRS, modified FRS, Atherosclerotic Cardiovascular Disease (ASCVD), and Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE (PREDICTS).
RESULTS: Sixty-four of the 366 patients (17.4%) experienced at least one cardiovascular event during the 10-year follow-up period. Of these patients 45% had a QRISK3 score >10%, whereas 20.5% of patients who did not have an event had a QRISK3 score >10% (p<0.001). The corresponding numbers for FRS, modified FRS, ASCVD and PREDICTS were 11.0% vs 7.2% (p=ns), 40.6% vs 28.0% (p=0.05), 12.2% vs 5.9% (p=ns), and 77% vs 32.1% (p<0.001), respectively. The areas under the receiver operating characteristic curve using QRISK3 >10% and high-risk PREDICTS were both larger than those using ASCVD >10%, FRS >10% and modified FRS >10%.
CONCLUSIONS: Both QRISK3 and PREDICTS demonstrated better performance in predicting risk of cardiovascular disease in this cohort of patients with SLE compared with FRS, modified FRS and ASCVD. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Cardiovascular disease is a significant cause of morbidity and mortality in patients with SLE; however, traditional 10-year cardiovascular risk calculators underestimate risk in these patients.

The 2017 QRISK3 cardiovascular risk calculator is the first to include SLE and corticosteroid use as risk factors.

This is the first study to assess the validity of the QRISK3 calculator in a cohort of patients with SLE in the USA and demonstrates that QRISK3 and PREDICTS (Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE) both outperform traditional cardiovascular risk calculators in this patient population.

The use of a cardiovascular risk prediction model including SLE and corticosteroid use in addition to traditional risk factors appears to improve performance in patients with SLE, although further work is needed to identify a combination of SLE-specific risk factors with even better predictive capacity.

Introduction

It has been well established that patients with SLE have an increased risk of developing cardiovascular disease (CVD). Studies have shown approximately twofold to tenfold increased risk of myocardial infarction (MI) in patients with SLE compared with patients in the general population.1 Notably the greatest relative risk for MI is seen in younger women, with one study reporting over 50-fold increased risk in women with SLE age 35–44 compared with age-matched controls.2 A similar pattern of increased risk has been reported with stroke.1

This increased cardiovascular risk is not explained by the presence of traditional risk factors alone. SLE disease-related and therapy-related factors such as systemic inflammation and corticosteroid use are also thought to contribute to the process of accelerated atherosclerosis.1

Traditional CVD risk calculators such as the Framingham Risk Score (FRS)3 and Atherosclerotic Cardiovascular Disease (ASCVD) calculators4 estimate a patient’s 10-year risk of CVD and have been validated for use in the general population. However, they do not take into account the presence of inflammatory rheumatological diseases such as SLE. Consequently, their utility in this patient population has been questioned. The FRS in particular has been shown to underestimate risk in women with SLE.5

As a result, there has been growing interest in developing more accurate methods of CVD risk stratification in patients with SLE. Urowitz and colleagues6 determined that doubling the FRS improved risk prediction in patients with SLE and termed this formula the modified Framingham Risk Score (mFRS). Our group previously developed the PREDICTS (Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE) score, a panel of four biomarkers (proinflammatory high-density lipoprotein, leptin, homocysteine and soluble tumor necrosis factor-like weak inducer of apoptosis (TWEAK) and two traditional cardiac risk factors (age ≥48 years and history of diabetes mellitus). We found that a high-risk PREDICTS profile, defined as either ≥3 positive biomarkers or ≥1 positive biomarker plus history of diabetes, was associated with 28-fold increased odds of longitudinal presence of carotid plaque and increased progression of intima-media thickness.7 More recently, we found that patients with high PREDICTS had a 3.7-fold increased risk of having a major adverse cardiovascular event (MACE).8

Published in 2007, the original QRISK calculator was developed and validated as a 10-year CVD risk prediction model for the UK primary care population. The newest version of the calculator, QRISK3, was published in 2017 and revised to incorporate several additional risk factors including SLE and corticosteroid use.9 The applicable age range was also extended to 25–84 years. QRISK3 has been shown to enhance CVD risk detection in a cohort of patients with SLE in the UK, but has not yet been studied in patients with SLE in the USA.10 QRISK3 was also examined in relation to subclinical parameters of atherosclerosis in a cohort of patients with SLE in Mexico. The authors found a significant correlation between QRISK3 score and mean carotid intima-media thickness, carotid pulse-wave velocity and distensibility.11

The purpose of this study was to assess the validity of QRISK3 compared with other cardiovascular risk calculators (FRS, mFRS and PREDICTS) in predicting CVD risk in a cohort of patients with SLE in the USA.

Methods

We studied a prospective cohort of 366 adult patients aged 25 and older with SLE without history of a cardiovascular event. Participants were recruited as part of the longitudinal ‘Biomarkers of Atherosclerosis in SLE’ cohort study from the rheumatology practices of the University of California Los Angeles and Cedars-Sinai Medical Center in Los Angeles from February 2004 to January 2019, as previously described.7

Baseline data on demographic factors, clinical diagnoses and clinical values were obtained via chart review and used to calculate QRISK3, ASCVD, FRS, mFRS and PREDICTS scores at the time of study enrolment. The various inputs into the QRISK3, ASCVD, FRS and PREDICTS calculators are listed in table 1. MACE was defined as death from any cause, coronary artery disease, MI, ischaemic stroke, transient ischaemic attack or peripheral arterial events (defined as arterial thrombosis requiring revascularisation). Patients under 25 years of age were excluded because this is the lower age limit for the QRISK3 calculator. ASCVD risk score could only be calculated for 43% (158 of 366 patients) of the cohort because it has a lower age limit of 40 years.

Table 1

Input for cardiovascular risk calculators

QRISK3	Framingham	ASCVD	PREDICTS
Age	Age	Age	piHDL
Sex	Sex	Sex	Leptin
Ethnicity	Smoking status	Race	Homocysteine
Diabetes	On blood pressure treatment	Diabetes	Soluble TWEAK
Smoking status	Total cholesterol	Smoking status	Age ≥48 years
Premature coronary artery disease in first-degree relative	HDL	On blood pressure treatment	Diabetes
CKD (stage 3, 4 or 5)	Systolic blood pressure	Total cholesterol
Atrial fibrillation		LDL
On blood pressure treatment		HDL
Migraines		Systolic blood pressure
Rheumatoid arthritis		Diastolic blood pressure
SLE		On statin
Severe mental illness		On aspirin
On atypical antipsychotic
On regular steroid tablets
Erectile dysfunction
Cholesterol to HDL ratio
Systolic blood pressure
SD of at least two most recent systolic blood pressure readings
Body mass index

ASCVD, Atherosclerotic Cardiovascular Disease; CKD, chronic kidney disease; HDL, high-density lipoprotein; LDL, low-density lipoprotein; piHDL, proinflammatory high-density lipoprotein; PREDICTS, Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE; TWEAK, tumor necrosis factor-like weak inducer of apoptosis.

χ2 test was used for dichotomous variables and Student’s t-test for continuous variables. Receiver operating characteristic (ROC) curves were created using SPSS software (V.25) to evaluate the diagnostic performance of QRISK3, ASCVD, FRS, mFRS and PREDICTS using a threshold of 10-year risk >10% for the first four calculators and ‘high-risk’ categorisation for PREDICTS.

Patient and public involvement

Patients and the public were not directly involved in designing or implementing this research project.

Results

Baseline demographic data are shown in table 2. Of the 366 patients assessed in this study, 64 (17.4%) experienced at least one MACE during the 10-year follow-up period, for a total of 74 MACEs. The types of MACE are shown in table 3.

Table 2

Demographic data

Parameter	Entire cohort(n=366)	ASCVD cohort, age ≥40 years(n=158)
Age (years)	42.5±13.1	49.2±5.1
Sex
Female	355 (97)	154 (97.5)
Male	11 (3)	4 (2.5)
Ethnicity
Caucasian	193 (53)	83 (53)
African-American	57 (16)	26 (16)
Hispanic	65 (18)	25 (16)
Asian	37 (10)	17 (11)
Mixed/other	14 (4)	7 (4)
Hypertension	127 (35)	58 (37)
Diabetes mellitus	19 (5)	12 (8)
Current smoker	30 (8)	11 (7)
Body mass index (kg/m2)	26.3±6.5	27.0±6.6
Disease duration (years)	12.0±8.9	14.4+9.2
Total cholesterol (mg/dL)	183.9±44.5	191±42.5
LDL (mg/dL)	105.4±36.2	108.1±35.8
HDL (mg/dL)	57.3±16.9	59.5±17.8
Baseline SLEDAI	3.8±4.0	1.6±2.0
Baseline SLICC damage	1.4±1.8	3.5±3.1
Any anticardiolipin antibody	132 (36)	57 (36)
Mean baseline daily prednisone dose (mg)	4.7±8.4	3.3±5.5
Lifetime prednisone dose
Prednisone <10 g	170 (46)	74 (47)
Prednisone 10–20 g	77 (21)	31 (20)
Prednisone >20 g	119 (33)	53 (33)

Data as mean±SD for continuous variables and number (%) for categorical variables.

ASCVD, Atherosclerotic Cardiovascular Disease; HDL, high-density lipoprotein; LDL, low-density lipoprotein; SLEDAI, Systemic Lupus Erythematosus Disease Activity Index; SLICC, Systemic Lupus International Collaborating Clinics.

Table 3

10-year incidence of MACE

Type of event	n=74
Coronary artery disease/myocardial infarction	15
Transient ischaemic attack/ischaemic stroke	33
Peripheral artery disease	9
Death from cardiac cause	17

MACE, major adverse cardiovascular event.

Of the patients who experienced at least one MACE, 45% had a QRISK3 score >10%, whereas 20.5% of patients who did not have an event had a QRISK3 score >10% (p<0.001). In comparison, 11% of patients who had at least one MACE had an FRS >10%, whereas 7.2% who did not have an event had an FRS >10% (p=ns). The corresponding numbers for mFRS, ASCVD and PREDICTS were 40.6% and 28.0% (p=0.05), 12.2% and 5.9% (p=ns), and 77% and 32.1% (p<0.001), respectively.

The area under the ROC using QRISK3 >10% (0.60) was larger than using ASCVD >10% (0.53), FRS >10% (0.52) and mFRS >10% (0.55), although it was slightly smaller than using high-risk PREDICTS score (0.65) (figure 1, table 4).

Figure 1

Comparison of ROC curves between risk calculators. ASCVD, Atherosclerotic Cardiovascular Disease; FRS, Framingham Risk Score; PREDICTS, Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE; ROC, receiver operating characteristic curve.

Table 4

Comparison of area under the ROC between risk calculators

	Area under the ROC	Asymptotic significance	95% CI
QRISK3 >10%	0.60	0.05	0.50 to 0.71
QRISK3 >20%	0.57	0.20	0.46 to 0.67
ASCVD >10%	0.53	0.57	0.43 to 0.64
FRS >10%	0.52	0.75	0.41 to 0.62
Modified FRS >10%	0.55	0.32	0.45 to 0.66
High-risk PREDICTS	0.65	0.000	0.55 to 0.74

ASCVD, Atherosclerotic Cardiovascular Disease; FRS, Framingham Risk Score; PREDICTS, Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE; ROC, receiver operating characteristic curve.

Using 95% CI, QRISK3 and PREDICTS outperformed the other calculators in terms of sensitivity and negative predictive value, although ASCVD had the highest specificity and positive predictive value (table 5).

Table 5

Comparison of sensitivity, specificity, PPV and NPV between risk calculators

	Sensitivity% (95% CI)	Specificity% (95% CI)	PPV% (95% CI)	NPV% (95% CI)
QRISK3 >10%	45.3 (32.8 to 58.2)	79.5 (74.5 to 83.9)	32.2 (25.1 to 40.3)	87.1 (84.3 to 89.5)
QRISK3 >20%	21.4 (12.5 to 32.9)	94.6 (91.5 to 96.9)	48.4 (32.8 to 64.8)	83.8 (82.0 to 85.4)
ASCVD >10%	12.2 (4.1 to 26.2)	94.0 (88.1 to 97.6)	41.7 (19.4 to 68.0)	75.3 (72.9 to 77.5)
FRS >10%	10.9 (4.5 to 21.3)	92.8 (89.1 to 95.5)	25.0 (12.9 to 42.9)	82.5 (81.2 to 83.8)
Modified FRS >10%	40.6 (28.5 to 53.6)	72.1 (66.5 to 77.2)	24.3 (18.5 to 13.3)	84.6 (81.6 to 87.2)
High-risk PREDICTS	73.4 (60.9 to 83.3)	65.2 (59.6 to 70.6)	30.9 (26.6 to 35.7)	92.1 (88.4 to 94.6)

ASCVD, Atherosclerotic Cardiovascular Disease; FRS, Framingham Risk Score; NPV, negative predictive value; PPV, positive predictive value; PREDICTS, Predictors of Risk for Elevated Flares, Damage Progression and Increased Cardiovascular Disease in Patients with SLE.

Supplemental analyses performed on the ASCVD cohort of patients aged 40 and above showed similar findings in terms of area under the ROC (online supplemental tables 1 and 2).

Discussion

Both QRISK3 and PREDICTS demonstrated better performance compared with ASCVD, FRS and mFRS in predicting risk of CVD in this cohort of patients with SLE. This is the first study to validate the use of the QRISK3 calculator in a longitudinal cohort of patients with SLE from the USA, and it indicates that QRISK3 may be a more useful risk assessment tool in this patient population compared with traditional calculators.

That said, the positive predictive value of QRISK3 >10% was only 32% and the area under the ROC only 0.60, reflecting limitations in its discriminatory ability. Although QRISK3 takes into account the presence of SLE and active corticosteroid use, it appears that this combination of variables is still neither sufficient nor precise enough to accurately gauge cardiovascular risk in this population. Given that QRISK3 and PREDICTS each performed better than the other calculators, perhaps a combination of the two approaches aiming to find the optimal combination of clinical and biochemical characteristics would produce better results.

There have yet to be clear recommendations on CVD prevention strategies (such as use of statins and aspirin for primary prevention) in SLE. Statins did not demonstrate a benefit in reducing subclinical atherosclerosis in a cohort of patients with SLE12; however, they did decrease all-cause mortality in another cohort of SLE patients with hyperlipidaemia.13 At this point the consensus is to use them in accordance with current guidelines for the general population.12

To help inform these questions, it is first necessary to create a more accurate CVD risk stratification tool to determine which patients in this population would be most likely to benefit from interventions or more frequent assessment. In a study by O’Neill and colleagues14 in London, institution of a CVD risk assessment protocol in their lupus clinic involving annual FRS calculation did not change management in most (96.5%) of cases, largely because only a small fraction had FRS high enough to warrant consideration of lipid-lowering therapy and the majority of those patients were already on therapy on account of prior CVD or diabetes mellitus. Perhaps this would not have been the case using a more accurate risk stratification tool better tailored to the SLE population, which could identify patients missed by traditional risk calculators.

Our study is unique in being the first validation study of the QRISK3 calculator in patients with SLE in the USA. This was a prospective study that included a relatively large sample size. Additionally, we compared QRISK3 with a variety of other cardiovascular risk calculators.

This study has some limitations. While there is of course a strong predilection for SLE to affect women more than men, this ratio was exaggerated in our cohort, which included only 3% men, which may limit the generalisability of the findings to male patients. Our sample was not large enough to stratify patients by demographic factors such as ethnicity and disease duration, which could have been helpful since these factors affect CVD risk. There is the potential for selection bias in using data from a cohort designed to study atherosclerosis biomarkers, as well as biases associated with missing data since data were collected via chart review. The calculators we studied differ in the number of data elements required to generate the risk score, which affects their ease of use. QRISK3 requires more extensive chart review, while PREDICTS requires laboratory tests that are not yet fully available commercially.

We acknowledge that the qualitative definition of ‘high risk’ varies between the calculators being compared. The UK National Institute for Health and Care Excellence classifies patients with QRISK3 score ≥10% as being at high risk of CVD, whereas in the USA, where FRS and ASCVD are used, scores of >20% are considered high risk. However, we decided to use the same threshold of 10-year CVD risk of >10% for the QRISK3, FRS, mFRS and ASCVD to compare quantitative rather than qualitative risk. Additionally, very few patients in our cohort had a risk score of >20% (only 1 patient with the traditional FRS, 3 patients with mFRS, 1 patient with ASCVD and 31 with QRISK3). The ‘high-risk’ PREDICTS profile does not correlate with a specific 10-year CVD risk, but we used this categorisation based on the limits of the risk model.

In summary, the incorporation of SLE and corticosteroid use into the QRISK3 calculator appears to be a step in the right direction towards establishing a more accurate CVD risk stratification tool for patients with SLE, although further optimisation is needed.