Enrichment Strategy for Systemic Sclerosis Clinical Trials Targeting Skin Fibrosis: A Prospective, Multiethnic Cohort Study.

OBJECTIVE: The modified Rodnan skin score (mRSS) is often used as a primary outcome measure in systemic sclerosis (SSc) randomized clinical trials (RCTs). Previous cohort studies with predominantly European Caucasian patients showed that setting an upper limit of mRSS as a selection criterion for RCTs leads effectively to enrichment with progressive patients. This study aimed to demonstrate this effect in an ethnically diverse cohort, rich in patients positive for anti-RNA polymerase III antibodies (Pol3).
METHODS: We selected from the Genetics versus Environment in Scleroderma Outcomes Study (GENISOS) cohort patients with diffuse cutaneous SSc (dcSSc), who had mRSS of 7 or more at inclusion and a documented mRSS after 12 ± 2 months. Progression of skin fibrosis was defined as an increase in mRSS greater than 5 points and 25% or more from baseline. To identify the optimal cutoff for the baseline mRSS yielding the highest sensitivity for progressive skin fibrosis, we developed ROC curves and logistic regression models with "progression" as the outcome variable and a binary variable of baseline mRSS cutoff point as predictor.
RESULTS: We included 152 patients (age and disease duration [mean ± SD, years]: 48.7 ± 13.0 and 2.4 ± 1.5 respectively, 22.4% males, 34.2% Pol3-positive). Seventeen patients (11.2%) had skin fibrosis progression after 12 ± 2 months. An mRSS cutoff of 27 or less had the highest probability of progression (odds ratio, 9.12; 95% confidence interval: 1.173-70.851; P = 0.035; area under the curve, 0.652; sensitivity, 94%).
CONCLUSION: We demonstrated in an ethnically diverse cohort of patients with early dcSSc and with a high proportion of patients who are Pol3-positive that setting an upper limit of the mRSS as a selection criterion leads effectively to cohort enrichment with progressors.

Previous studies in the large, Caucasian‐predominant European Scleroderma Trials and Research (EUSTAR) cohort showed that, in patients with early diffuse cutaneous systemic sclerosis (dcSSc), setting an upper threshold limit for the modified Rodnan skin score (mRSS) as a study inclusion criterion increases the proportion of patients with progression of skin fibrosis at 12 months and reduces the absolute number of patients with regression of skin fibrosis under standard of care. We validated these findings in the Genetics versus Environment in Scleroderma Outcomes Study (GENISOS) cohort, which is characterized by an ethnically diverse patient population and a different antibody profile, with a high prevalence of anti‐RNA polymerase III antibodies (Pol3), both of which are associated with different disease dynamics and disease course.

In our study, Pol3‐positive patients had higher baseline mRSS than Pol3‐negative patients, but only 1 of 50 Pol3‐positive patients had progression of skin fibrosis at 12 months. This suggests that peak mRSS had been reached in this patient subset before study enrollment and that antibody status and disease duration play an important role in the course of mRSS, with important consequences for the design of randomized controlled trials in early dcSSc.

INTRODUCTION

Skin fibrosis is a main domain in the assessment of patients with systemic sclerosis (SSc). The modified Rodnan skin score (mRSS) is the most frequently used outcome measure in SSc randomized clinical trials (RCTs), in which the treatment of interest is aimed to control fibrosis (1). Despite the fact that mRSS has failed in a number of recent RCTs as a primary outcome (2, 3, 4, 5), it remains the key component of the composite response index for SSc (CRISS) (6), which is currently used by a number of ongoing trials. The mRSS also remains the most frequently used measure of skin fibrosis as a secondary outcome in RCTs related to SSc (7). Thus, detailed information on the performance of the mRSS in clinical trials continues to be of key relevance for optimal study design in SSc.

The mRSS is a fully validated and reliable outcome measure; however, its main drawback is its difficult‐to‐predict course in the individual patient, with highly variable time to peak and spontaneous improvement under standard of care (8). Analyses performed on the large European Scleroderma Trials and Research Group (EUSTAR) cohort in patients with diffuse cutaneous SSc (dcSSc) have shown that the exclusion of patients with advanced skin fibrosis leads to cohort enrichment with patients with progressive skin disease (9, 10). However, the generalizability of these analyses was limited by the high predominance of Caucasian ethnicity and the very low prevalence of anti‐RNA Polymerase III (Pol3) antibodies in the EUSTAR cohort (11). These results might thus not be generalizable to other ethnicities and to countries with a higher prevalence of patients with Pol3 antibodies, such as the United States.

To address these limitations, we analyzed the prospectively collected data from the Texas‐based Genetics versus Environment in Scleroderma Outcomes Study (GENISOS) cohort, which is an ethnically diverse cohort and includes a large proportion of Pol3‐positive patients (12). The objectives of the present study were to analyze whether lower mRSS holds true to be an important enrichment factor for progression of skin fibrosis in this cohort and to identify the optimal cutoff for the upper threshold of baseline mRSS that yields the highest sensitivity for progressive skin fibrosis.

Patients

GENISOS is a prospective cohort of patients with early SSc, aged 18 years or older at inclusion, with disease duration of 5 years or less since the first non‐Raynaud symptom (12). Patients need to have a defined ethnicity, with all four grandparents from the same ethnic group. Patients with SSc‐like illnesses associated with environmental, ingested, or injected agents were excluded. All enrolled subjects were evaluated according to a standard protocol every 6 months for the first 3 years, and the same investigators performed the mRSS assessment. This study was based in three University of Texas institutions in Houston, Galveston, and San Antonio. It was approved by the local institutional review board (Research Ethics Board number: HSC‐MS‐02‐161), and all subjects provided written informed consent either in English or Spanish in their GENISOS center. All contributing GENISOS centers have obtained approval from their respective ethics committee for including a patient's data in the GENISOS database after the patient has given written informed consent. The mRSS evaluators (MDM, SA) were trained in an investigator training session prior to the Scleroderma Clinical Trials Consortium certification process and certified once this became available (13). All SSc‐related antibodies were determined according to the gold standard method in a central laboratory (for further details regarding the GENISOS cohort, see Assassi et al (12)).

We selected from the GENISOS cohort patients aged 18 years or more, fulfilling the 1980 ACR and the 2013 ACR/EULAR criteria for SSc (14, 15) and subclassification criteria dcSSc at inclusion, as defined by LeRoy et al (16), with an mRSS of 7 or more at inclusion, and a follow‐up (FUP) visit with documented mRSS at 12 ± 2 months. We chose a follow‐up time of 12 months, as this is the classical trial duration in SSc RCTs. Patients who had an increase in mRSS of more than 5 points and 25% or more from baseline at the FUP visit were defined as progressors, whereas patients who had a decrease in mRSS of more than 5 points and 25% or more from baseline were defined as regressors. Patients who did not qualify as either progressors or regressors were defined as stable patients. Our inclusion criteria and definition of progressors and regressors were identical to those applied in the EUSTAR studies by Maurer et al and Dobrota et al and are based on the minimally clinically important differences in patients with dcSSc (9, 10).

Statistical methods

The analysis was performed with the statistics software IBM SPSS 20.0. A P value < 0.05 was considered statistically significant. Comparisons between groups were performed with the χ2 test (for categorical variables) or with the independent samples t test (for normally distributed numeric variables). Distribution was considered normal if skewness was less than 1. We developed receiver operating characteristic (ROC) curves and univariable logistic regression models with “progressor” status as outcome (dependent) variable and a binary variable (less than or equal to/greater than) of baseline mRSS cutoff point as predictor (independent variable), aiming to select the mRSS cutoff with the highest area under the curve (AUC).

Description of the study cohort and progressor status at 12 ± 2 months

Among the 443 patients in the cohort, we identified 152 patients matching the inclusion criteria (22.4% males, age and disease duration [mean ± SD, years] 48.7 ± 13.0 and 2.4 ± 1.5, respectively). Reasons for exclusions are shown in Supplementary Table 1. As expected, ethnicity was diverse: there were 31 (20.4%) African Americans, 34 (22.4%) Hispanics, 77 (50.7%) Caucasians, and 10 (6.5%) patients with another ethnicity. The cohort included 50 (34.2%) Pol3‐positive patients, which is substantially higher than in the previously analyzed EUSTAR cohorts (9, 10). Pol3‐positive patients were slightly older than Pol3‐negative patients (50.9 ± 11.23 vs 47.26 ± 12.97 years), had higher mRSS at baseline (29.14 ± 9.48 vs 20.59 ± 8.97) and shorter disease duration (2.08 ± 1.59 vs 2.62 ± 1.44 years). During follow‐up, 71 of 151 (47.0%) patients were treated with glucocorticoids, 29 of 151 (19.2%) with glucocorticoids >5 mg/d prednisone (or equivalent), and 71 of 151 (47.0%) with immunosuppressive drugs (17.8% methotrexate, 17.2% mycophenolate mofetil, 6.0% cyclophosphamide, and 8.5% other). Immunosuppressive treatment allocation varied among the different antibody subsets, from 27 of 50 (54.0%) in patients who are Pol3‐positive to 8 of 26 (30.8%) in patients who are anti‐topoisomerase–positive and 3 of 11 (27.3%) in patients who are anti‐centromere–positive.

At 1 year of follow‐up, there were 17 (11.2%) progressors, 51 (33.6%) regressors, and 84 (55.2%) stable patients. For further analysis, we focused on parameters that had been proposed to differentiate progressors from nonprogressors (9, 10). Progressors had a significantly shorter disease duration and a lower baseline mRSS compared with patients in whom skin fibrosis did not progress, confirming the previous EUSTAR analyses (9, 10). Progressors were also more frequently positive for anti‐topoisomerase 1 antibodies and more frequently negative for Pol3 antibodies. Joint synovitis was not different between progressors and nonprogressors. Demographic and clinical data at inclusion are illustrated in Table 1.

Table 1

Demographic and baseline clinical data of the patients (N = 152)

	All (N = 152)	Progressors (n = 17)	Stable or Improving (n = 135)	P
Age, mean ± SD (y)	48.7 ± 13.0	47.6 ± 9.50	48.9 ± 13.4	0.626 a
Gender, n males (%)	34 (22.4%)	3 (17.6%)	31 (23%)	0.620 b
Race, n (%)
African Americans	31 (20.4%)	5 (29.4%)	26 (19.3%)	0.343 b
Hispanics	34 (22.4%)	2 (11.8%)	32 (27.3%)	0.266 b
Caucasians	77 (50.7%)	9 (52.9%)	68 (50.8%)	0.842 b
Disease duration mean ± SD, years	2.4 ± 1.5	1.5 ± 1.2	2.5 ± 1.5	0.005 a
Antibodies, n (%)
Anti‐topoisomerase 1	26/147 (17.7%)	6 (37.5%)	20 (15.3%)	0.028 b
Anti‐centromere	11/148 (7.4%)	1 (6.2%)	10 (7.6%)	0.849 b
Anti‐RNA polymerase III	50/146 (34.2%)	1 (6.2%)	49 (37.7%)	0.012 b
mRSS, mean ± SD	23.6 ± 10.0	18.2 ± 8.5	24.2 ± 10.0	0.012 a
Joint synovitis, n (%)	46 (30.3%)	3 (17.6%)	43 (31.9%)	0.230 b
Treatment, n (%)
Glucocorticoids (>5 mg/d prednisone or equivalent)	29 (19.2%)	3 (18.8%)	26 (19.3%)	0.961 b
Immunosuppressants	71 (47%)	7 (43.8%)	64 (47.4%)	0.782 b
Glucocorticoids (>5 mg/d prednisone or equivalent) and immunosuppressants	85 (56.3%)	8 (50.0%)	77 (57.0%)	0.592 b

Values highlighted in bold represent statistical significance.

P values are for comparisons between progressors and nonprogressors by independent samples t test.

P values are for comparisons between progressors and nonprogressors by the χ2 test.

Influence of baseline mRSS on the proportion of progressors and regressors at 12 ± 2 months

We next aimed to identify the optimal cutoff of the mRSS to differentiate between progressors and nonprogressors. We found that for an mRSS of up to 27 points, the proportion of progressors increased continuously, with a relatively stable proportion of regressors. With an mRSS cutoff of more than 27 points, the proportion of regressors increased, with no further gain in progressors (Figure 1, Table 2).

Figure 1

Number of progressors and regressors depending on different cutoffs for baseline mRSS. The histogram displays the number of patients (blue bars represent those with progression of skin fibrosis and orange bars represent those with regression of skin fibrosis, at 12 ± 2 months), who would be selected from the total of 152 patients with dcSSc and baseline mRSS of 7 or greater by applying the following criterion: baseline mRSS that meets or is less than the cutoff.

Table 2

Proportion of progressors and regressors depending on different cutoffs for baseline mRSS

mRSS Cutoff	N of 152	Progressors n (% of N)	Regressors n (% of N)	Ratio of Progressors to Regressors	OR	95% CI	P	AUC	95% CI	P
≤18	55	8 (14.5)	16 (29.1)	0.50	1.664	0.603–4.597	0.326	0.561	0.414–0.709	0.411
≤19	59	8 (13.6)	17 (28.8)	0.47	1.464	0.531–4.036	0.461	0.546	0.399–0.693	0.534
≤20	63	8 (12.7)	19 (30.2)	0.42	1.292	0.470–3.558	0.619	0.532	0.385–0.678	0.672
≤21	68	9 (13.2)	21 (30.9)	0.43	1.449	0.527–3.984	0.472	0.546	0.400–0.692	0.535
≤22	75	11 (14.7)	22 (29.3)	0.50	2.034	0.711–5.815	0.185	0.586	0.445–0.728	0.246
≤23	76	12 (15.8)	22 (28.9)	0.55	2.662	0.889–7.971	0.080	0.616	0.497–0.753	0.120
≤24	80	12 (15.0)	25 (31.2)	0.48	2.365	0.790–7.079	0.124	0.601	0.463–0.739	0.175
≤25	85	14 (16.5)	27 (31.8)	0.52	4.207	1.156–15.311	0.029	0.649	0.522–0.775	0.046
≤26	91	14 (15.4)	29 (31.9)	0.48	3.515	0.965–12.804	0.057	0.627	0.497–0.756	0.089
≤27	102	16 (15.7)	33 (32.4)	0.48	9.116	1.173–70.851	0.035	0.652	0.536–0.769	0.041
≤28	107	16 (15.0)	34 (31.8)	0.47	7.736	0.994–60.220	0.051	0.634	0.514–0.754	0.073
≤29	109	16 (14.7)	35 (32.1)	0.46	7.220	0.928–56.291	0.059	0.626	0.505–0.748	0.091
≤30	112	16 (14.3)	35 (31.2)	0.46	6.500	0.833–50.708	0.074	0.615	0.491–0.739	0.123
≤31	118	16 (13.6)	40 (33.9)	0.40	5.176	0.661–40.536	0.117	0.593	0.465–0.721	0.213
≤32	122	16 (13.1)	42 (34.4)	0.38	4.377	0.557–34.400	0.160	0.578	0.447–0.705	0.295
≤33	127	16 (12.6)	45 (35.4)	0.36	3.459	0.437–27.358	0.239	0.559	0.425–0.694	0.425
≤34	129	16 (12.4)	46 (35.7)	0.35	3.115	0.393–24.718	0.282	0.552	0.416–0.688	0.485

N is the number of patients from the cohort that have an mRSS equal to or less than the specified cutoff. Progressors and regressors are shown as number (n) and percentage (%) from the total number of patients less than or equal to the specified cutoff (N). Total numbers of progressors and regressors were 17 and 51, respectively. ROC analysis and logistic regression models with “progression” as outcome variable and a binary variable of baseline mRSS cutoff point as predictor. Abbreviation: AUC, area under the curve; CI, confidence interval; mRSS, modified Rodnan skin score; OR, odds ratio; ROC, receiver operating characteristic.

From the entire cohort, 102 patients had a baseline mRSS of 27 or less. Among them, 16 of all 17 progressors were represented, but only 33 of all 51 regressors. Using this cutoff as an inclusion criterion in a clinical trial (versus no cutoff) would have led to including 94% of all progressors (94% sensitivity) but only 65% of all regressors. This corresponds to 67% of all patients of the study cohort (36% specificity), increasing the percentage of subjects with skin progression from 11.2% to 15.7% and ensuring a ratio of progressors/improvers of about 1:2. With a more stringent upper limit of mRSS (≤25), specificity would increase to 47% but sensitivity would decrease to 84% and include only 85 of 152 (54%) patients.

Using ROC analysis to explore the relationship between different baseline mRSS cutoff points and the proportion of progressors included in the cohort, we found that the mRSS cutoff of 27 points or less had the highest probability of progression, with the highest AUC (odds ratio [OR], 9.1; 95% confidence interval [CI]: 1.2‐70.9; P < 0.035; AUC, 0.652). The second‐best performing cutoff was 25, with higher specificity (47.4%) but lower sensitivity (82%) and lower AUC (OR, 4.2; 95% CI: 1.2‐15.3; P < 0.029; AUC, 0.649). The results of the ROC analysis are displayed in Table 2.

DISCUSSION

We demonstrated in an ethnically diverse cohort of patients with dcSSc, among which were a high proportion of Pol3‐positive patients, that setting an upper limit of the mRSS as a selection criterion of patients who were potential candidates for enrollment in a clinical trial leads to cohort enrichment with progressors while limiting the number of regressors under standard‐of‐care treatment. Although this may set a significant limitation on recruitment (in our analysis, only 67% of all patients were eligible if we chose the mRSS cutoff of 27 or less), it increases the proportion of progressors and diminishes the absolute number of regressors (from 11.2% to 15.7% and from 51 to 34, respectively).

Our results are in line with those of the previous studies on the EUSTAR cohort (9, 10), which had the same inclusion criteria and definitions of regressors and progressors as the present study. Maurer et al, in their analysis on the EUSTAR cohort, identified a short disease duration (<15 months), the presence of joint synovitis, and a low mRSS at baseline (≥7 of 51 and ≤22 of 51) as independent predictors of skin progression in patients with dcSSc (9). Similar results have been obtained by a second analysis on the EUSTAR cohort, in which a baseline mRSS between 7 and 18 performed best, selecting the highest proportion of progressors (78.9%) and the lowest proportion of regressors (35.3%), whereas a baseline mRSS between 18 and 25 still allowed identification of a reasonably high rate of progressors over regressors (10).

In our analysis, a higher mRSS cutoff of 27 was identified by ROC analysis to perform best for cohort enrichment with progressors. These slight differences in the cutoff between different cohorts have to be interpreted with caution and might be explained by cohort characteristics. In our data, we highlighted the close performance of two different mRSS cutoffs (25 and 27 points), of which the higher one was selected because of higher sensitivity and AUC. Compared with the previous studies (9, 10), the baseline mRSS in the current cohort was higher than in the EUSTAR study patients (mean mRSS in the analyzed GENISOS patients was 23.6, versus 17.7 in the cohort studied by Maurer et al). This finding is most likely due to high proportion of RNA Pol3‐positivity in the GENISOS cohort. Most importantly, the optimal cutoff selected for a specific clinical study also depends on the specific study hypothesis and study design. For example, as the CRISS detects improvement, but not prevention of progression, a cutoff that includes a larger number of regressors might be favorable for a study design that has chosen the CRISS as a primary endpoint.

An (at first sight) surprising finding was that only one of the 50 patients who were Pol3‐positive was a progressor. This differs from data reported by the European Scleroderma Observational Study (ESOS) study, an observational, prospective international cohort of 326 patients with early dcSSc, with a disease duration of 3 years or less from the onset of skin thickening, in which 28% (14 out of 50) Pol3‐positive patients were progressors (17). This difference can be explained by the longer disease duration of Pol3‐positive patients in GENISOS (with a mean disease duration of about 2 years) than in ESOS (median disease duration of Pol3‐positive patients was 11.2 months). Indeed, patients who were Pol3‐positive showed a rapid increase in mRSS after disease onset, reaching the peak mRSS faster than other patients with dcSSc (17). However, we cannot exclude the possibility that patients who were Pol3‐positive from GENISOS might have been more aggressively treated than other groups, and their nonprogressor status could be a consequence of treatment, especially as the proportion of patients receiving immunosuppressive treatment was highest in this subset. The fact that 49 of 50 of our Pol3‐positive patients were not progressors is an important finding, suggesting that the combination of very short disease duration and Pol3 positivity is particularly meaningful to successfully recruit patients with progressive skin fibrosis into clinical studies. Our results, as well as results from the ESOS study, show that the skin progression rate is influenced by baseline mRSS, antibody status, and disease duration, with important consequences on the design of RCTs in early dcSSc. Moreover, these findings suggest that for cohort enrichment with skin fibrosis progressors, disease duration might need to be stratified by antibody status. However, this finding should be confirmed in larger cohorts.

Although the high quality of prospectively collected data with very few missing variables (<3% in the antibody status) adds to the strength of our study, the relatively small number of patients was a limitation of the present study. For this reason, we could neither analyze other markers as potential predictors of progression of skin fibrosis nor perform multivariable models, where antibody status, disease duration, and treatment would have been included as covariates. The small number of patients also accounts for the relatively low AUC. Another limitation was that, unlike the similar studies on patients included in EUSTAR, we didn’t have any data on previous course of skin fibrosis prior to study inclusion. Finally, we need to point out that, regardless of any selection based on baseline mRSS, more than 80% of patients had either stable or improving skin fibrosis at 12 months, and the increase of the proportion of progressors was not large, most probably due to considering a single baseline predictor of outcome (mRSS). This suggests that other baseline parameters should be taken into account for cohort enrichment.

In conclusion, lowering the upper threshold of mRSS at study inclusion increases the proportion of skin progressors and reduces the absolute number of regressors, even in an ethnically diverse patient population that has a high proportion of Pol3 antibodies. This confirms that this recruitment strategy should be used for clinical trial design in early dcSSc aiming at prevention of progression of skin fibrosis.

AUTHOR CONTRIBUTIONS

All authors were involved in drafting the article or revising it critically, and all authors approved the final version to be published and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study conception and design

Mihai, Dobrota, Distler.

Acquisition of data

Assassi, Mayes.

Analysis and interpretation of the data

Mihai, Dobrota, Assassi. Mayes, Distler.

Table S1

Click here for additional data file.