Risk Factors and Predictive Model for Dermatomyositis Associated with Rapidly Progressive Interstitial Lung Disease.

Background: Rapidly progressive interstitial lung disease (RP-ILD) is a significant complication that determines the prognosis of dermatomyositis (DM). Early RP-ILD diagnosis can improve screening and diagnostic efficiency and provide meaningful guidance to carry out early and aggressive treatment.
Methods: A retrospective screening of 284 patients with DM was performed. Clinical and laboratory characteristics of the patients were recorded. The risk factors of RP-ILD in DM patients were screened by logistic regression (LR) and machine learning methods, and the prediction models of RP-ILD were developed by machine learning methods, namely least absolute shrinkage and selection operator (LASSO), random forest (RF), and extreme gradient boosting (XGBoost).
Results: According to the result of univariate LR, disease duration is a protective factor for RP-ILD, and ESR, CRP, anti-Ro-52 antibody and anti-MDA5 antibody are risk factors for RP-ILD. The top 10 important variables of the 3 machine learning models were intersected to obtain common important variables, and there were 5 common important variables, namely disease duration, LDH, CRP, anti-Ro-52 antibody and anti-MDA5 antibody. The AUC of LASSO, RF and XGBoost test set were 0.661, 0.667 and 0.867, respectively. We further validated the performance of these three models on validation set, and the results showed that, the AUC of LASSO, RF and XGBoost were 0.764, 0.727 and 0.909, respectively. Based on the results of the models, XGBoost is the optimal model in this study.
Conclusion: Disease duration, LDH, CRP, anti-Ro-52 antibody and anti-MDA5 antibody are vital risk factors for RP-ILD in DM. The prediction model constructed using XGBoost can be used for risk identification and early intervention in DM patients with RP-ILD and practical application.

Introduction

The main manifestations of dermatomyositis (DM) are varying degrees of inflammation of the muscles, skin and lungs, and are closely related to the severity and prognosis of the disease.1 Interstitial lung disease (ILD) and malignancy are significant complications that determine the prognosis of patients with DM.2 Rapidly progressive ILD (RP-ILD) is a subtype of DM-ILD, which is rapidly progressive, challenging to treat, and has a high mortality rate compared to DM-ILD, and is critical in the clinical management of DM. A large cohort study of myositis in the United States showed that the 1-, 5- and 10-year survival rates of myositis patients with ILD were 97, 91 and 81%, respectively.3 The 6-month survival rate of patients with RP-ILD is 28%-66% after several aggressive treatments.4,5 Therefore, early diagnosis of RP-ILD can improve the efficiency of screening and diagnosis and provide meaningful guidance for carrying out early and aggressive treatment.6

Positive anti-melanoma differentiation-associated gene 5 (MDA5) antibody is the one antibody that is known to be highly associated with RP-ILD.7 About half of the patients with MDA5-positive DM develop RP-ILD.8,9 The relationship between other myositis-associated antibodies and RP-ILD is unknown. In addition to this, Myofascia-dominant involvement, elevated CRP, decreased lymphocyte count, elevated Soluble CD206, elevated interleukin-15 (IL-15), elevated carcinoembryonic antigen, and prolonged activated partial thromboplastin time (aPTT) ratio are also risk factors for the development of RP-ILD in DM patients.10–15 High plasma ferritin levels, generalized worsening of pulmonary infiltrates, ground glass shadow, and elevated serum neopterin levels are significantly associated with reduced survival in DM patients with RP-ILD.16,17 Studies on RP-ILD associated with juvenile DM, on the other hand, found that IL-18, KL-6, ferritin and anti-MDA5 antibodies were associated with RP-ILD.18 However, most of the risk factors reported in these studies are single clinical features or laboratory indicators, and it would be a better method to use multiple clinical features or laboratory indicators to develop a disease prediction model. The purpose of this study was to collect clinical characteristics and laboratory data from DM patients, to identify risk factors for RP-ILD in DM patients, and to develop a valid predictive model for RP-ILD in DM patients.

Materials and Methods

Study Cohort and Patients

All 257 patients with DM included in this study were from January 2017 to December 2020 inpatients of the Affiliated Huai’an No.1 People’s Hospital of Nanjing Medical University, and 27 patients with DM in the validation cohort were from January 2021 to August 2021 inpatients of the Affiliated Huai’an No.1 People’s Hospital of Nanjing Medical University. The diagnosis of DM [including clinically amyopathic DM (CADM)] was based on the Bohan and Peter classification criteria. In this cohort, myositis-specific autoantibodies (MSAs) were measured by immunoblotting (EUROLINE, EUROIMMUN AG, Germany) by the same central laboratory in all patients. The medical records of all patients were retrospectively analyzed. Demographic data, including sex, age at onset and disease duration; clinical features, including muscle weakness, Gottron’s sign, heliotrope rash, V rush, shawl sign, periungueal erythema, arthritis, mechanic’s hand, skin ulcers (clinical features come from patient’s symptom record and are confirmed by doctor’s physical examination record); laboratory characteristics such as glutamate transaminase (ALT), aspartate transaminase (AST), lactate dehydrogenase (LDH), creatine kinase (CK), erythrocyte sedimentation rate (ESR), C-reactive protein (CRP), serum ferritin (SF), antinuclear antibody (ANA), anti-Ro-52 antibody, anti-aminoacyl-tRNA synthetase (ARS) antibodies, and anti-MDA5 antibody were extracted from the electronic records of patients at the beginning of hospitalization. The study complies with the Declaration of Helsinki, and the ethics committee of the Affiliated Huai’an No.1 People’s Hospital of Nanjing Medical University approved the study protocol. The requirement for informed consent was waived due to the study’s retrospective nature and anonymous data.

Inclusion and Exclusion Criteria

The diagnosis of ILD is based on the clinical signs and characteristics of CT scans. Diagnosis of ILD (notable ILD findings such as hairy glass attenuation, solid, reticular and honeycomb) is based on respiratory symptoms (dry cough and dyspnea on exertion), physical examination (such as velcro rales at the base of the lungs) and high-resolution CT (HRCT) findings, excluding interstitial changes due to infection and medications. RP-ILD is defined as the presence of one of the following four conditions within one month: 1) Progressive worsening of dyspnea or cough symptoms with a significant decrease in quality of life; 2) A decrease in lung function, including forceful lung volume (FVC) of more than 10% or a decrease in pulmonary diffusion capacity for carbon monoxide (DLCO) of more than 15%; 3) A persistent increase in the degree of interstitial pneumonia on high-resolution CT (HRCT) of the chest; 4) Arterial blood gas analysis demonstrating respiratory failure or a decrease in partial pressure of oxygen more significant than 10 mmHg, independently determined by a senior physician in the rheumatology and respiratory departments. Exclusion criteria include (1) patients with other autoimmune diseases, including rheumatoid arthritis, systemic lupus erythematosus and systemic sclerosis; (2) patients with lung cancer and chronic lung disease, including environmental, occupational and chronic obstructive pulmonary disease.

Vacancy Value Processing

Patients with more missing data (≥ 3 variables) are directly deleted; for patients with less missing data (< 3 variables), the KNN (K-Nearest Neighbor) algorithm is used to fill in the missing values.

Logistic Regression

The R package “rms” was used to perform univariate logistic regression (LR) for all independent variables, and the R package “caret” was used to perform multivariate LR for independent variables with significant differences, and the “step” function was used to achieve stepwise regression to filter the characteristic variables to develop the predictive model.

Machine Learning Approach

To develop predictive models of RP-ILD, three machine learning algorithms were applied: least absolute shrinkage and selection operator (LASSO), random forest (RF), and extreme gradient boosting (XGBoost). These machine algorithms (LASSO, RF, and XGBoost) were applied using the R package glmnet,19 ranger20 and xgboost,21 respectively. Metabolomics data was divided into two parts: the training set (70%) and the test set (30%). We performed a repeated K-fold cross-validation (repeats = 5, K = 5) on the models in order to obtain an unbiased estimate for their performance. Using the R package pROC, the receiver operating characteristic (ROC) curve was prepared.22

Results

A total of 284 patients with DM were included in this study, 257 of which were used for model development and internal validation studies and 27 for external validation. The main characteristics of the participants are shown in Table 1 and .

Table 1

Clinical Characteristic of DM with RP-ILD and DM Without RP-ILD

	DM with RP-ILD (N = 41)	DM without RP-ILD (N = 216)	P-value
Age (years)	55.10 ± 9.60	51.92 ± 13.51	0.172
Female (%)	29 (70.73)	157(72.69)	0.798
Disease duration (months)	1.98 ± 1.40	12.27 ± 31.57	0.001
Muscle weakness (%)	38 (92.68)	180(83.33)	0.126
Gottron’s sign (%)	29(70.73)	124(57.41)	0.111
Heliotrope rash (%)	18 (43.90)	104 (48.15)	0.618
V rush (%)	14(34.15)	71(32.87)	0.874
Shawl sign (%)	11(26.83)	40(18.52)	0.221
Periungueal erythema (%)	9(21.95)	35(16.20)	0.370
Arthritis (%)	13(31.71)	67(31.02)	0.930
Mechanic’s hand (%)	11(26.83)	45(20.83)	0.394
Skin ulcers (%)	5(12.20)	26 (12.04)	0.977
ALT (U/L)	75.51 ± 115.70	78.11 ± 93.87	0.329
AST (U/L)	99.98 ± 164.60	94.73 ± 126.70	0.611
LDH (U/L)	481.66 ± 321.15	386.82 ± 252.44	0.038
CK (U/L)	125.98 ± 168.04	474.93 ± 1706.02	0.785
ESR (mm/h)	47.85 ± 21.72	36.56 ± 21.47	0.001
CRP (mg/L)	18.51 ± 28.29	10.03 ± 19.57	<0.001
Serum ferritin (ng/mL)	960.12 ± 515.81	790.99 ± 1223.41	0.002
ANA positive (%)	25(60.98)	113(52.31)	0.308
Anti-Ro-52 antibody positive (%)	31(75.61)	80(37.04)	<0.001
Anti-ARS antibody positive (%)	3(7.32)	25(11.57)	0.423
Anti-MDA5 antibody positive (%)	33(80.49)	67(31.02)	<0.001

Notes: Bold values are statistically significant (P < 0.05).

Abbreviations: DM, dermatomyositis; RP-ILD, rapidly progressive interstitial lung disease; ALT, glutamate transaminase; AST, aspartate transaminase; LDH, lactate dehydrogenase; CK, creatine kinase; ESR, erythrocyte sedimentation rate; CRP, C-reactive protein; SF, serum ferritin; ANA, antinuclear antibody; anti-ARS antibodies, anti-aminoacyl-tRNA synthetase antibodies; anti-MDA5 antibody, anti-melanoma differentiation-associated gene 5 antibody.

Since the difference between the results in the univariate LR analysis does not reflect the effect of the factor on the outcome event very realistically, we used the variables with P < 0.2 in the results of the univariate LR analysis as the first echelon of candidate variables23 and screened out nine independent variables, namely age at onset, disease duration, muscle weakness, Gottron’s sign, LDH, ESR, CRP, anti-Ro-52 antibody, and anti-MDA5 antibody (Table 2). Among them, disease duration is a protective factor for RP-ILD, and ESR, CRP, anti-Ro-52 antibody and anti-MDA5 antibody are risk factors for RP-ILD. Previous studies have shown that when risk factors are screened using logistic regression, the regression results are more stable and more consistent with the original results when the events per variable (EPV) is greater than or equal to 10.24,25 There were 9 variables screened by univariate LR in this study; therefore, the number of EPVs required needed to exceed 90, and the number of EPVs required to select variables with P < 0.5 needed to exceed 50, and the number of DM patients with RP-ILD occurred in 41 cases; therefore, it was not suitable for further analysis using multivariate LR.

Table 2

Univariate Logistic Regression Analysis of Clinical and Laboratory Characteristics

Variables	OR	2.5% CI	97.5% CI	P-value
Sex	1.10	0.53	2.30	0.798
Age at onset	1.02	0.99	1.05	0.152
Disease duration	0.78	0.66	0.92	0.004
Muscle weakness	2.53	0.74	8.66	0.138
Gottron’s sign	1.79	0.87	3.70	0.114
Heliotrope rash	0.84	0.43	1.65	0.618
V rush	1.06	0.52	2.14	0.874
Shawl sign	1.61	0.75	3.49	0.224
Periungueal erythema	1.45	0.64	3.31	0.372
Arthritis	1.03	0.50	2.12	0.93
Mechanic’s hand	1.39	0.65	2.99	0.395
Skin ulcers	1.01	0.37	2.82	0.977
ALT	1.00	1.00	1.00	0.878
AST	1.00	1.00	1.00	0.815
LDH	1.00	1.00	1.00	0.044
CK	1.00	1.00	1.00	0.261
ESR	1.02	1.01	1.04	0.003
CRP	1.01	1.00	1.03	0.036
Serum ferritin	1.00	1.00	1.00	0.406
ANA	1.42	0.72	2.82	0.309
Anti-Ro-52 antibody	5.27	2.45	11.32	<0.001
Anti-ARS antibody	0.60	0.17	2.10	0.427
Anti-MDA5 antibody	9.17	4.02	20.92	<0.001

Notes: Bold values are statistically significant (P < 0.05).

Abbreviations: OR, odds ratio; CI, confidence interval; ALT, glutamate transaminase; AST, aspartate transaminase; LDH, lactate dehydrogenase; CK, creatine kinase; ESR, erythrocyte sedimentation rate; CRP, C-reactive protein; SF, serum ferritin; ANA, antinuclear antibody; anti-ARS antibodies, anti-aminoacyl-tRNA synthetase antibodies; anti-MDA5 antibody, anti-melanoma differentiation-associated gene 5 antibody.

To develop predictive models for RP-ILD, we adopted three machine learning methods. The AUC, sensitivity, and specificity of LASSO test set were 0.669, 0.417 and 0.922, respectively (Figure 1A); the AUC, sensitivity, and specificity of RF test set were 0.542, 0.083 and 1.000, respectively (Figure 1B); the AUC, sensitivity, and specificity of XGBoost test set were 0.663, 0.881 and 0.444, respectively (Figure 1C); and the performance summary of RF, LASSO and XGBoost on test set is shown in Table 3. On the basis of the first round of machine learning, we screened out the important variables in the models. There were 13 important variables in LASSO, as shown in Figure 1D. Figure 1E showed the importance of each variable in RF. There are 17 important variables in XGBoost, as shown in Figure 1F. The top 10 important variables of the 3 models were intersected to obtain common important variables. As shown in the Venn diagram (Figure 1G), there were 5 common important variables, namely disease duration, LDH, CRP, anti-Ro-52 antibody and anti-MDA5 antibody. The second round of machine learning is based on these five common variables. The AUC, sensitivity, and specificity of LASSO test set were 0.661, 0.417 and 0.906, respectively (Figure 2A, Table 3); the AUC, sensitivity, and specificity of RF test set were 0.667, 0.333 and 1.000, respectively (Figure 2B); the AUC, sensitivity, and specificity of XGBoost test set were 0.867, 0.900 and 0.833, respectively (Figure 2C); and the performance summary of RF, LASSO and XGBoost on test set is shown in Table 4.

Figure 1

Different machine learning methods on 23 variables. (A) ROC curve of LASSO on the test set. (B) ROC curve of RF on the test set. (C) ROC curve of XGBoost on the test set. (D) Important variables screening of LASSO. (E) Important variable screening of RF. (F) Important variable screening of XGBoost. (G) Venn diagram of machine learning methods.

Table 3

Performance Summary of Different Machine Learning Models Based on 23 Variables

	Accuracy	Recall	NPV	PPV	F1-Score
LASSO on test set	0.842	0.417	0.894	0.500	0.455
RF on test set	0.855	0.083	0.853	1.000	0.154
XGBoost on test set	0.829	0.881	0.333	0.922	0.901

Abbreviations: LASSO, least absolute shrinkage and selection operator; RF, random; XGBoost, extreme gradient boosting; NPV, negative predictive value; PPV, positive predictive value.

Figure 2

ROC curve of prediction models based on 5 variables. (A) ROC curve of LASSO on the test set. (B) ROC curve of RF on the test set. (C) ROC curve of XGBoost on the test set. (D) ROC curve of LASSO on the validation set. (E) ROC curve of RF on the validation set. (F) ROC curve of XGBoost on the validation set.

Table 4

Performance Summary of Different Machine Learning Models Based on 5 Variables

	Accuracy	Recall	NPV	PPV	F1-Score
LASSO on test set	0.828	0.417	0.892	0.455	0.435
LASSO on validation set	0.741	0.800	0.941	0.400	0.533
RF on test set	0.895	0.333	0.889	1.000	0.500
RF on validation set	0.704	0.600	0.889	0.333	0.429
XGBoost on test set	0.894	0.900	0.417	0.984	0.940
XGBoost on validation set	0.852	1.000	1.000	0.555	0.714

Abbreviations: LASSO, least absolute shrinkage and selection operator; RF, random; XGBoost, extreme gradient boosting; NPV, negative predictive value; PPV, positive predictive value.

Machine Learning Models Validation

We further validated the performance of these three models on validation set, and the results showed that, the AUC, sensitivity, and specificity of LASSO were 0.764, 0.800 and 0.727, respectively (Figure 2D); the AUC, sensitivity, and specificity of RF were 0.664, 0.600 and 0.727, respectively (Figure 2E); the AUC, sensitivity, and specificity of XGBoost were 0.909, 1.000 and 0.818, respectively (Figure 2F); and the performance summary of RF, LASSO and XGBoost on validation set is shown in Table 4. Based on the results of the models, XGBoost is the optimal model in this study. To make the prediction model more convenient and easy to use, we used the R package “shiny” to create a web version of the predictive model tool based on XGBoost at .

Discussion

The mechanism by which RP-ILD occurs is unclear, and multiple cytokines are involved in the pathogenesis of RP-ILD, such as interferon (IFN)-γ, IFN-α, IFN-inducible protein-10 (IP-10), IL-6, IL-8, IL-10, IL-15, and tumor necrosis factor-α (TNF-α).26,27 There is no standard treatment protocol for DM-related RP-ILD, but aggressive combinations with high-dose glucocorticoids, calcium-modulated neurophosphatase inhibitors and intravenous cyclophosphamide have been proposed.17,28 Plasma exchange, tocilizumab and rituximab have been used to treat refractory RP-ILD.29–31 Treatments such as veno-venous extracorporeal membrane lung oxygenation or even lung transplantation should be considered in life-threatening cases.17,28

Early identification of RP-ILD is vital in the management of patients with DM, and a considerable number of studies have reported multiple risk factors associated with RP-ILD. Most of the risk factors reported in these studies are single clinical features or laboratory indicators, and it is better to use multiple clinical features or laboratory indicators to develop a disease prediction model. Disease prediction models are mainly various types of condition scoring systems. These tools are mainly developed based on LR or Cox regression models, and have been applied to the prediction of many diseases.32–34 Multivariable models can better assess disease severity, predict outcomes, and guide individualized treatment than univariate models. However, in this study, there are many independent variables, and the number of DM patients is relatively small, so it is impossible to establish a prediction model by LR. Therefore, we used three different machine learning methods to develop prediction models for RP-ILD, and their AUC values were 0.669, 0.542 and 0.663, respectively. All three models have low AUC values.

After further screening of the variables according to their importance to each model, as shown in the results of the venn diagram, we obtained five most important variables, namely disease duration, LDH, CRP, anti-Ro-52 antibody and anti-MDA5 antibody. Univariate LR results showed that anti-MDA5 antibody was the strongest risk factor for RP-ILD (OR = 9.17, P < 0.001), followed by anti-Ro-52 antibody (OR = 5.27, P < 0.001). Several studies have confirmed that anti-Ro-52 antibody positivity is an independent risk factor for myositis and even in ILD associated with Sjogren’s syndrome.35–38 RP-ILD occurs more frequently in anti-ARS antibody positive patients who coexist with anti-Ro-52 antibody than in patients without anti-Ro-52 antibody.39 In addition, the positive rate of MSAs was high in patients with isolated anti-Ro-52-ILD.40 The effect of anti-Ro-52 antibody positivity on RP-ILD in DM patients has not been specifically studied. Our study showed that anti-Ro-52 antibody positivity is also a vital risk factor for RP-ILD in patients with DM. The AUCs of the prediction models developed based on the five variables were 0.661, 0.667 and 0.867, respectively, with XGBoost performing the best. After validation using an independent validation set, it is still XGBoost that has the highest AUC value. Therefore, we chose fewer variables to construct the XGBoost-based prediction model. We created a web-based version of the prediction tool on this basis, which is better able to predict the risk of RP-ILD in DM patients at early disease stages.

There are still many deficiencies in this research. First of all, this is a single-center cohort study with a small sample; Second, this is a retrospective study that only included hospitalized patients, and there may be deviations in patient selection, and some patients’ medical records are incomplete and some data are missing. Thirdly, all patients in this study are Chinese, and the prediction effect on other races needs to be further verified. Fourthly, in addition to the laboratory data we collected, there are many laboratory indicators that are highly correlated with RP-ILD, such as cytokines, soluble CD 206, Krebs von den Lungen-6 and serum neopterin levels.12,16,18 However, this study is a retrospective study. The vast majority of patients lack these indicators. We will continue this work in the future, providing a better reference for the early diagnosis and treatment of RP-ILD.

Conclusions

In conclusion, anti-MDA5 antibody and anti-Ro-52 antibody are vital risk factors for RP-ILD in DM. The prediction model constructed using XGBoost can be used for risk identification and early intervention in DM patients with RP-ILD and practical application.