| Literature DB >> 33970338 |
Silvia Tommasin1, Sirio Cocozza2, Alessandro Taloni3, Costanza Giannì4, Nikolaos Petsas5, Giuseppe Pontillo2,6, Maria Petracca4,7, Serena Ruggieri4,8, Laura De Giglio4,9, Carlo Pozzilli4, Arturo Brunetti2, Patrizia Pantano4,5.
Abstract
OBJECTIVES: To evaluate the accuracy of a data-driven approach, such as machine learning classification, in predicting disability progression in MS.Entities:
Keywords: Disability progression; Machine learning; Magnetic resonance imaging; Multiple sclerosis
Mesh:
Year: 2021 PMID: 33970338 PMCID: PMC8563671 DOI: 10.1007/s00415-021-10605-7
Source DB: PubMed Journal: J Neurol ISSN: 0340-5354 Impact factor: 4.849
Fig. 1Flowchart of the methods. a MRI preprocessing. From 3D-T1-weighted images cerebellar volume was calculated via SUIT, thalamic volume using FSL’s FIRST and gray matter and white matter volumes via FSL’s SIENAX. On the T2-weighted images lesions were identified and segmented using JIM, to calculate the lesion load. By combining lesion and white matter masks, we calculated the normal appearing white matter mask for each subject. From diffusion-weighted images both fractional anisotropy and mean diffusivity maps were calculated, and results and were combined with the normal appearing mask previously obtained to extract microstructural metrics of the normal appearing white matter. b Feature selection. Clinical and neuroradiological, were selected together with binary classes (stable patients = 0, patients with disease progression = 1) and a random feature and used to describe the sample of patients with multiple sclerosis. c Machine learning classifier. After having checked features for co-linearity, a random forest classifier was applied 1000 times feature built on both clinical and radiological features, clinical features alone, radiological features alone. Out-of-Bag test was used to avoid overfitting and performances were evaluated via the confusion matrix of the surviving classifiers. DWI diffusion-weighted images, GM gray matter, WM white matter, FA fractional anisotropy, MD mean diffusivity
Clinical and neuroradiological features
| All subjects (Site 1 + Site 2) | Subjects at Site 1 | Subjects at Site 2 | Between sites comparison | |
|---|---|---|---|---|
| Average (std) | Average (std) | Average (std) | ||
| Number | 163 | 105 | 58 | |
| Age [years] | 39.66 (10.23) | 38.29 (9.75) | 42.13 (10.68) | |
| Sex (F/M) | 104/59 | 80/25 | 24/34 | |
| Phenotype (RR/P) | 122/41 | 85/20 | 37/21 | |
| Disease duration [years] | 9.90 (8.06) | 8.27 (7.97) | 12.87 (7.40) | |
| EDSS at baseline | 3.0 [0.0–7.5]** | 2.0 [0.0–7.5]** | 3.5 [2.0–7.5]** | |
| Time to follow-up [years] | 3.93 (0.95) | 4.2 (0.93) | 3.38 (0.72) | |
| Therapy (1st line, 2nd line, none) | 53, 65, 45 | 32, 31, 42 | 21, 34, 3 | – |
| Disability progression (Yes/No) | 58/105 | 36/69 | 22/36 | 0.22 (0.64)* |
| T2LL [ml] | 9.02 (10.31) | 6.77 (6.94) | 13.26 (13.62) | |
| GM Volume [ml] | 719.69 (92.18) | 737.38 (84.62) | 687.662 (97.32) | |
| WM Volume [ml] | 733.98 (96.91) | 768.98 (84.31) | 670.638 (85.94) | |
| Thalamic Volume [ml] | 17.64 (3.21) | 18.04 (3.22) | 16.92 (3.10) | |
| Cerebellar Volume [ml] | 113.66 (13.88) | 110.016 (12.92) | 120.25 (13.22) | |
| FA-NAWM | 0.40 (0.06) | 0.43 (0.03) | 0.33 (0.3) | |
| MD-NAWM × 10−3 | 0.75 (0.05) | 0.73 (0.07) | 0.80 (0.04) |
Clinical and neuroradiological features of Site 1 and Site 2 samples. Mann–Whitney test was used to test significant differences between groups. Significant differences are highlighted in bold font
Std standard deviation, F female; M male; RR relapsing remitting form; P progressive form, EDSS expanded disability status scale, T2LL T2 lesion load, GM Gray Matter, WM White Matter, FA-NAWM fractional anisotropy of normal appearing white matter, MD-NAWM mean diffusivity of normal appearing white matter
*χ-square statistics was used
**Median [range]
Correlation matrix, Site 1 sample
| AGE | DD | EDSS@base | T2LL | FA-NAWM | MD-NAWM | Thalamic volume | GM volume | WM volume | Cerebellar volume | |
|---|---|---|---|---|---|---|---|---|---|---|
| AGE | – | 0.18 | − 0.02 | − 0.12 | − 0.25 | 0.01 | − 0.14 | 0.08 | − 0.18 | |
| DD | – | 0.27 | 0.10 | 0.22 | − 0.10 | 0.07 | 0.03 | − 0.12 | ||
| EDSS@base | – | 0.11 | 0.15 | 0.08 | 0.07 | 0.02 | − 0.19 | − 0.16 | ||
| T2LL | – | − 0.25 | 0.04 | − 0.26 | − 0.15 | 0.19 | 0.18 | |||
| FA-NAWM | – | − | − 0.01 | − 0.20 | 0.13 | 0.29 | ||||
| MD-NAWM | – | − 0.11 | − 0.05 | 0.06 | 0.15 | |||||
| Thalamic volume | – | 0.28 | − 0.05 | |||||||
| GM volume | – | − 0.02 | ||||||||
| WM volume | – | 0.14 | ||||||||
| Cerebellar volume | – |
Multiple correlation analysis of clinical and radiological features. Spearman correlation coefficients are reported, significant correlations are highlighted in bold font and relative p-values are shown between round brackets
DD disease duration, EDSS expanded disability status scale, T2LL T2 lesion load, FA-NAWM fractional anisotropy of normal appearing white matter, MD-NAWM mean diffusivity of normal appearing white matter
Correlation matrix, Site 2 sample
| AGE | DD | EDSS@base | T2LL | FA-NAWM | MD-NAWM | Thalamic volume | GM volume | WM volume | Cerebellar volume | |
|---|---|---|---|---|---|---|---|---|---|---|
| AGE | – | − 0.07 | − 0.14 | 0.10 | 0.00 | 0.03 | − 0.11 | 0.04 | − 0.31 | |
| DD | – | 0.13 | 0.26 | − 0.09 | 0.02 | 0.05 | − 0.21 | 0.14 | 0.11 | |
EDSS @base | – | 0.23 | 0.17 | 0.23 | 0.13 | − 0.13 | 0.00 | − 0.23 | ||
| T2LL | – | − 0.17 | − 0.02 | − 0.42 | 0.22 | 0.19 | − 0.10 | |||
FA- NAWM | – | − | 0.30 | − 0.01 | − 0.14 | 0.16 | ||||
MD- NAWM | – | 0.20 | 0.01 | − 0.10 | − 0.06 | |||||
| Thalamic Volume | – | 0.23 | 0.09 | |||||||
GM Volume | – | − 0.14 | ||||||||
WM Volume | – | − 0.01 | ||||||||
Cerebellar Volume | – |
Multiple correlation analysis of clinical and radiological features. Spearman correlation coefficients are reported, significant correlations are highlighted in bold font and relative p-values are shown between round brackets
DD disease duration, EDSS expanded disability status scale, T2LL T2 lesion load, FA-NAWM fractional anisotropy of normal appearing white matter, MD-NAWM mean diffusivity of normal appearing white matter
Fig. 2Metrics of classifier built on clinical and radiological features. Histogram of accuracy (ACC) and area under the true positive versus true negative rate curve (AUC), sensitivity and specificity obtained from the 150 classifiers, surviving the Out-of-Bag test, performed on the sample features. On the y-axis number of classifiers is displayed
Metrics
| Accuracy | AUC | Sensitivity | Specificity | ||
|---|---|---|---|---|---|
| ALL | 162 | 0.79 | 0.81 | 0.90 | 0.71 |
| Radiological | 329 | 0.92 | 0.92 | 0.92 | 0.91 |
| Clinical | 128 | 0.71 | 0.72 | 0.69 | 0.75 |
Accuracy, area under the true positive versus true negative curve (AUC), sensitivity and specificity of the best performing machine learning classifier built on all clinical and radiological features, and either on clinical or radiological features. N represents the number of classifiers surviving the Out-of-Bag test
Fig. 3Metrics of classifier built on radiological/clinical features. Histograms of accuracy (ACC) and area under the true positive versus true negative rate curve (AUC), sensitivity and specificity, surviving the Out-of-Bag test obtained from the 309 classifiers built on neuroradiological features (top) and the 128 classifiers built on clinical features (bottom). On the y-axis number of classifiers is displayed
Fig. 4Important features frequency. Figure showing, for each feature, the percentage of time (i.e. classifiers) more important than the random feature in predicting disability progression in our MS group. EDSSbase expanded disability status scale at baseline, TimeFU time between first visit and follow-up, CrbV cerebellar gray matter volume, GMV gray matter volume, FA-NAWM fractional anisotropy of normal appearing white matter