Jan Yperman1,2,3, Thijs Becker4, Dirk Valkenborg4, Veronica Popescu5,6, Niels Hellings7, Bart Van Wijmeersch5,6, Liesbet M Peeters4,7. 1. Theoretical Physics, Hasselt University, Diepenbeek, Belgium. jan_yperman@uhasselt.be. 2. I-Biostat, Data Science Institute, Hasselt University,, Diepenbeek, Belgium. jan_yperman@uhasselt.be. 3. Department of Immunology, Biomedical Research Institute, Hasselt University, Diepenbeek, 3590, Belgium. jan_yperman@uhasselt.be. 4. I-Biostat, Data Science Institute, Hasselt University,, Diepenbeek, Belgium. 5. Rehabilitation and MS-Center, Pelt, 3900, Belgium. 6. REVAL Rehabilitation Research Center, BIOMED, Faculty of Rehabilitation Sciences, Hasselt University, Hasselt, Belgium. 7. Department of Immunology, Biomedical Research Institute, Hasselt University, Diepenbeek, 3590, Belgium.
Abstract
BACKGROUND: Evoked potentials (EPs) are a measure of the conductivity of the central nervous system. They are used to monitor disease progression of multiple sclerosis patients. Previous studies only extracted a few variables from the EPs, which are often further condensed into a single variable: the EP score. We perform a machine learning analysis of motor EP that uses the whole time series, instead of a few variables, to predict disability progression after two years. Obtaining realistic performance estimates of this task has been difficult because of small data set sizes. We recently extracted a dataset of EPs from the Rehabiliation & MS Center in Overpelt, Belgium. Our data set is large enough to obtain, for the first time, a performance estimate on an independent test set containing different patients. METHODS: We extracted a large number of time series features from the motor EPs with the highly comparative time series analysis software package. Mutual information with the target and the Boruta method are used to find features which contain information not included in the features studied in the literature. We use random forests (RF) and logistic regression (LR) classifiers to predict disability progression after two years. Statistical significance of the performance increase when adding extra features is checked. RESULTS: Including extra time series features in motor EPs leads to a statistically significant improvement compared to using only the known features, although the effect is limited in magnitude (ΔAUC = 0.02 for RF and ΔAUC = 0.05 for LR). RF with extra time series features obtains the best performance (AUC = 0.75±0.07 (mean and standard deviation)), which is good considering the limited number of biomarkers in the model. RF (a nonlinear classifier) outperforms LR (a linear classifier). CONCLUSIONS: Using machine learning methods on EPs shows promising predictive performance. Using additional EP time series features beyond those already in use leads to a modest increase in performance. Larger datasets, preferably multi-center, are needed for further research. Given a large enough dataset, these models may be used to support clinicians in their decision making process regarding future treatment.
BACKGROUND: Evoked potentials (EPs) are a measure of the conductivity of the central nervous system. They are used to monitor disease progression of multiple sclerosispatients. Previous studies only extracted a few variables from the EPs, which are often further condensed into a single variable: the EP score. We perform a machine learning analysis of motor EP that uses the whole time series, instead of a few variables, to predict disability progression after two years. Obtaining realistic performance estimates of this task has been difficult because of small data set sizes. We recently extracted a dataset of EPs from the Rehabiliation & MS Center in Overpelt, Belgium. Our data set is large enough to obtain, for the first time, a performance estimate on an independent test set containing different patients. METHODS: We extracted a large number of time series features from the motor EPs with the highly comparative time series analysis software package. Mutual information with the target and the Boruta method are used to find features which contain information not included in the features studied in the literature. We use random forests (RF) and logistic regression (LR) classifiers to predict disability progression after two years. Statistical significance of the performance increase when adding extra features is checked. RESULTS: Including extra time series features in motor EPs leads to a statistically significant improvement compared to using only the known features, although the effect is limited in magnitude (ΔAUC = 0.02 for RF and ΔAUC = 0.05 for LR). RF with extra time series features obtains the best performance (AUC = 0.75±0.07 (mean and standard deviation)), which is good considering the limited number of biomarkers in the model. RF (a nonlinear classifier) outperforms LR (a linear classifier). CONCLUSIONS: Using machine learning methods on EPs shows promising predictive performance. Using additional EP time series features beyond those already in use leads to a modest increase in performance. Larger datasets, preferably multi-center, are needed for further research. Given a large enough dataset, these models may be used to support clinicians in their decision making process regarding future treatment.
Authors: Stijn Denissen; Oliver Y Chén; Johan De Mey; Maarten De Vos; Jeroen Van Schependom; Diana Maria Sima; Guy Nagels Journal: J Pers Med Date: 2021-12-11