| Literature DB >> 28590432 |
Dylan Drover1, Jennifer Howcroft2, Jonathan Kofman3, Edward D Lemaire4,5.
Abstract
Faller classification in elderly populations can facilitate preventative care before a fall occurs. A novel wearable-sensor based faller classification method for the elderly was developed using accelerometer-based features from straight walking and turns. Seventy-six older individuals (74.15 ± 7.0 years), categorized as prospective fallers and non-fallers, completed a six-minute walk test with accelerometers attached to their lower legs and pelvis. After segmenting straight and turn sections, cross validation tests were conducted on straight and turn walking features to assess classification performance. The best "classifier model-feature selector" combination used turn data, random forest classifier, and select-5-best feature selector (73.4% accuracy, 60.5% sensitivity, 82.0% specificity, and 0.44 Matthew's Correlation Coefficient (MCC)). Using only the most frequently occurring features, a feature subset (minimum of anterior-posterior ratio of even/odd harmonics for right shank, standard deviation (SD) of anterior left shank acceleration SD, SD of mean anterior left shank acceleration, maximum of medial-lateral first quartile of Fourier transform (FQFFT) for lower back, maximum of anterior-posterior FQFFT for lower back) achieved better classification results, with 77.3% accuracy, 66.1% sensitivity, 84.7% specificity, and 0.52 MCC score. All classification performance metrics improved when turn data was used for faller classification, compared to straight walking data. Combining turn and straight walking features decreased performance metrics compared to turn features for similar classifier model-feature selector combinations.Entities:
Keywords: accelerometer; elderly; faller classification; faller prediction; falls; feature selection; machine learning; prospective fallers; wearable sensors
Mesh:
Year: 2017 PMID: 28590432 PMCID: PMC5492293 DOI: 10.3390/s17061321
Source DB: PubMed Journal: Sensors (Basel) ISSN: 1424-8220 Impact factor: 3.576
Figure 1Vertical accelerometer signals from right shank (RS) and left shank (LS) with segmented turn.
Figure 2Section of left and right vertical accelerometer signals. Periodic drops in vertical acceleration magnitude locate turns in the 6MWT.
Figure 3Model performance evaluations.
Figure 4Overview of data processing and classification process.
Figure 5Test III procedure for testing most frequently occurring feature subsets.
Straight-walking section five-fold cross validation (5FCV) results. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, S5B: select-5-best, SEL: false positive and discovery rate method, RFE: recursive feature eliminator, RF: random forest, kNN: k-nearest neighbour, SVM: support vector machine, linear: linear kernel, poly: polynomial kernel.
| Classifier, Feature Selector | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 Score | MCC | Rank Sum |
|---|---|---|---|---|---|---|---|---|
| RF S5B | 62.0 | 46.4 | 72.1 | 52.0 | 67.4 | 0.49 | 0.19 | 24 |
| SVM (poly = 3) S5B | 56.3 | 78.6 | 41.9 | 46.8 | 75.0 | 0.59 | 0.21 | 26 |
| RF SEL | 57.7 | 46.4 | 65.1 | 46.4 | 65.1 | 0.46 | 0.12 | 36 |
| RF RFE | 62.0 | 32.1 | 81.4 | 52.9 | 64.8 | 0.40 | 0.16 | 44 |
| SVM (poly = 5) SEL | 54.9 | 57.1 | 53.5 | 44.4 | 65.7 | 0.50 | 0.10 | 46 |
| SVM (poly = 3) RFE | 52.1 | 71.4 | 39.5 | 43.5 | 68.0 | 0.54 | 0.11 | 47 |
| SVM (poly = 3) SEL | 52.1 | 71.4 | 39.5 | 43.5 | 68.0 | 0.54 | 0.11 | 47 |
| 56.3 | 42.9 | 65.1 | 44.4 | 63.6 | 0.44 | 0.08 | 51 | |
| 54.9 | 50.0 | 58.1 | 43.8 | 64.1 | 0.47 | 0.08 | 51 | |
| 56.3 | 39.3 | 67.4 | 44.0 | 63.0 | 0.42 | 0.07 | 60 | |
| SVM (linear) S5B | 53.5 | 50.0 | 55.8 | 42.4 | 63.2 | 0.46 | 0.06 | 63 |
| SVM (linear) SEL | 53.5 | 50.0 | 55.8 | 42.4 | 63.2 | 0.46 | 0.06 | 64 |
| SVM (poly = 5) RFE | 52.1 | 46.4 | 55.8 | 40.6 | 61.5 | 0.43 | 0.02 | 78 |
| SVM (linear) RFE | 52.1 | 39.3 | 60.5 | 39.3 | 60.5 | 0.39 | 0.00 | 85 |
| 50.7 | 35.7 | 60.5 | 37.0 | 59.1 | 0.36 | −0.04 | 97 | |
| SVM (poly = 5) S5B | 49.3 | 39.3 | 55.8 | 36.7 | 58.5 | 0.38 | −0.05 | 100 |
| 47.9 | 35.7 | 55.8 | 34.5 | 57.1 | 0.35 | −0.08 | 109 | |
| 46.5 | 35.7 | 53.5 | 33.3 | 56.1 | 0.34 | −0.11 | 119 |
Turn section five-fold cross validation (5FCV) results. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, S5B: select-5-best method, SEL: false positive and discovery rate method, RFE: recursive feature eliminator, RF: random forest, kNN: k-nearest neighbour, SVM: support vector machine, linear: linear kernel, poly: polynomial kernel.
| Classifier, Feature Selector | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 Score | MCC | Rank Sum |
|---|---|---|---|---|---|---|---|---|
| RF S5B | 77.5 | 67.9 | 83.7 | 73.1 | 80.0 | 0.70 | 0.52 | 12 |
| RF SEL | 77.5 | 64.3 | 86.0 | 75.0 | 78.7 | 0.69 | 0.52 | 14 |
| RF RFE | 69.0 | 53.6 | 79.1 | 62.5 | 72.3 | 0.58 | 0.34 | 38 |
| 69.0 | 50.0 | 81.4 | 63.6 | 71.4 | 0.56 | 0.33 | 45 | |
| 71.8 | 42.9 | 90.7 | 75.0 | 70.9 | 0.55 | 0.39 | 46 | |
| SVM (linear) S5B | 67.6 | 53.6 | 76.7 | 60.0 | 71.7 | 0.57 | 0.31 | 50 |
| SVM (linear) SEL | 66.2 | 57.1 | 72.1 | 57.1 | 72.1 | 0.57 | 0.29 | 55 |
| 67.6 | 50.0 | 79.1 | 60.9 | 70.8 | 0.55 | 0.30 | 57 | |
| SVM (poly = 3) RFE | 62.0 | 67.9 | 58.1 | 51.4 | 73.5 | 0.58 | 0.25 | 58 |
| 66.2 | 50.0 | 76.7 | 58.3 | 70.2 | 0.54 | 0.28 | 70 | |
| SVM (poly = 3) SEL | 60.6 | 64.3 | 58.1 | 50.0 | 71.4 | 0.56 | 0.22 | 73 |
| SVM (poly = 5) SEL | 54.9 | 78.6 | 39.5 | 45.8 | 73.9 | 0.58 | 0.19 | 76 |
| SVM (poly = 5) S5B | 60.6 | 60.7 | 60.5 | 50.0 | 70.3 | 0.55 | 0.21 | 81 |
| 63.4 | 35.7 | 81.4 | 55.6 | 66.0 | 0.43 | 0.19 | 89 | |
| SVM (linear) RFE | 60.6 | 50.0 | 67.4 | 50.0 | 67.4 | 0.50 | 0.17 | 94 |
| SVM (poly = 3) S5B | 59.2 | 57.1 | 60.5 | 48.5 | 68.4 | 0.52 | 0.17 | 97 |
| 62.0 | 39.3 | 76.7 | 52.4 | 66.0 | 0.45 | 0.17 | 98 | |
| SVM (poly = 5) RFE | 57.7 | 46.4 | 65.1 | 46.4 | 65.1 | 0.46 | 0.12 | 114 |
Straight-walking section results for 2500-iteration random-shuffle-split cross validation (2500-RSS), ordered by ranked performance. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, S5B: select-5-best method, SEL: false positive and discovery rate method, RFE: recursive feature eliminator, RF: random forest, kNN: k-nearest neighbour, SVM: support vector machine, linear: linear kernel, poly: polynomial kernel, : mean, SD: standard deviation, CI: 95% confidence interval.
| Classifier, Feature Selection | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 | MCC | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | ||||||||
| 55.5 | 12.0 | 0.47 | 46.1 | 21.2 | 0.83 | 61.8 | 16.2 | 0.64 | 44.6 | 16.9 | 0.66 | 63.2 | 11.5 | 0.45 | 0.45 | 0.17 | 0.007 | 0.08 | 0.26 | 0.010 | |
| RF S5B | 56.2 | 11.4 | 0.45 | 39.8 | 20.3 | 0.80 | 67.2 | 15.9 | 0.62 | 44.7 | 19.6 | 0.77 | 62.6 | 9.9 | 0.39 | 0.42 | 0.18 | 0.007 | 0.07 | 0.26 | 0.010 |
| RF SEL | 56.9 | 11.2 | 0.44 | 34.5 | 20.3 | 0.79 | 71.9 | 18.3 | 0.72 | 45.0 | 25.2 | 0.99 | 62.2 | 8.7 | 0.34 | 0.39 | 0.18 | 0.007 | 0.07 | 0.30 | 0.012 |
| SVM (poly = 3) SEL | 51.7 | 11.1 | 0.43 | 59.7 | 33.6 | 1.32 | 46.4 | 30.3 | 1.19 | 42.6 | 18.8 | 0.74 | 63.3 | 25.6 | 1.00 | 0.50 | 0.20 | 0.008 | 0.06 | 0.39 | 0.015 |
| 55.0 | 11.8 | 0.46 | 43.6 | 21.8 | 0.85 | 62.7 | 17.0 | 0.67 | 43.8 | 18.3 | 0.72 | 62.5 | 11.2 | 0.44 | 0.44 | 0.18 | 0.007 | 0.06 | 0.26 | 0.010 | |
| SVM (linear) SEL | 53.4 | 12.1 | 0.48 | 50.3 | 23.7 | 0.93 | 55.5 | 23.7 | 0.93 | 43.0 | 17.3 | 0.68 | 62.6 | 15.9 | 0.62 | 0.46 | 0.16 | 0.006 | 0.06 | 0.30 | 0.012 |
| SVM (linear) S5B | 50.9 | 11.9 | 0.47 | 53.6 | 25.4 | 0.99 | 49.1 | 19.3 | 0.76 | 41.3 | 15.0 | 0.59 | 61.4 | 16.4 | 0.64 | 0.47 | 0.17 | 0.007 | 0.03 | 0.27 | 0.011 |
| 54.0 | 11.4 | 0.45 | 37.5 | 19.9 | 0.78 | 65.1 | 17.3 | 0.68 | 41.7 | 19.6 | 0.77 | 61.0 | 9.7 | 0.38 | 0.39 | 0.17 | 0.007 | 0.03 | 0.25 | 0.010 | |
| SVM (poly = 3) S5B | 48.7 | 10.4 | 0.41 | 61.6 | 33.6 | 1.32 | 40.1 | 26.2 | 1.03 | 40.7 | 16.1 | 0.63 | 61.0 | 26.4 | 1.03 | 0.49 | 0.20 | 0.008 | 0.02 | 0.35 | 0.014 |
| 53.8 | 10.8 | 0.42 | 34.6 | 19.9 | 0.78 | 66.6 | 17.7 | 0.69 | 40.8 | 20.5 | 0.80 | 60.4 | 9.2 | 0.36 | 0.37 | 0.17 | 0.007 | 0.01 | 0.26 | 0.010 | |
Turn section results for 2500-iteration random-shuffle-split cross validation (2500-RSS), ordered by ranked performance. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, S5B: select-5-best method, SEL: false positive and discovery rate method, RFE: recursive feature eliminator, RF: random forest, kNN: k-nearest neighbour, SVM: support vector machine, linear: linear kernel, poly: polynomial kernel, : mean, SD: standard deviation, CI: 95% confidence interval.
| Classifier, Feature Selector | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 | MCC | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | ||||||||
| RF S5B | 73.4 | 10.6 | 0.42 | 60.5 | 20.5 | 0.81 | 82.0 | 12.8 | 0.50 | 69.1 | 18.2 | 0.71 | 75.7 | 10.2 | 0.40 | 0.65 | 0.17 | 0.007 | 0.44 | 0.24 | 0.009 |
| RF SEL | 71.6 | 10.9 | 0.43 | 58.3 | 20.7 | 0.81 | 80.4 | 13.3 | 0.52 | 66.5 | 18.5 | 0.72 | 74.3 | 10.3 | 0.41 | 0.62 | 0.17 | 0.007 | 0.40 | 0.24 | 0.010 |
| 69.2 | 11.2 | 0.44 | 49.0 | 21.4 | 0.84 | 82.7 | 13.3 | 0.52 | 65.3 | 22.5 | 0.88 | 70.8 | 9.7 | 0.38 | 0.56 | 0.19 | 0.008 | 0.34 | 0.27 | 0.011 | |
| 68.0 | 11.2 | 0.44 | 50.8 | 20.7 | 0.81 | 79.6 | 13.9 | 0.55 | 62.4 | 20.9 | 0.82 | 70.8 | 9.8 | 0.39 | 0.56 | 0.18 | 0.007 | 0.32 | 0.26 | 0.010 | |
| SVM (linear) S5B | 66.7 | 11.7 | 0.46 | 57.6 | 20.8 | 0.82 | 72.8 | 16.0 | 0.63 | 58.5 | 17.7 | 0.69 | 72.0 | 11.7 | 0.46 | 0.58 | 0.16 | 0.006 | 0.30 | 0.25 | 0.010 |
| SVM (linear) SEL | 64.7 | 13.0 | 0.51 | 57.6 | 24.2 | 0.95 | 69.5 | 17.9 | 0.70 | 55.7 | 19.3 | 0.76 | 71.1 | 14.4 | 0.56 | 0.57 | 0.19 | 0.007 | 0.27 | 0.31 | 0.012 |
| 67.2 | 12.5 | 0.49 | 48.7 | 21.6 | 0.85 | 79.5 | 14.9 | 0.58 | 61.3 | 23.2 | 0.91 | 69.9 | 10.5 | 0.41 | 0.54 | 0.20 | 0.008 | 0.30 | 0.29 | 0.012 | |
| 66.8 | 12.7 | 0.50 | 50.0 | 21.3 | 0.83 | 78.0 | 15.2 | 0.60 | 60.3 | 22.2 | 0.87 | 70.1 | 10.8 | 0.42 | 0.55 | 0.19 | 0.008 | 0.29 | 0.29 | 0.011 | |
| SVM (poly = 3) SEL | 61.8 | 13.1 | 0.51 | 50.7 | 25.3 | 0.99 | 69.2 | 24.8 | 0.97 | 52.3 | 25.1 | 0.98 | 67.8 | 15.4 | 0.61 | 0.51 | 0.18 | 0.007 | 0.20 | 0.33 | 0.013 |
| SVM (poly = 3) S5B | 60.7 | 13.8 | 0.54 | 55.7 | 23.6 | 0.93 | 64.1 | 22.3 | 0.87 | 50.8 | 20.0 | 0.78 | 68.4 | 15.8 | 0.62 | 0.53 | 0.17 | 0.007 | 0.20 | 0.30 | 0.012 |
Figure 6Histogram of selected straight-walking feature frequency above 8% (200) of 2500 total selections using the combination of Select False Positive Rate and Select False Discovery Rate methods (SEL) for 2500 random-shuffle-split iterations.
Figure 7Histogram of selected straight-walking feature frequency above 8% (200) of 2500 total selections using the select-5-best (S5B) method for 2500 random-shuffle-split iterations.
Figure 8Histogram of selected turn-based feature frequency above 8% (200) of 2500 total selections using the combination of Select False Positive Rate and Select False Discovery Rate methods (SEL) for 2500 random-shuffle-split iterations.
Figure 9Histogram of selected turn-based feature frequency above 8% (200) of 2500 total selections using the select-5-best (S5B) method for 2500 random-shuffle-split iterations.
Most frequently occurring (MFO) feature subsets for straight-walking section results and 3NN classifier using 2500-iteration random-shuffle-split cross validation (2500-RSS), ordered by ranked performance. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, : mean, SD: standard deviation, CI: 95% confidence interval.
| # Features | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 | MCC | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | ||||||||
| 5 | 64.1 | 10.8 | 0.42 | 59.9 | 19.2 | 0.75 | 66.9 | 14.5 | 0.57 | 54.7 | 14.3 | 0.56 | 71.4 | 11.0 | 0.43 | 0.57 | 0.14 | 0.006 | 0.26 | 0.23 | 0.009 |
| 3 | 63.1 | 11.3 | 0.44 | 61.2 | 20.0 | 0.78 | 64.4 | 14.9 | 0.59 | 53.4 | 14.0 | 0.55 | 71.3 | 11.9 | 0.47 | 0.57 | 0.15 | 0.006 | 0.25 | 0.24 | 0.009 |
| 4 | 62.2 | 10.8 | 0.42 | 57.7 | 18.9 | 0.74 | 65.2 | 15.0 | 0.59 | 52.5 | 14.6 | 0.57 | 69.8 | 10.7 | 0.42 | 0.55 | 0.14 | 0.006 | 0.23 | 0.23 | 0.009 |
| 9 | 61.5 | 10.4 | 0.41 | 42.1 | 18.8 | 0.74 | 74.5 | 14.0 | 0.55 | 52.4 | 20.0 | 0.79 | 65.9 | 8.5 | 0.33 | 0.47 | 0.17 | 0.007 | 0.17 | 0.24 | 0.009 |
| 10 | 60.7 | 11.1 | 0.43 | 44.7 | 19.9 | 0.78 | 71.4 | 14.6 | 0.57 | 51.1 | 19.1 | 0.75 | 66.0 | 9.5 | 0.37 | 0.48 | 0.17 | 0.007 | 0.17 | 0.25 | 0.010 |
| 6 | 60.6 | 12.3 | 0.48 | 56.1 | 20.1 | 0.79 | 63.6 | 16.7 | 0.66 | 50.7 | 16.0 | 0.63 | 68.5 | 12.2 | 0.48 | 0.53 | 0.16 | 0.006 | 0.20 | 0.26 | 0.010 |
| 2 | 60.0 | 11.5 | 0.45 | 57.2 | 19.3 | 0.76 | 61.8 | 15.9 | 0.62 | 50.0 | 14.5 | 0.57 | 68.4 | 11.7 | 0.46 | 0.53 | 0.15 | 0.006 | 0.19 | 0.24 | 0.009 |
| 8 | 60.6 | 10.3 | 0.40 | 38.5 | 18.7 | 0.73 | 75.4 | 13.8 | 0.54 | 51.1 | 21.6 | 0.85 | 64.8 | 8.2 | 0.32 | 0.44 | 0.17 | 0.007 | 0.15 | 0.25 | 0.010 |
| 11 | 59.6 | 11.1 | 0.43 | 41.9 | 19.4 | 0.76 | 71.4 | 14.9 | 0.59 | 49.4 | 19.7 | 0.77 | 64.8 | 9.3 | 0.37 | 0.45 | 0.17 | 0.007 | 0.14 | 0.25 | 0.010 |
| 7 | 59.2 | 11.2 | 0.44 | 44.0 | 18.8 | 0.74 | 69.3 | 15.1 | 0.59 | 48.9 | 18.6 | 0.73 | 65.0 | 9.4 | 0.37 | 0.46 | 0.16 | 0.006 | 0.14 | 0.24 | 0.010 |
| 1 | 57.0 | 11.0 | 0.43 | 50.2 | 20.1 | 0.79 | 61.5 | 15.7 | 0.62 | 46.5 | 14.8 | 0.58 | 64.9 | 10.9 | 0.43 | 0.48 | 0.15 | 0.006 | 0.12 | 0.24 | 0.009 |
| 13 | 57.8 | 10.9 | 0.43 | 37.4 | 19.1 | 0.75 | 71.4 | 14.5 | 0.57 | 46.6 | 20.4 | 0.80 | 63.1 | 8.9 | 0.35 | 0.41 | 0.17 | 0.007 | 0.09 | 0.25 | 0.010 |
| 14 | 57.6 | 10.5 | 0.41 | 37.6 | 18.9 | 0.74 | 70.9 | 14.5 | 0.57 | 46.3 | 19.7 | 0.77 | 63.0 | 8.5 | 0.33 | 0.42 | 0.17 | 0.007 | 0.09 | 0.24 | 0.010 |
| 12 | 57.0 | 10.5 | 0.41 | 36.5 | 19.1 | 0.75 | 70.6 | 14.0 | 0.55 | 45.3 | 20.2 | 0.79 | 62.5 | 8.5 | 0.33 | 0.40 | 0.17 | 0.007 | 0.07 | 0.25 | 0.010 |
Most frequently occurring (MFO) feature subsets for turn section results and random forest classifier using 2500-iteration random-shuffle-split cross validation (2500-RSS), ordered by ranked performance. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, : mean, SD: standard deviation, CI: 95% confidence interval.
| # Features | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 | MCC | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | ||||||||
| 5 | 77.3 | 9.1 | 0.36 | 66.1 | 19.6 | 0.77 | 84.7 | 11.4 | 0.45 | 74.3 | 15.5 | 0.61 | 79.0 | 9.7 | 0.38 | 0.70 | 0.15 | 0.006 | 0.52 | 0.20 | 0.008 |
| 6 | 77.1 | 9.4 | 0.37 | 66.2 | 19.5 | 0.76 | 84.4 | 11.7 | 0.46 | 73.9 | 15.9 | 0.62 | 78.9 | 9.7 | 0.38 | 0.70 | 0.15 | 0.006 | 0.52 | 0.21 | 0.008 |
| 3 | 77.0 | 9.6 | 0.38 | 67.7 | 18.9 | 0.74 | 83.2 | 12.2 | 0.48 | 72.9 | 15.7 | 0.62 | 79.5 | 9.7 | 0.38 | 0.70 | 0.14 | 0.006 | 0.52 | 0.21 | 0.008 |
| 9 | 76.3 | 9.6 | 0.38 | 63.3 | 19.8 | 0.78 | 84.9 | 11.6 | 0.46 | 73.6 | 16.7 | 0.66 | 77.6 | 9.6 | 0.38 | 0.68 | 0.15 | 0.006 | 0.50 | 0.22 | 0.009 |
| 2 | 76.4 | 9.4 | 0.37 | 65.9 | 18.9 | 0.74 | 83.4 | 12.0 | 0.47 | 72.6 | 15.8 | 0.62 | 78.6 | 9.6 | 0.38 | 0.69 | 0.14 | 0.006 | 0.50 | 0.20 | 0.008 |
| 7 | 75.8 | 9.6 | 0.38 | 62.4 | 19.5 | 0.76 | 84.7 | 12.0 | 0.47 | 73.2 | 16.7 | 0.66 | 77.2 | 9.5 | 0.37 | 0.67 | 0.15 | 0.006 | 0.49 | 0.21 | 0.008 |
| 8 | 75.7 | 9.7 | 0.38 | 62.4 | 19.7 | 0.77 | 84.5 | 12.0 | 0.47 | 72.9 | 16.9 | 0.66 | 77.1 | 9.6 | 0.38 | 0.67 | 0.15 | 0.006 | 0.48 | 0.22 | 0.009 |
| 4 | 75.5 | 9.5 | 0.37 | 63.3 | 19.5 | 0.76 | 83.7 | 11.9 | 0.47 | 72.2 | 16.5 | 0.65 | 77.4 | 9.6 | 0.38 | 0.67 | 0.15 | 0.006 | 0.48 | 0.21 | 0.008 |
| 1 | 75.3 | 9.4 | 0.37 | 61.5 | 19.5 | 0.76 | 84.6 | 11.7 | 0.46 | 72.7 | 16.8 | 0.66 | 76.7 | 9.3 | 0.36 | 0.67 | 0.15 | 0.006 | 0.48 | 0.21 | 0.008 |
Combined straight and turn-walking feature results for 2500-iteration random-shuffle-split cross validation (2500-RSS), ordered by ranked performance. PPV: positive predictive value, NPV: negative predictive value, MCC: Matthews correlation coefficient, S5B: select-5-best method, SEL: false positive and discovery rate method, RFE: recursive feature eliminator, RF: random forest, kNN: k-nearest neighbour, SVM: support vector machine, linear: linear kernel, : mean, SD: standard deviation, CI: 95% confidence interval.
| Classifier, Feature Selection | Accuracy (%) | Sensitivity (%) | Specificity (%) | PPV (%) | NPV (%) | F1 | MCC | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | SD | CI | ||||||||
| RF S5B | 71.6 | 10.8 | 0.42 | 57.5 | 20.8 | 0.82 | 81.1 | 13.2 | 0.52 | 66.9 | 18.9 | 0.74 | 74.1 | 10.2 | 0.40 | 0.62 | 0.17 | 0.007 | 0.40 | 0.25 | 0.010 |
| RF SEL | 69.5 | 11.7 | 0.46 | 54.3 | 21.1 | 0.83 | 79.7 | 14.5 | 0.57 | 64.1 | 20.7 | 0.81 | 72.3 | 10.4 | 0.41 | 0.59 | 0.18 | 0.007 | 0.35 | 0.27 | 0.010 |
| 67.4 | 11.2 | 0.44 | 48.6 | 21.3 | 0.84 | 80.0 | 13.9 | 0.55 | 61.8 | 21.6 | 0.85 | 70.0 | 9.9 | 0.39 | 0.54 | 0.19 | 0.007 | 0.30 | 0.27 | 0.011 | |
| SVM (linear) S5B | 65.7 | 11.6 | 0.45 | 56.1 | 21.0 | 0.83 | 72.1 | 15.9 | 0.62 | 57.3 | 17.7 | 0.70 | 71.1 | 11.5 | 0.45 | 0.57 | 0.16 | 0.006 | 0.28 | 0.25 | 0.010 |
| 65.9 | 11.4 | 0.45 | 49.3 | 20.6 | 0.81 | 77.0 | 14.4 | 0.56 | 58.8 | 20.6 | 0.81 | 69.5 | 10.0 | 0.39 | 0.54 | 0.18 | 0.007 | 0.27 | 0.26 | 0.010 | |
| SVM (linear) SEL | 63.7 | 12.5 | 0.49 | 54.9 | 23.8 | 0.94 | 69.6 | 17.9 | 0.70 | 54.6 | 19.9 | 0.78 | 69.8 | 13.5 | 0.53 | 0.55 | 0.19 | 0.007 | 0.24 | 0.30 | 0.012 |
| 65.3 | 12.7 | 0.50 | 49.0 | 21.1 | 0.83 | 76.2 | 15.3 | 0.60 | 57.9 | 21.8 | 0.85 | 69.2 | 10.8 | 0.42 | 0.53 | 0.19 | 0.007 | 0.26 | 0.29 | 0.011 | |
| 65.4 | 12.5 | 0.49 | 47.2 | 22.0 | 0.86 | 77.5 | 15.1 | 0.59 | 58.3 | 23.3 | 0.91 | 68.8 | 10.7 | 0.42 | 0.52 | 0.20 | 0.008 | 0.26 | 0.29 | 0.012 | |