Acuity VEP: improved with machine learning.

PURPOSE: Acuity-VEP approaches basically all use the information obtained across a number of check sizes (or spatial frequencies) to derive a measure of acuity. Amplitude is always used, sometimes combined with phase or a noise measure. In our approach, we employ steady-state brief-onset low-contrast checkerboard stimulation and obtain amplitude and significance for six different check sizes, yielding 12 numbers. The rule-based "heuristic algorithm" (Bach et al. in Br J Ophthalmol 92:396-403, 2008. https://doi.org/10.1136/bjo.2007.130245 ) is successful in over 95% with a limit of agreement (LoA) of ± 0.3LogMAR between behavioral and objective acuity for 109 cases. We here aimed to test whether machine learning techniques with this relatively small dataset could achieve a similar LoA.
METHODS: Given recent advances in machine learning (ML), we applied a wide class of ML algorithms to this dataset. This was done within the "caret" framework of R using altogether 89 methods, of which rule-based and multiple regression approaches performed best. For cross-validation, using a jackknife (leave-one-out) approach, we predicted each case based on an ML model having been trained on all remaining 108 cases.
RESULTS: The ML approach predicted visual acuity well across many different types of ML algorithms. Using amplitude values only (discarding the p values) improved the outcome. Nearly half of the tested ML algorithms achieved an LoA better than the heuristic algorithm; several "Random Forest"- or "multiple regression"-type algorithms achieved an LoA of below ± 0.3. In the cases where the heuristic approach failed, acuity was predicted successfully. We then applied the ML model trained with the Bach et al. [1] dataset to a new dataset from 2018 (78 cases) and found both for the heuristic algorithm and for the ML approach an LoA of ± 0.259, a nearly one-line improvement.
CONCLUSIONS: The ML approach appears to be a useful alternative to rule-based analysis of acuity-VEP data. The achieved accuracy is comparable or better (in no case the ML-based acuity differed more than ± 0.29 LogMAR from behavioral acuity), and testability is higher, nearly 100%. Possible pitfalls are examined.