INTRODUCTION: Changes in speech have the potential to provide important information on the diagnosis and progression of various neurological diseases. Many researchers have relied on open-source speech features to develop algorithms for measuring speech changes in clinical populations as they are convenient and easy to use. However, the repeatability of open-source features in the context of neurological diseases has not been studied. METHODS: We used a longitudinal sample of healthy controls, individuals with amyotrophic lateral sclerosis, and individuals with suspected frontotemporal dementia, and we evaluated the repeatability of acoustic and language features separately on these 3 data sets. RESULTS: Repeatability was evaluated using intraclass correlation (ICC) and the within-subjects coefficient of variation (WSCV). In 3 sets of tasks, the median ICC were between 0.02 and 0.55, and the median WSCV were between 29 and 79%. CONCLUSION: Our results demonstrate that the repeatability of speech features extracted using open-source tool kits is low. Researchers should exercise caution when developing digital health models with open-source speech features. We provide a detailed summary of feature-by-feature repeatability results (ICC, WSCV, SE of measurement, limits of agreement for WSCV, and minimal detectable change) in the online supplementary material so that researchers may incorporate repeatability information into the models they develop.
INTRODUCTION: Changes in speech have the potential to provide important information on the diagnosis and progression of various neurological diseases. Many researchers have relied on open-source speech features to develop algorithms for measuring speech changes in clinical populations as they are convenient and easy to use. However, the repeatability of open-source features in the context of neurological diseases has not been studied. METHODS: We used a longitudinal sample of healthy controls, individuals with amyotrophic lateral sclerosis, and individuals with suspected frontotemporal dementia, and we evaluated the repeatability of acoustic and language features separately on these 3 data sets. RESULTS: Repeatability was evaluated using intraclass correlation (ICC) and the within-subjects coefficient of variation (WSCV). In 3 sets of tasks, the median ICC were between 0.02 and 0.55, and the median WSCV were between 29 and 79%. CONCLUSION: Our results demonstrate that the repeatability of speech features extracted using open-source tool kits is low. Researchers should exercise caution when developing digital health models with open-source speech features. We provide a detailed summary of feature-by-feature repeatability results (ICC, WSCV, SE of measurement, limits of agreement for WSCV, and minimal detectable change) in the online supplementary material so that researchers may incorporate repeatability information into the models they develop.
Authors: Florian Lipsmeier; Kirsten I Taylor; Timothy Kilchenmann; Detlef Wolf; Alf Scotland; Jens Schjodt-Eriksen; Wei-Yi Cheng; Ignacio Fernandez-Garcia; Juliane Siebourg-Polster; Liping Jin; Jay Soto; Lynne Verselis; Frank Boess; Martin Koller; Michael Grundman; Andreas U Monsch; Ronald B Postuma; Anirvan Ghosh; Thomas Kremer; Christian Czech; Christian Gossens; Michael Lindemann Journal: Mov Disord Date: 2018-04-27 Impact factor: 10.338
Authors: Jan Rusz; Jan Hlavnička; Michal Novotný; Tereza Tykalová; Amelie Pelletier; Jacques Montplaisir; Jean-Francois Gagnon; Petr Dušek; Andrea Galbiati; Sara Marelli; Paul C Timm; Luke N Teigen; Annette Janzen; Mahboubeh Habibi; Ambra Stefani; Evi Holzknecht; Klaus Seppi; Elisa Evangelista; Anna Laura Rassu; Yves Dauvilliers; Birgit Högl; Wolfgang Oertel; Erik K St Louis; Luigi Ferini-Strambi; Evžen Růžička; Ronald B Postuma; Karel Šonka Journal: Ann Neurol Date: 2021-05-07 Impact factor: 10.422
Authors: Visar Berisha; Chelsea Krantsevich; P Richard Hahn; Shira Hahn; Gautam Dasarathy; Pavan Turaga; Julie Liss Journal: NPJ Digit Med Date: 2021-10-28
Authors: Larry Zhang; Anthony Ngo; Jason A Thomas; Hannah A Burkhardt; Carolyn M Parsey; Rhoda Au; Reza Hosseini Ghomi Journal: Explor Med Date: 2021-06-30