Katharina Schultebraucks1,2,3, Vijay Yadav4, Isaac R Galatzer-Levy2,4. 1. Vagelos School of Physicians and Surgeons, Department of Emergency Medicine, Columbia University Medical Center, New York, New York, USA. 2. Department of Psychiatry, New York University School of Medicine, New York, New York, USA. 3. Data Science Institute, Columbia University, New York, New York, USA. 4. AiCure, New York, New York, USA.
Abstract
BACKGROUND: Alterations in multiple domains of cognition have been observed in individuals who have experienced a traumatic stressor. These domains may provide important insights in identifying underlying neurobiological dysfunction driving an individual's clinical response to trauma. However, such assessments are burdensome, costly, and time-consuming. To overcome barriers, efforts have emerged to measure multiple domains of cognitive functioning through the application of machine learning (ML) models to passive data sources. METHODS: We utilized automated computer vision and voice analysis methods to extract facial, movement, and speech characteristics from semi-structured clinical interviews in 81 trauma survivors who additionally completed a cognitive assessment battery. A ML-based regression framework was used to identify variance in visual and auditory measures that relate to multiple cognitive domains. RESULTS: Models derived from visual and auditory measures collectively accounted for a large variance in multiple domains of cognitive functioning, including motor coordination (R2 = 0.52), processing speed (R2 = 0.42), emotional bias (R2 = 0.52), sustained attention (R2 = 0.51), controlled attention (R2 = 0.44), cognitive flexibility (R2 = 0.43), cognitive inhibition (R2 = 0.64), and executive functioning (R2 = 0.63), consistent with the high test-retest reliability of traditional cognitive assessments. Face, voice, speech content, and movement have all significantly contributed to explaining the variance in predicting functioning in all cognitive domains. CONCLUSIONS: The results demonstrate the feasibility of automated measurement of reliable proxies of cognitive functioning through low-burden passive patient evaluations. This makes it easier to monitor cognitive functions and to intervene earlier and at a lower threshold without requiring a time-consuming neurocognitive assessment by, for instance, a licensed psychologist with specialized training in neuropsychology.
BACKGROUND: Alterations in multiple domains of cognition have been observed in individuals who have experienced a traumatic stressor. These domains may provide important insights in identifying underlying neurobiological dysfunction driving an individual's clinical response to trauma. However, such assessments are burdensome, costly, and time-consuming. To overcome barriers, efforts have emerged to measure multiple domains of cognitive functioning through the application of machine learning (ML) models to passive data sources. METHODS: We utilized automated computer vision and voice analysis methods to extract facial, movement, and speech characteristics from semi-structured clinical interviews in 81 trauma survivors who additionally completed a cognitive assessment battery. A ML-based regression framework was used to identify variance in visual and auditory measures that relate to multiple cognitive domains. RESULTS: Models derived from visual and auditory measures collectively accounted for a large variance in multiple domains of cognitive functioning, including motor coordination (R2 = 0.52), processing speed (R2 = 0.42), emotional bias (R2 = 0.52), sustained attention (R2 = 0.51), controlled attention (R2 = 0.44), cognitive flexibility (R2 = 0.43), cognitive inhibition (R2 = 0.64), and executive functioning (R2 = 0.63), consistent with the high test-retest reliability of traditional cognitive assessments. Face, voice, speech content, and movement have all significantly contributed to explaining the variance in predicting functioning in all cognitive domains. CONCLUSIONS: The results demonstrate the feasibility of automated measurement of reliable proxies of cognitive functioning through low-burden passive patient evaluations. This makes it easier to monitor cognitive functions and to intervene earlier and at a lower threshold without requiring a time-consuming neurocognitive assessment by, for instance, a licensed psychologist with specialized training in neuropsychology.
Authors: Katharina Schultebraucks; Vijay Yadav; Arieh Y Shalev; George A Bonanno; Isaac R Galatzer-Levy Journal: Psychol Med Date: 2020-08-03 Impact factor: 7.723