Shifali Singh1,2, Roger W Strong1,2, Laneé Jung1,2, Frances Haofei Li1,2, Liz Grinspoon1,2, Luke S Scheuer1,2, Eliza J Passell1,2, Paolo Martini1,2, Naomi Chaytor3, Jason R Soble4,5, Laura Germine1,2. 1. Institute for Technology in Psychiatry, McLean Hospital, Belmont, MA, USA. 2. Department of Psychiatry, Harvard Medical School, Boston, MA, USA. 3. Elson S. Floyd College of Medicine, Washington State University, Spokane, WA, USA. 4. Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA. 5. Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA.
Abstract
INTRODUCTION: To allow continued administration of neuropsychological evaluations remotely during the pandemic, tests from the not-for-profit platform, TestMyBrain.org (TMB), were used to develop the TMB Digital Neuropsychology Toolkit (DNT). This study details the psychometric characteristics of the DNT, as well as the infrastructure and development of the DNT. METHOD: The DNT was primarily distributed for clinical use, with (72.8%) of individuals requesting access for clinical purposes. To assess reliability and validity of the DNT, anonymous data from DNT test administrations were analyzed and compared to a large, non-clinical normative sample from TMB. RESULTS: DNT test scores showed acceptable to very good split-half reliability (.68-.99). Factor analysis revealed three latent factors, corresponding to processing speed, working memory, and a broader general cognitive ability factor that included perceptual reasoning and episodic memory. Average test scores were slightly poorer for the DNT sample than for the TMB comparison sample, as expected given the clinical use of the DNT. CONCLUSIONS: Initial estimates of reliability and validity of DNT tests support their use as digital measures of neuropsychological functioning. Tests within cognitive domains correlated highly with each other and demonstrated good reliability and validity. Future work will seek to validate DNT tests in specific clinical populations and determine best practices for using DNT outcome measures to assess engagement and psychological symptomatology.
INTRODUCTION: To allow continued administration of neuropsychological evaluations remotely during the pandemic, tests from the not-for-profit platform, TestMyBrain.org (TMB), were used to develop the TMB Digital Neuropsychology Toolkit (DNT). This study details the psychometric characteristics of the DNT, as well as the infrastructure and development of the DNT. METHOD: The DNT was primarily distributed for clinical use, with (72.8%) of individuals requesting access for clinical purposes. To assess reliability and validity of the DNT, anonymous data from DNT test administrations were analyzed and compared to a large, non-clinical normative sample from TMB. RESULTS: DNT test scores showed acceptable to very good split-half reliability (.68-.99). Factor analysis revealed three latent factors, corresponding to processing speed, working memory, and a broader general cognitive ability factor that included perceptual reasoning and episodic memory. Average test scores were slightly poorer for the DNT sample than for the TMB comparison sample, as expected given the clinical use of the DNT. CONCLUSIONS: Initial estimates of reliability and validity of DNT tests support their use as digital measures of neuropsychological functioning. Tests within cognitive domains correlated highly with each other and demonstrated good reliability and validity. Future work will seek to validate DNT tests in specific clinical populations and determine best practices for using DNT outcome measures to assess engagement and psychological symptomatology.
Entities:
Keywords:
Digital neuropsychology; assessment; cognition; psychometrics; remote testing; teleneuropsychology
Authors: Jeremy B Wilmer; Laura Germine; Christopher F Chabris; Garga Chatterjee; Mark Williams; Eric Loken; Ken Nakayama; Bradley Duchaine Journal: Proc Natl Acad Sci U S A Date: 2010-02-22 Impact factor: 11.205
Authors: Francesca C Fortenbaugh; Joseph DeGutis; Laura Germine; Jeremy B Wilmer; Mallory Grosso; Kathryn Russo; Michael Esterman Journal: Psychol Sci Date: 2015-08-07
Authors: Melissa Treviño; Xiaoshu Zhu; Yi Yi Lu; Luke S Scheuer; Eliza Passell; Grace C Huang; Laura T Germine; Todd S Horowitz Journal: Cogn Res Princ Implic Date: 2021-07-22
Authors: Jeremy B Wilmer; Laura Germine; Christopher F Chabris; Garga Chatterjee; Margaret Gerbasi; Ken Nakayama Journal: Cogn Neuropsychol Date: 2012 Impact factor: 2.468
Authors: Lauren A Rutter; Ipsit V Vahia; Brent P Forester; Kerry J Ressler; Laura Germine Journal: Front Aging Neurosci Date: 2020-03-06 Impact factor: 5.750