Olivier Godefroy1, Laura Gibbons2, Momar Diouf3, David Nyenhuis4, Martine Roussel5, Sandra Black6, Jean Marc Bugnicourt5. 1. Department of Neurology and Laboratory of Functional Neurosciences University Hospital of Amiens, France. Electronic address: godefroy.olivier@chu-amiens.fr. 2. Department of General Internal Medicine, University of Washington, Harborview Medical Center, Seattle, WA, USA. 3. Department of Biostatistics, University Hospital of Amiens, France. 4. Hauenstein Neuroscience Center, Saint Mary's Health Care, Grand Rapids, MI, USA. 5. Department of Neurology and Laboratory of Functional Neurosciences University Hospital of Amiens, France. 6. Brill Chair in Neurology, Dept of Medicine, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada.
Abstract
INTRODUCTION: Although accurate diagnosis of deficit of mild intensity is critical, various methods are used to assess, dichotomize and integrate performance, with no validated gold standard. This study described and validated a framework for the analysis of cognitive performance. METHODS: This study was performed by using the Groupe de Réflexion sur L'Evaluation des Fonctions EXécutives (GREFEX) database (724 controls and 461 patients) examined by 7 tests assessing executive functions. The first phase determined the criteria for the cutoff scores, the second phase, the effect of test number on diagnostic accuracy and the third phase, the best methods for combining test scores into an overall summary score. Four validation criteria were used: determination of impaired performance as compared to expected one, false-positive rate ≤5%, detection of both single and multiple impairments with optimal sensitivity. RESULTS: The procedure based on 5th percentile cutoffs determined from standardized residuals was the most appropriate procedure. Although area under the curve (AUC) increased with the number of scores (p = .0001), the false-positive rate also increased (p = .0001), resulting in suboptimal sensitivity for detecting selective impairment. Two overall summary scores, the average of the seven process scores and the Item Response Theory (IRT) score, had significantly (p = .0001) higher AUCs, even for patients with a selective impairment, and provided higher resulting prevalence of dysexecutive disorders (p = .0001). CONCLUSIONS: The present study provides and validates a generative framework for the interpretation of cognitive data. Two overall summary score met all 4 validation criteria. A practical consequence is the need to profoundly modify the analysis and interpretation of cognitive assessments for both routine use and clinical research.
INTRODUCTION: Although accurate diagnosis of deficit of mild intensity is critical, various methods are used to assess, dichotomize and integrate performance, with no validated gold standard. This study described and validated a framework for the analysis of cognitive performance. METHODS: This study was performed by using the Groupe de Réflexion sur L'Evaluation des Fonctions EXécutives (GREFEX) database (724 controls and 461 patients) examined by 7 tests assessing executive functions. The first phase determined the criteria for the cutoff scores, the second phase, the effect of test number on diagnostic accuracy and the third phase, the best methods for combining test scores into an overall summary score. Four validation criteria were used: determination of impaired performance as compared to expected one, false-positive rate ≤5%, detection of both single and multiple impairments with optimal sensitivity. RESULTS: The procedure based on 5th percentile cutoffs determined from standardized residuals was the most appropriate procedure. Although area under the curve (AUC) increased with the number of scores (p = .0001), the false-positive rate also increased (p = .0001), resulting in suboptimal sensitivity for detecting selective impairment. Two overall summary scores, the average of the seven process scores and the Item Response Theory (IRT) score, had significantly (p = .0001) higher AUCs, even for patients with a selective impairment, and provided higher resulting prevalence of dysexecutive disorders (p = .0001). CONCLUSIONS: The present study provides and validates a generative framework for the interpretation of cognitive data. Two overall summary score met all 4 validation criteria. A practical consequence is the need to profoundly modify the analysis and interpretation of cognitive assessments for both routine use and clinical research.
Authors: B Winblad; K Palmer; M Kivipelto; V Jelic; L Fratiglioni; L-O Wahlund; A Nordberg; L Bäckman; M Albert; O Almkvist; H Arai; H Basun; K Blennow; M de Leon; C DeCarli; T Erkinjuntti; E Giacobini; C Graff; J Hardy; C Jack; A Jorm; K Ritchie; C van Duijn; P Visser; R C Petersen Journal: J Intern Med Date: 2004-09 Impact factor: 8.989
Authors: Paul K Crane; Kaavya Narasimhalu; Laura E Gibbons; Otto Pedraza; Kala M Mehta; Yuxiao Tang; Jennifer J Manly; Bruce R Reed; Dan M Mungas Journal: J Int Neuropsychol Soc Date: 2008-09 Impact factor: 2.892
Authors: Patrick M Bossuyt; Johannes B Reitsma; David E Bruns; Constantine A Gatsonis; Paul P Glasziou; Les M Irwig; Jeroen G Lijmer; David Moher; Drummond Rennie; Henrica C W de Vet Journal: BMJ Date: 2003-01-04
Authors: Jessica W Lo; John D Crawford; David W Desmond; Olivier Godefroy; Hanna Jokinen; Simin Mahinrad; Hee-Joon Bae; Jae-Sung Lim; Sebastian Köhler; Elles Douven; Julie Staals; Christopher Chen; Xin Xu; Eddie J Chong; Rufus O Akinyemi; Rajesh N Kalaria; Adesola Ogunniyi; Mélanie Barbay; Martine Roussel; Byung-Chul Lee; Velandai K Srikanth; Christopher Moran; Nagaendran Kandiah; Russell J Chander; Behnam Sabayan; J Wouter Jukema; Susanna Melkas; Timo Erkinjuntti; Henry Brodaty; Régis Bordet; Stéphanie Bombois; Hilde Hénon; Darren M Lipnicki; Nicole A Kochan; Perminder S Sachdev Journal: Neurology Date: 2019-11-11 Impact factor: 9.910