Gabriel P Ovsiew1, Zachary J Resch1,2, Kritika Nayar1,3, Christopher P Williams1,2, Jason R Soble1,4. 1. Department of Psychiatry, University of Illinois College of Medicine , Chicago, IL, USA. 2. Department of Psychology, Rosalind Franklin University of Medicine and Science , North Chicago, IL, USA. 3. Department of Psychiatry and Behavioral Sciences, Northwestern Feinberg School of Medicine , Chicago, IL, USA. 4. Department of Neurology, University of Illinois College of Medicine , Chicago, IL, USA.
Abstract
INTRODUCTION: Validity indicators embedded within standard neuropsychological tests have received increasing attention as more efficient measures for sampling performance validity throughout an evaluation. This cross-sectional study examined multiple performance validity tests (PVTs) embedded in the Wechsler Adult Intelligence Scale-IV (WAIS-IV) Working Memory (WMI) and Processing Speed (PSI) Indices for detecting invalid test performance. METHOD: This cross-sectional study examined data from a mixed clinical neuropsychiatric sample of 110 patients referred for outpatient evaluation. The sample was composed of 85 patients with valid neuropsychological performance and 25 with invalid performance based on multiple independent criterion PVTs. Among the patients with valid performance, 54% were cognitively impaired, whereas 46% were cognitively unimpaired. RESULTS: Among the overall sample, performance on WMI, PSI, and 3/4 constituent subtests (i.e., Digit Span, Symbol Search, Coding) was significantly worse among the invalid group compared to the valid group (ηp 2 =.06-.16) with areas under the curve (AUCs) of.67-.76 and 24-32% sensitivity (≥88% specificity) for identifying invalid performance at cut-scores that maximized accuracy. When the sample was subdivided by cognitive impairment status, AUCs of.68-.87 and 36-56% sensitivity (≥87% specificity) for detecting invalidity at cut-scores that maximized accuracy were found among those without cognitive impairment. In contrast, for patients with cognitive impairment, Digit Span, Arithmetic, WMI, and Coding were nonsignificant, and AUCs of.66-.67. Further, notably reduced sensitivities of 16-28% (≥91% specificity) were found for the remaining significant indices. CONCLUSION: Overall, results indicated that embedded WAIS-IV WMI and PSI are useful embedded PVTs in conditions in which cognitive impairment is not expected; however, these embedded PVTs demonstrated questionable utility among patients with cognitive impairment due to poor sensitivity, if adequate specificity is maintained, suggesting limited efficacy among patients with cognitive impairment due to risk of false-positive classification.
INTRODUCTION: Validity indicators embedded within standard neuropsychological tests have received increasing attention as more efficient measures for sampling performance validity throughout an evaluation. This cross-sectional study examined multiple performance validity tests (PVTs) embedded in the Wechsler Adult Intelligence Scale-IV (WAIS-IV) Working Memory (WMI) and Processing Speed (PSI) Indices for detecting invalid test performance. METHOD: This cross-sectional study examined data from a mixed clinical neuropsychiatric sample of 110 patients referred for outpatient evaluation. The sample was composed of 85 patients with valid neuropsychological performance and 25 with invalid performance based on multiple independent criterion PVTs. Among the patients with valid performance, 54% were cognitively impaired, whereas 46% were cognitively unimpaired. RESULTS: Among the overall sample, performance on WMI, PSI, and 3/4 constituent subtests (i.e., Digit Span, Symbol Search, Coding) was significantly worse among the invalid group compared to the valid group (ηp 2 =.06-.16) with areas under the curve (AUCs) of.67-.76 and 24-32% sensitivity (≥88% specificity) for identifying invalid performance at cut-scores that maximized accuracy. When the sample was subdivided by cognitive impairment status, AUCs of.68-.87 and 36-56% sensitivity (≥87% specificity) for detecting invalidity at cut-scores that maximized accuracy were found among those without cognitive impairment. In contrast, for patients with cognitive impairment, Digit Span, Arithmetic, WMI, and Coding were nonsignificant, and AUCs of.66-.67. Further, notably reduced sensitivities of 16-28% (≥91% specificity) were found for the remaining significant indices. CONCLUSION: Overall, results indicated that embedded WAIS-IV WMI and PSI are useful embedded PVTs in conditions in which cognitive impairment is not expected; however, these embedded PVTs demonstrated questionable utility among patients with cognitive impairment due to poor sensitivity, if adequate specificity is maintained, suggesting limited efficacy among patients with cognitive impairment due to risk of false-positive classification.
Entities:
Keywords:
Performance validity; assessment; processing speed; psychometrics; working memory
Authors: Zachary J Resch; Troy A Webber; Matthew T Bernstein; Tasha Rhoads; Gabriel P Ovsiew; Jason R Soble Journal: Neuropsychol Rev Date: 2021-01-12 Impact factor: 7.444