Mathias Basner1, Emanuel Hermosillo1, Jad Nasrini1, Salil Saxena1, David F Dinges1, Tyler M Moore2, Ruben C Gur2. 1. Unit for Experimental Psychiatry, Division of Sleep and Chronobiology, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA, USA. 2. Brain Behavior Laboratory, Neuropsychiatry Section, Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania , Philadelphia, PA, USA.
Abstract
INTRODUCTION: Practice effects associated with the repeated administration of cognitive tests often confound true therapeutic or experimental effects. Alternate test forms help reduce practice effects, but generating stimulus sets with identical properties can be difficult. The main objective of this study was to disentangle practice and stimulus set effects for Cognition, a battery of 10 brief cognitive tests specifically designed for high-performing populations with 15 unique versions for repeated testing. A secondary objective was to investigate the effects of test-retest interval on practice effects. METHODS: The 15 versions of Cognition were administered in three groups of 15-16 subjects (total N = 46, mean±SD age 32.5 ± 7.2 years, range 25-54 years, 23 male) in a randomized but balanced fashion with administration intervals of ≥10 days, ≤5 days, or 4 times per day. Mixed effect models were used to investigate linear and logarithmic trends across repeated administrations in key speed and accuracy outcomes, whether these trends differed significantly between administration interval groups, and whether stimulus sets differed significantly in difficulty. RESULTS: Protracted, non-linear practice effects well beyond the second administration were observed for most of the 10 Cognition tests both in accuracy and speed, but test-retest administration interval significantly affected practice effects only for 3 out of the 10 tests and only in the speed domain. Stimulus set effects were observed for the 6 Cognition tests that use unique sets of stimuli. Factors were established that allow for correcting for both practice and stimulus set effects. CONCLUSIONS: Practice effects are pronounced and probably under-appreciated in cognitive testing. The correction factors established in this study are a unique feature of the Cognition battery that can help avoid masking practice effects, address noise generated by differences in stimulus set difficulty, and facilitate interpretation of results from studies with repeated assessments.
RCT Entities:
INTRODUCTION: Practice effects associated with the repeated administration of cognitive tests often confound true therapeutic or experimental effects. Alternate test forms help reduce practice effects, but generating stimulus sets with identical properties can be difficult. The main objective of this study was to disentangle practice and stimulus set effects for Cognition, a battery of 10 brief cognitive tests specifically designed for high-performing populations with 15 unique versions for repeated testing. A secondary objective was to investigate the effects of test-retest interval on practice effects. METHODS: The 15 versions of Cognition were administered in three groups of 15-16 subjects (total N = 46, mean±SD age 32.5 ± 7.2 years, range 25-54 years, 23 male) in a randomized but balanced fashion with administration intervals of ≥10 days, ≤5 days, or 4 times per day. Mixed effect models were used to investigate linear and logarithmic trends across repeated administrations in key speed and accuracy outcomes, whether these trends differed significantly between administration interval groups, and whether stimulus sets differed significantly in difficulty. RESULTS: Protracted, non-linear practice effects well beyond the second administration were observed for most of the 10 Cognition tests both in accuracy and speed, but test-retest administration interval significantly affected practice effects only for 3 out of the 10 tests and only in the speed domain. Stimulus set effects were observed for the 6 Cognition tests that use unique sets of stimuli. Factors were established that allow for correcting for both practice and stimulus set effects. CONCLUSIONS: Practice effects are pronounced and probably under-appreciated in cognitive testing. The correction factors established in this study are a unique feature of the Cognition battery that can help avoid masking practice effects, address noise generated by differences in stimulus set difficulty, and facilitate interpretation of results from studies with repeated assessments.
Keywords:
Cognition; learning; practice; test difficulty; testing interval
Authors: Robert L Heilbronner; Jerry J Sweet; Deborah K Attix; Kevin R Krull; George K Henry; Robert P Hart Journal: Clin Neuropsychol Date: 2010-11 Impact factor: 3.535
Authors: Ruben C Gur; Jan Richard; Monica E Calkins; Rosetta Chiavacci; John A Hansen; Warren B Bilker; James Loughead; John J Connolly; Haijun Qiu; Frank D Mentch; Patrick M Abou-Sleiman; Hakon Hakonarson; Raquel E Gur Journal: Neuropsychology Date: 2012-01-16 Impact factor: 3.295
Authors: David R Roalf; Kosha Ruparel; Raquel E Gur; Warren Bilker; Raphael Gerraty; Mark A Elliott; R Sean Gallagher; Laura Almasy; Michael F Pogue-Geile; Konasale Prasad; Joel Wood; Vishwajit L Nimgaonkar; Ruben C Gur Journal: Neuropsychology Date: 2013-12-23 Impact factor: 3.295
Authors: Mathias Basner; Adam Savitt; Tyler M Moore; Allison M Port; Sarah McGuire; Adrian J Ecker; Jad Nasrini; Daniel J Mollicone; Christopher M Mott; Thom McCann; David F Dinges; Ruben C Gur Journal: Aerosp Med Hum Perform Date: 2015-11 Impact factor: 1.053
Authors: Mathias Basner; Tyler M Moore; Emanuel Hermosillo; Jad Nasrini; David F Dinges; Ruben C Gur; Bernd Johannes Journal: Aerosp Med Hum Perform Date: 2020-11-01 Impact factor: 1.053
Authors: K Casario; K Howard; M Cordoza; E Hermosillo; L Ibrahim; O Larson; J Nasrini; M Basner Journal: Acta Astronaut Date: 2021-09-24 Impact factor: 2.413
Authors: Edith V Sullivan; Wesley K Thompson; Ty Brumback; Devin Prouty; Susan F Tapert; Sandra A Brown; Michael D De Bellis; Kate B Nooner; Fiona C Baker; Ian M Colrain; Duncan B Clark; Bonnie J Nagel; Kilian M Pohl; Adolf Pfefferbaum Journal: BMC Med Res Methodol Date: 2022-06-24 Impact factor: 4.612
Authors: Mathias Basner; David F Dinges; Kia Howard; Tyler M Moore; Ruben C Gur; Christian Mühl; Alexander C Stahn Journal: Front Physiol Date: 2021-03-17 Impact factor: 4.566
Authors: Rachael D Seidler; Claudia Stern; Mathias Basner; Alexander C Stahn; Floris L Wuyts; Peter Zu Eulenburg Journal: Front Neural Circuits Date: 2022-08-04 Impact factor: 3.342