Jeanne A Teresi1, Katja Ocepek-Welikson2, Marjorie Kleinman3, Andrea Cheville4, Mildred Ramirez5. 1. Columbia University Stroud Center, New York, NY; New York State Psychiatric Institute, New York, NY; Division of Geriatrics and Palliative Medicine, Weill Cornell Medical Center, New York, NY; Research Division, Hebrew Home at Riverdale, RiverSpring Health, Bronx, NY. Electronic address: jat61@cumc.columbia.edu. 2. Research Division, Hebrew Home at Riverdale, RiverSpring Health, Bronx, NY. 3. New York State Psychiatric Institute, New York, NY. 4. Department of Physical Medicine and Rehabilitation and Cardiovascular Research, Mayo Clinic, Rochester, MN. 5. Division of Geriatrics and Palliative Medicine, Weill Cornell Medical Center, New York, NY; Research Division, Hebrew Home at Riverdale, RiverSpring Health, Bronx, NY.
Abstract
OBJECTIVE: To present challenges in assessment of applied cognition and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT). DESIGN: Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and sex groups. DIF hypotheses informed the evaluation of the results. SETTING: Hospital-based rehabilitation from a single hospital system. PARTICIPANTS: A total of 2216 hospitalized patients (N=2216). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Applied cognition item pool from multiple sources. RESULTS: Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (>0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude and 19 with accompanying supportive hypotheses. CONCLUSIONS: A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one-third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and sex. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; 4 were deleted from the bank and 2 had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.
OBJECTIVE: To present challenges in assessment of applied cognition and the results of differential item functioning (DIF) analyses used to inform the development of a computerized adaptive test (CAT). DESIGN: Measurement evaluation cohort study. DIF analyses of 107 items were conducted across educational, age, and sex groups. DIF hypotheses informed the evaluation of the results. SETTING: Hospital-based rehabilitation from a single hospital system. PARTICIPANTS: A total of 2216 hospitalized patients (N=2216). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: Applied cognition item pool from multiple sources. RESULTS: Many items were hypothesized to show DIF, particularly for age. Information was moderately high in the lower (cognitive disability) tail of the distribution, but some items were not informative. Reliability estimates were high (>0.89) across all studied groups, regardless of estimation method. There were 35 items with DIF of high magnitude and 19 with accompanying supportive hypotheses. CONCLUSIONS: A key clinical tool in inpatient rehabilitation medicine is assessment of applied functional cognitive ability to inform patient-centered rehabilitation strategies to improve function. This was the first study to evaluate measurement equivalence of the applied cognition item pool across large samples of hospitalized patients. Although about one-third of the item pool evidenced DIF or low discrimination, results supported placement of most items into the bank and its use across groups differing in education, age, and sex. Six items were classified with salient DIF, defined as consistent DIF of high magnitude and or impact, with confirmatory directional DIF hypotheses, generated by content experts. These were recommended for adjustment or removal from the bank; 4 were deleted from the bank and 2 had lowered CAT exposure (administration frequency) rates. Many items hypothesized to show DIF contained content measuring constructs other than applied cognition such as physical frailty, perceptual difficulties, or skills reflective of greater educational attainment. Challenges in measurement of this construct are discussed.
Authors: Bryce B Reeve; Ron D Hays; Jakob B Bjorner; Karon F Cook; Paul K Crane; Jeanne A Teresi; David Thissen; Dennis A Revicki; David J Weiss; Ronald K Hambleton; Honghu Liu; Richard Gershon; Steven P Reise; Jin-shei Lai; David Cella Journal: Med Care Date: 2007-05 Impact factor: 2.983
Authors: Robert Fieo; Katja Ocepek-Welikson; Marjorie Kleinman; Joseph P Eimicke; Paul K Crane; David Cella; Jeanne A Teresi Journal: Psychol Test Assess Model Date: 2016
Authors: Sarah Tomaszewski Farias; Dan Mungas; Danielle J Harvey; Amanda Simmons; Bruce R Reed; Charles Decarli Journal: Alzheimers Dement Date: 2011-11 Impact factor: 21.566
Authors: Frances M Yang; Kevin C Heslin; Kala M Mehta; Cheng-Wu Yang; Katja Ocepek-Welikson; Marjorie Kleinman; Leo S Morales; Ron D Hays; Anita L Stewart; Dan Mungas; Richard N Jones; Jeanne A Teresi Journal: Psychol Test Assess Model Date: 2011
Authors: Elizabeth Marfeo; Pengsheng Ni; Chun Wang; David Weiss; Andrea L Cheville Journal: Arch Phys Med Rehabil Date: 2021-12-26 Impact factor: 4.060