Hsin-Yu Chiang1, Shih-Chieh Lee2, Po-Han Lin3, Chia-Yeh Chou4, Ching-Lin Hsieh5. 1. Hsin-Yu Chiang, ScD, is Associate Professor, Department of Occupational Therapy, College of Medicine, Fu Jen Catholic University, New Taipei City, Taiwan. 2. Shih-Chieh Lee, PhD, is Postdoctoral Researcher, School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan. 3. Po-Han Lin, PhD, is Assistant Professor, International Master's Program of Learning and Instruction, National Taipei University of Education, Taipei, Taiwan. 4. Chia-Yeh Chou, MS, is Associate Professor, Department of Occupational Therapy, College of Medicine, Fu Jen Catholic University, New Taipei City, Taiwan. 5. Ching-Lin Hsieh, PhD, OTR, is Professor, School of Occupational Therapy, College of Medicine, National Taiwan University, and Department of Physical Medicine and Rehabilitation, National Taiwan University Hospital, Taipei, Taiwan; clhsieh@ntu.edu.tw.
Abstract
IMPORTANCE: A psychometrically sound measure of social knowledge (SK) is necessary to assess people with schizophrenia because they tend to have moderate to severe deficits in SK. OBJECTIVE: To develop a computerized adaptive test (CAT) for assessing SK in people with schizophrenia. DESIGN: Two phases, consisting of (1) development and validation of an SK item bank and (2) determination of the best stopping rules for the CAT. SETTING: Two psychiatric hospitals. PARTICIPANTS: Two hundred thirty-six people diagnosed with schizophrenia through convenience sampling. MEASURE: Computerized Adaptive Test-Social Knowledge (CAT-SK). RESULTS: The SK items were examined using Rasch analysis. A CAT simulation was performed to determine the best set of stopping rules for achieving high reliability and efficiency. After unsuitable items were removed, 71 items remained with acceptable model fit (infit and outfit mean square <1.4) and no gender bias. Two suboptimal alternative sets of rules were identified. The most efficient set used 21 items to achieve acceptable Rasch reliability (.81). The most reliable set used 40 items to achieve satisfactory Rasch reliability (.88). High correlations (r > .93) between CAT-SK scores and scores on the SK item bank support the concurrent validity of the CAT-SK. CONCLUSIONS AND RELEVANCE: The CAT-SK appears to be a valid assessment that can provide reliable or efficient measures of SK. If high reliability is needed, examiners can adopt the most reliable set of 40 items. If efficiency is the primary concern, they can adopt the most efficient set of 21 items. WHAT THIS ARTICLE ADDS: The CAT-SK is a valid measure of SK with flexibility to meet examiners' needs.
IMPORTANCE: A psychometrically sound measure of social knowledge (SK) is necessary to assess people with schizophrenia because they tend to have moderate to severe deficits in SK. OBJECTIVE: To develop a computerized adaptive test (CAT) for assessing SK in people with schizophrenia. DESIGN: Two phases, consisting of (1) development and validation of an SK item bank and (2) determination of the best stopping rules for the CAT. SETTING: Two psychiatric hospitals. PARTICIPANTS: Two hundred thirty-six people diagnosed with schizophrenia through convenience sampling. MEASURE: Computerized Adaptive Test-Social Knowledge (CAT-SK). RESULTS: The SK items were examined using Rasch analysis. A CAT simulation was performed to determine the best set of stopping rules for achieving high reliability and efficiency. After unsuitable items were removed, 71 items remained with acceptable model fit (infit and outfit mean square <1.4) and no gender bias. Two suboptimal alternative sets of rules were identified. The most efficient set used 21 items to achieve acceptable Rasch reliability (.81). The most reliable set used 40 items to achieve satisfactory Rasch reliability (.88). High correlations (r > .93) between CAT-SK scores and scores on the SK item bank support the concurrent validity of the CAT-SK. CONCLUSIONS AND RELEVANCE: The CAT-SK appears to be a valid assessment that can provide reliable or efficient measures of SK. If high reliability is needed, examiners can adopt the most reliable set of 40 items. If efficiency is the primary concern, they can adopt the most efficient set of 21 items. WHAT THIS ARTICLE ADDS: The CAT-SK is a valid measure of SK with flexibility to meet examiners' needs.