OBJECTIVE: Systematic literature reviews contribute to evidence-based occupational therapy, yet no data capture tool currently exists to validly and reliably appraise the characteristics and quality of primary studies. METHOD: We determined the psychometrics of Systematic Process for Investigating and Describing Evidence-Based Research (SPIDER) and piloted it with 201 studies included in a systematic literature review. RESULTS: Content validity showed item relevance with 73% agreement between two experts. For the quality construct, seven of nine quality indicators were positively (p < .05) correlated with the overall quality score. The quality scores were positively correlated (p < .05) with two objective measures, inferring criterion validity. Intrarater reliability was moderate to perfect (kappa = 0.4-1.0). Cross-tab analyses showed less variation in experienced reviewers' interrater reliability. CONCLUSION: SPIDER provides plausible opportunities for occupational therapy researchers and graduate students to appraise the characteristics and quality of primary studies but requires testing across other settings.
OBJECTIVE: Systematic literature reviews contribute to evidence-based occupational therapy, yet no data capture tool currently exists to validly and reliably appraise the characteristics and quality of primary studies. METHOD: We determined the psychometrics of Systematic Process for Investigating and Describing Evidence-Based Research (SPIDER) and piloted it with 201 studies included in a systematic literature review. RESULTS: Content validity showed item relevance with 73% agreement between two experts. For the quality construct, seven of nine quality indicators were positively (p < .05) correlated with the overall quality score. The quality scores were positively correlated (p < .05) with two objective measures, inferring criterion validity. Intrarater reliability was moderate to perfect (kappa = 0.4-1.0). Cross-tab analyses showed less variation in experienced reviewers' interrater reliability. CONCLUSION: SPIDER provides plausible opportunities for occupational therapy researchers and graduate students to appraise the characteristics and quality of primary studies but requires testing across other settings.