G Rae1, P Hyland. 1. School of Psychology and Communication, University of Ulster, Coleraine, Co. Londonderry, Northern Ireland, BT52 1SA. G.Rae@ulst.ac.uk
Abstract
BACKGROUND: Scoring systems to evaluate children's human drawings for intellectual maturity have been found to have good intra- and inter-scorer reliability. However, there is some evidence (McCarthy, 1944) that such scores may not be stable over time. AIM: The aim of the study was to investigate raters and occasions as potential sources of error in children's Draw-a-Person scores using generalisability and classical test theory. SAMPLE: The sample consisted of 85 school children (45 girls and 40 boys) aged between 8 years 1 month to 9 years and 7 months. METHOD: The Koppitz Draw-A-Person (1968) test was administered as a class test on two occasions with exactly a two-week interval between the occasions. All the children's drawings were scored by the same four raters. RESULTS: Generalisability analyses of the Koppitz scores indicated that the variance components for raters and its interaction with both persons and occasions were very small, suggesting that very little measurement error was associated with the raters. However, the estimated variance component for the interaction of persons by occasions was substantial. With four raters and two occasions the generalisability coefficient was .47. These results were consistent with the classical test theory analysis which indicated generally high inter-rater reliabilities but a low test-retest reliability, based on a composite of the four raters. CONCLUSION: If satisfactory levels of reliability/generalisability are to be achieved with the Koppitz scoring system children have to be tested on several occasions.
BACKGROUND: Scoring systems to evaluate children's human drawings for intellectual maturity have been found to have good intra- and inter-scorer reliability. However, there is some evidence (McCarthy, 1944) that such scores may not be stable over time. AIM: The aim of the study was to investigate raters and occasions as potential sources of error in children's Draw-a-Person scores using generalisability and classical test theory. SAMPLE: The sample consisted of 85 school children (45 girls and 40 boys) aged between 8 years 1 month to 9 years and 7 months. METHOD: The Koppitz Draw-A-Person (1968) test was administered as a class test on two occasions with exactly a two-week interval between the occasions. All the children's drawings were scored by the same four raters. RESULTS: Generalisability analyses of the Koppitz scores indicated that the variance components for raters and its interaction with both persons and occasions were very small, suggesting that very little measurement error was associated with the raters. However, the estimated variance component for the interaction of persons by occasions was substantial. With four raters and two occasions the generalisability coefficient was .47. These results were consistent with the classical test theory analysis which indicated generally high inter-rater reliabilities but a low test-retest reliability, based on a composite of the four raters. CONCLUSION: If satisfactory levels of reliability/generalisability are to be achieved with the Koppitz scoring system children have to be tested on several occasions.