R Hébert1, G Bravo. 1. Department of Family Medicine, Université de Sherbrooke Faculty of Medicine, Quebec, Canada. rhebert@courrier.usherb.ca
Abstract
PURPOSE: To develop and explore the validity of the Tutotest, the first standardized instrument designed to assess the skills and attitudes of medical students working in tutorials in a problem-based learning curriculum. METHOD: The Tutotest, consisting of 44 items rated on a four-point Likert-type scale, was developed from well-defined theoretical frameworks after consultation with students and teachers at the Université de Sherbrooke Faculty of Medicine. In 1988-89, 40 tutors were asked to use the Tutotest to evaluate a total of 100 students at the end of each of the four units of the first-year curriculum. Data were analyzed to examine the homogeneity and reliability of the instrument, explore its factorial structure, and document its correlation with students' grades. RESULTS: A total of 28 tutors (70%) conducted Tutotest evaluations; of a possible 400 evaluations, 270 (67.5%) were returned with complete data. Exploratory factor analysis revealed that the Tutotest was structured around four factors that accounted for 82% of the variance: effectiveness in the group (23 items), communication and leadership skills (13 items), scientific curiosity (four items), and respect for colleagues (four items). The Tutotest had a Cronbach's alpha coefficient of .98 and a split-half coefficient of .98. The intraclass correlation coefficient was estimated to be .46 and would reach up to .81 after averaging five Tutotest evaluations. Tutotest scores showed good correlation (r = .64) with the official tutor's global evaluation and a moderate correlation (r = .39) with students' written examination results. CONCLUSION: The Tutotest was found to be a standardized reliable and valid instrument that can significantly improve the evaluation of students' skills and attitudes during tutorials. Although lengthier than the usual tutor evaluation form, the Tutotest is a good compromise between an unreliable short global scale and a more comprehensive but impractical instrument.
PURPOSE: To develop and explore the validity of the Tutotest, the first standardized instrument designed to assess the skills and attitudes of medical students working in tutorials in a problem-based learning curriculum. METHOD: The Tutotest, consisting of 44 items rated on a four-point Likert-type scale, was developed from well-defined theoretical frameworks after consultation with students and teachers at the Université de Sherbrooke Faculty of Medicine. In 1988-89, 40 tutors were asked to use the Tutotest to evaluate a total of 100 students at the end of each of the four units of the first-year curriculum. Data were analyzed to examine the homogeneity and reliability of the instrument, explore its factorial structure, and document its correlation with students' grades. RESULTS: A total of 28 tutors (70%) conducted Tutotest evaluations; of a possible 400 evaluations, 270 (67.5%) were returned with complete data. Exploratory factor analysis revealed that the Tutotest was structured around four factors that accounted for 82% of the variance: effectiveness in the group (23 items), communication and leadership skills (13 items), scientific curiosity (four items), and respect for colleagues (four items). The Tutotest had a Cronbach's alpha coefficient of .98 and a split-half coefficient of .98. The intraclass correlation coefficient was estimated to be .46 and would reach up to .81 after averaging five Tutotest evaluations. Tutotest scores showed good correlation (r = .64) with the official tutor's global evaluation and a moderate correlation (r = .39) with students' written examination results. CONCLUSION: The Tutotest was found to be a standardized reliable and valid instrument that can significantly improve the evaluation of students' skills and attitudes during tutorials. Although lengthier than the usual tutor evaluation form, the Tutotest is a good compromise between an unreliable short global scale and a more comprehensive but impractical instrument.