Evaluation of Web-based computer-aided instruction in a basic science course.

PURPOSE: To demonstrate the applicability of server statistics, in combination with user surveys, to evaluate utilization of Web-based computer-aided instruction (CAI) in the undergraduate medical curriculum.
METHOD: Individual user surveys with students' names provided information about computer literacy prior to the course and use of CAI during the course. Utilization of specific web-based CAI developed for the course was recorded by server software and the daily logs correlated with course content. Regression analyses were used to measure correlation of server access logs of individual students versus information from user surveys and performances in the course based on in-course examinations.
RESULTS: There was no correlation between computer literacy of students at matriculation and their subsequent levels of use of CAI in the curriculum. Utilization of CAI developed for specific course objectives coincided closely with course content, which is an indication of the effectiveness of the applications in achieving their curricular objectives. In contrast, student use of tutorials coincided most closely with in-course examinations. Students' responses to surveys were generally substantiated by server statistics, but discrepancies were sufficiently large (10% to 20%) to call into question the validity of these surveys. Significant differences in CAI utilization correlated with the performances of students in the course.
CONCLUSIONS: This study demonstrates an important advantage of web-based applications to collect and evaluate CAI utilization efficiently and objectively at both the level of the class and the level of the individual student.