BACKGROUND: Independent learning with simulators might be improved if simulators could "diagnose the learner" by identifying common novice problems, thereby directing self-guided learning. Our goal was to determine if data collected by a virtual reality simulator could be used to predict the problem areas in novice trainees' laparoscopic performance. METHOD: Fourteen expert laparoscopists were interviewed to identify common problem areas experienced by novices as they learn laparoscopy. Two expert laparoscopists rated 20 novices' simulator performances regarding the extent of each problem area. RESULTS: Moderate interrater reliability and high "interproblem" correlations suggest that experts did not reliably distinguish between the five identified problem areas as expected. CONCLUSIONS: The process by which expert teachers "diagnose" student difficulties did not reduce well to numeric assessments using linear independent scales in the simulated context. This finding raises challenges for our ability to identify such difficulties using the data collected by simulators.
BACKGROUND: Independent learning with simulators might be improved if simulators could "diagnose the learner" by identifying common novice problems, thereby directing self-guided learning. Our goal was to determine if data collected by a virtual reality simulator could be used to predict the problem areas in novice trainees' laparoscopic performance. METHOD: Fourteen expert laparoscopists were interviewed to identify common problem areas experienced by novices as they learn laparoscopy. Two expert laparoscopists rated 20 novices' simulator performances regarding the extent of each problem area. RESULTS: Moderate interrater reliability and high "interproblem" correlations suggest that experts did not reliably distinguish between the five identified problem areas as expected. CONCLUSIONS: The process by which expert teachers "diagnose" student difficulties did not reduce well to numeric assessments using linear independent scales in the simulated context. This finding raises challenges for our ability to identify such difficulties using the data collected by simulators.