BACKGROUND: The script concordance (SC) test was conceived to measure knowledge organization, the presence of links between items of knowledge which allow for interpretation of data in clinical decision making situations. Earlier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This study explores the predictive validity of the test. OBJECTIVES: To verify whether scores obtained by students at the end of clerkship predict their clinical reasoning performance at the end of residency. DESIGN: Comparison of scores obtained on a SC test taken at the end of clerkship with those obtained 2 years later at the end of residency on two clinical reasoning assessments of known validity, called the short-answer management problems (SAMPs) and the simulated office orals (SOOs), and an objective structured clinical examination (OSCE) aimed at assessing hands-on skills and clinical reasoning. Data were treated by Pearson correlation analysis. SUBJECTS AND SETTING: A cohort of 24 students from a medical school in Quebec was followed up to the end of their residency in family medicine, completed in several schools across Quebec. RESULTS: The observed Pearson correlation coefficients of the SC test were statistically significant (0.451, P=0.013; 0.447; P=0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically significant (0.340, P=0.052) when compared with the OSCE. CONCLUSION: The authors assumed that the richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on the measures of clinical reasoning (SAMP and SOO), but would predict less well performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study are coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also indicates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in training, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if the later measures bear on a wider clinical domain.
BACKGROUND: The script concordance (SC) test was conceived to measure knowledge organization, the presence of links between items of knowledge which allow for interpretation of data in clinical decision making situations. Earlier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This study explores the predictive validity of the test. OBJECTIVES: To verify whether scores obtained by students at the end of clerkship predict their clinical reasoning performance at the end of residency. DESIGN: Comparison of scores obtained on a SC test taken at the end of clerkship with those obtained 2 years later at the end of residency on two clinical reasoning assessments of known validity, called the short-answer management problems (SAMPs) and the simulated office orals (SOOs), and an objective structured clinical examination (OSCE) aimed at assessing hands-on skills and clinical reasoning. Data were treated by Pearson correlation analysis. SUBJECTS AND SETTING: A cohort of 24 students from a medical school in Quebec was followed up to the end of their residency in family medicine, completed in several schools across Quebec. RESULTS: The observed Pearson correlation coefficients of the SC test were statistically significant (0.451, P=0.013; 0.447; P=0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically significant (0.340, P=0.052) when compared with the OSCE. CONCLUSION: The authors assumed that the richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on the measures of clinical reasoning (SAMP and SOO), but would predict less well performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study are coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also indicates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in training, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if the later measures bear on a wider clinical domain.
Authors: Louis Sibert; Stefan J Darmoni; Badisse Dahamna; Jacques Weber; Bernard Charlin Journal: BMC Med Inform Decis Mak Date: 2005-06-20 Impact factor: 2.796
Authors: Louis Sibert; Stefan J Darmoni; Badisse Dahamna; Marie-France Hellot; Jacques Weber; Bernard Charlin Journal: BMC Med Educ Date: 2006-08-28 Impact factor: 2.463
Authors: Jonathan S Ilgen; Judith L Bowen; Lalena M Yarris; Rongwei Fu; Robert A Lowe; Kevin Eva Journal: Acad Emerg Med Date: 2011-10 Impact factor: 3.451