BACKGROUND: Testing of knowledge is an important component in a successful skills curriculum. Nonetheless, structured testing of basic procedure-relevant knowledge in the surgical domains is not ordinary practice. A regional need assessment showed insufficient knowledge regarding basic laparoscopy for first-year residents in obstetrics and gynecology. This study therefore aimed to develop and validate a framework for a theoretical knowledge test, a multiple-choice test, in basic theory related to laparoscopy. METHODS: The content of the multiple-choice test was determined by conducting informal conversational interviews with experts in laparoscopy. The subsequent relevance of the test questions was evaluated using the Delphi method involving regional chief physicians. Construct validity was tested by comparing test results from three groups with expected different clinical competence and knowledge levels: senior medical students, first-year residents, and chief physicians. RESULTS: The four conversational interviews resulted in the development of 47 test questions, which were narrowed down to 37 test questions after two Delphi rounds involving 12 chief physicians. Significant differences were found between the test scores from the senior medical students (n = 14) and the first-year residents (n = 52) (median test scores, 18 vs. 24, respectively; p = 0.001), and between the first-year residents and the chief physicians (n = 12) (median test scores, 24 vs. 33, respectively; p = 0.001). Internal consistency (Cronbach's alpha) was 0.82. There was no evidence of differential item functioning between the three groups tested. CONCLUSIONS: A newly developed knowledge test in basic laparoscopy proved to have content and construct validity. The formula for the development and validation of a theoretical test could potentially be used for any topics that require structured testing of knowledge.
BACKGROUND: Testing of knowledge is an important component in a successful skills curriculum. Nonetheless, structured testing of basic procedure-relevant knowledge in the surgical domains is not ordinary practice. A regional need assessment showed insufficient knowledge regarding basic laparoscopy for first-year residents in obstetrics and gynecology. This study therefore aimed to develop and validate a framework for a theoretical knowledge test, a multiple-choice test, in basic theory related to laparoscopy. METHODS: The content of the multiple-choice test was determined by conducting informal conversational interviews with experts in laparoscopy. The subsequent relevance of the test questions was evaluated using the Delphi method involving regional chief physicians. Construct validity was tested by comparing test results from three groups with expected different clinical competence and knowledge levels: senior medical students, first-year residents, and chief physicians. RESULTS: The four conversational interviews resulted in the development of 47 test questions, which were narrowed down to 37 test questions after two Delphi rounds involving 12 chief physicians. Significant differences were found between the test scores from the senior medical students (n = 14) and the first-year residents (n = 52) (median test scores, 18 vs. 24, respectively; p = 0.001), and between the first-year residents and the chief physicians (n = 12) (median test scores, 24 vs. 33, respectively; p = 0.001). Internal consistency (Cronbach's alpha) was 0.82. There was no evidence of differential item functioning between the three groups tested. CONCLUSIONS: A newly developed knowledge test in basic laparoscopy proved to have content and construct validity. The formula for the development and validation of a theoretical test could potentially be used for any topics that require structured testing of knowledge.
Authors: Jeffrey H Peters; Gerald M Fried; Lee L Swanstrom; Nathaniel J Soper; Lelan F Sillin; Bruce Schirmer; Kaaren Hoffman Journal: Surgery Date: 2004-01 Impact factor: 3.982