Eta S Berner1, Amanda D Dorsey2, Robert L Garrie2, Haiyan Qu2. 1. Professor and Director, Center for Health Informatics for Patient Safety/Quality, Department of Health Services Administration, University of Alabama at Birmingham, Birmingham, AL, USA eberner@uab.edu. 2. Department of Health Services Administration, University of Alabama at Birmingham, Birmingham, AL, USA.
Abstract
OBJECTIVE: Informatics programs need assurance that their curricula prepare students for intended roles as well as ensuring that students have mastered the appropriate competencies. The objective of this study is to describe a method for using assessment data to identify areas for curriculum, student selection, and assessment improvement. MATERIALS AND METHODS: A multiple-choice examination covering the content in the Commission for Health Accreditation of Informatics and Information Management Education curricular facets/elements was developed and administered to 2 cohorts of entering students prior to the beginning of the program and to the first cohort after completion of the first year's courses. The reliability of the examination was assessed using Cronbach's alpha. Content validity was assessed by having 2 raters assess the match of the items to the Commission for Health Accreditation of Informatics and Information Management Education requirements. Construct validation included comparison of exam performance of instructed vs uninstructed students. Criterion-related validity was assessed by examining the relationship of background characteristics to exam performance and by comparing examination performance to graduate Grade Point Average (GPA). RESULTS: Reliability of the examination was 0.91 and 0.82 (Cohort 1 pre/post-tests) and 0.43 (Cohort 2 pretest). Both raters judged 76% of the test items as appropriate. There were statistically significant differences between the instructed (Cohort 1 post-test) and uninstructed (Cohort 2 pretest) students (t = 2.95 P < .01), as well as between the Cohort 1 pre/post-tests (t = 6.52, P < .001). Neither the background variables nor the graduate GPA were significantly correlated with the examination scores. CONCLUSION: We found that the examination had generally good psychometric properties and the exceptions could be used to identify areas for curriculum and assessment improvement.
OBJECTIVE: Informatics programs need assurance that their curricula prepare students for intended roles as well as ensuring that students have mastered the appropriate competencies. The objective of this study is to describe a method for using assessment data to identify areas for curriculum, student selection, and assessment improvement. MATERIALS AND METHODS: A multiple-choice examination covering the content in the Commission for Health Accreditation of Informatics and Information Management Education curricular facets/elements was developed and administered to 2 cohorts of entering students prior to the beginning of the program and to the first cohort after completion of the first year's courses. The reliability of the examination was assessed using Cronbach's alpha. Content validity was assessed by having 2 raters assess the match of the items to the Commission for Health Accreditation of Informatics and Information Management Education requirements. Construct validation included comparison of exam performance of instructed vs uninstructed students. Criterion-related validity was assessed by examining the relationship of background characteristics to exam performance and by comparing examination performance to graduate Grade Point Average (GPA). RESULTS: Reliability of the examination was 0.91 and 0.82 (Cohort 1 pre/post-tests) and 0.43 (Cohort 2 pretest). Both raters judged 76% of the test items as appropriate. There were statistically significant differences between the instructed (Cohort 1 post-test) and uninstructed (Cohort 2 pretest) students (t = 2.95 P < .01), as well as between the Cohort 1 pre/post-tests (t = 6.52, P < .001). Neither the background variables nor the graduate GPA were significantly correlated with the examination scores. CONCLUSION: We found that the examination had generally good psychometric properties and the exceptions could be used to identify areas for curriculum and assessment improvement.
Authors: Charles Safran; M Michael Shabot; Benson S Munger; John H Holmes; Elaine B Steen; John R Lumpkin; Don E Detmer Journal: J Am Med Inform Assoc Date: 2008-12-11 Impact factor: 4.497
Authors: Vishnu Mohan; Patricia Abbott; Shelby Acteson; Eta S Berner; Corkey Devlin; William E Hammond; Rita Kukafka; William Hersh Journal: J Am Med Inform Assoc Date: 2013-07-05 Impact factor: 4.497
Authors: C U Lehmann; C A Longhurst; W Hersh; V Mohan; B P Levy; P J Embi; J T Finnell; A M Turner; R Martin; J Williamson; B Munger Journal: Appl Clin Inform Date: 2015-04-15 Impact factor: 2.342
Authors: Steven D Miller; Phillip Stablein; Jay Syed; Valerie Smothers; Emily Marx; Peter Greene; Harold Lehmann; Paul G Nagy Journal: Appl Clin Inform Date: 2019-08-28 Impact factor: 2.342