J A Morzinski1, D B Schubot. 1. Department of Family and Community Medicine, Medical College of Wisconsin, USA. jmorzins@mcw.edu
Abstract
BACKGROUND AND OBJECTIVES: Many faculty development (FD) programs depend on external private and public sponsors that routinely require systematic studies on FD outcomes. The results of evaluation studies can influence whether or not programs continue to be funded. To better evaluate program outcomes on academic productivity, this paper presents and illustrates an evaluation method that uses the curricula vitae (CVs) of FD program graduates. METHODS: The evaluation method is implemented by first preparing a record-review template of coding categories that is applied to FD graduates' updated CVs. Next, semi-structured interviews are held with subjects to resolve uncertain CV codes. Finally, coded data are entered into standardized forms and analyzed to yield descriptive findings. The method was piloted with two groups of FD graduates (n = 17) to determine its utility and limitations. RESULTS: Results show excellent inter-rater reliability (Cohen's Kappa = .79). There was an overall increase in productivity, measured by the CV, during and after the FD program. CONCLUSIONS: CV analysis can be a useful method for assessing FD program outcomes. Several limitations of the method, such as incomplete CVs and self-report bias, must be considered.
BACKGROUND AND OBJECTIVES: Many faculty development (FD) programs depend on external private and public sponsors that routinely require systematic studies on FD outcomes. The results of evaluation studies can influence whether or not programs continue to be funded. To better evaluate program outcomes on academic productivity, this paper presents and illustrates an evaluation method that uses the curricula vitae (CVs) of FD program graduates. METHODS: The evaluation method is implemented by first preparing a record-review template of coding categories that is applied to FD graduates' updated CVs. Next, semi-structured interviews are held with subjects to resolve uncertain CV codes. Finally, coded data are entered into standardized forms and analyzed to yield descriptive findings. The method was piloted with two groups of FD graduates (n = 17) to determine its utility and limitations. RESULTS: Results show excellent inter-rater reliability (Cohen's Kappa = .79). There was an overall increase in productivity, measured by the CV, during and after the FD program. CONCLUSIONS: CV analysis can be a useful method for assessing FD program outcomes. Several limitations of the method, such as incomplete CVs and self-report bias, must be considered.