Thomas J Beckman1, Jayawant N Mandrekar. 1. Division of General Internal Medicine, Department of Internal Medicine, Mayo Clinic and Mayo Foundation, Rochester, Minnesota 55905, USA. Beckman.Thomas@mayo.edu
Abstract
BACKGROUND: We are unaware of any hypothesis-driven studies showing that teaching assessments are comprised solely of interpersonal and cognitive domains. Moreover, previous teaching assessments have been biased by heterogeneous samples of evaluators. Consequently, we investigated the construct validity of faculty assessments comprised of interpersonal and cognitive domains, utilising evaluations obtained from resident doctors on an internal medicine hospital service. METHODS: A total of 1000 inpatient evaluations were completed on 60 general internal medicine faculty members. Education theory supported a 2-dimensional, 14-item scale. Principal factor analysis was used to explore the scale's dimensionality. Internal reliability and interobserver agreement were determined. Relationships between domains and instructor characteristics were also examined. RESULTS: Principal factor analysis revealed interpersonal, clinical teaching and efficiency domains. Internal reliabilities of all domains are high (alpha > 0.90). Interobserver agreement is good (range 0.64-0.83). In the interpersonal domain there is a trend towards higher scores for lower ranking faculty. Significant findings are higher overall scores in the interpersonal domain (P < 0.001), higher scores for assistant professors in the interpersonal domain (P = 0.008) and higher scores for male than female faculty in the interpersonal (P = 0.041) and clinical teaching (P = 0.008) domains. CONCLUSIONS: Clinical teaching evaluations are reducible to interpersonal, clinical teaching and efficiency domains. Evidence for construct validity includes predicted domains and high internal and interobserver reliabilities. Utilising a homogenous sample of evaluators minimised variance. Interestingly, lower ranking faculty scored higher in the interpersonal domain, suggesting that lower ranking faculty may focus more attention on teaching activities than full professors do.
BACKGROUND: We are unaware of any hypothesis-driven studies showing that teaching assessments are comprised solely of interpersonal and cognitive domains. Moreover, previous teaching assessments have been biased by heterogeneous samples of evaluators. Consequently, we investigated the construct validity of faculty assessments comprised of interpersonal and cognitive domains, utilising evaluations obtained from resident doctors on an internal medicine hospital service. METHODS: A total of 1000 inpatient evaluations were completed on 60 general internal medicine faculty members. Education theory supported a 2-dimensional, 14-item scale. Principal factor analysis was used to explore the scale's dimensionality. Internal reliability and interobserver agreement were determined. Relationships between domains and instructor characteristics were also examined. RESULTS: Principal factor analysis revealed interpersonal, clinical teaching and efficiency domains. Internal reliabilities of all domains are high (alpha > 0.90). Interobserver agreement is good (range 0.64-0.83). In the interpersonal domain there is a trend towards higher scores for lower ranking faculty. Significant findings are higher overall scores in the interpersonal domain (P < 0.001), higher scores for assistant professors in the interpersonal domain (P = 0.008) and higher scores for male than female faculty in the interpersonal (P = 0.041) and clinical teaching (P = 0.008) domains. CONCLUSIONS: Clinical teaching evaluations are reducible to interpersonal, clinical teaching and efficiency domains. Evidence for construct validity includes predicted domains and high internal and interobserver reliabilities. Utilising a homogenous sample of evaluators minimised variance. Interestingly, lower ranking faculty scored higher in the interpersonal domain, suggesting that lower ranking faculty may focus more attention on teaching activities than full professors do.
Authors: Matthew R Thomas; Thomas J Beckman; Karen F Mauck; Stephen S Cha; Kris G Thomas Journal: J Gen Intern Med Date: 2011-03-03 Impact factor: 5.128
Authors: Staci M Lee; Mark C Lee; Darcy A Reed; Andrew J Halvorsen; Elie F Berbari; Furman S McDonald; Thomas J Beckman Journal: J Grad Med Educ Date: 2014-12
Authors: Christopher M Wittich; Karen F Mauck; Jayawant N Mandrekar; Karol A Gluth; Colin P West; Scott C Litin; Thomas J Beckman Journal: J Gen Intern Med Date: 2011-09-27 Impact factor: 5.128
Authors: S S Lenny Lases; Onyebuchi A Arah; E G J M Robert Pierik; Erik Heineman; M J M H Kiki Lombarts Journal: World J Surg Date: 2014-11 Impact factor: 3.352
Authors: Amy T Wang; Nicole P Sandhu; Christopher M Wittich; Jayawant N Mandrekar; Thomas J Beckman Journal: Mayo Clin Proc Date: 2012-11-07 Impact factor: 7.616
Authors: Cornelia R M G Fluit; Sanneke Bolhuis; Richard Grol; Roland Laan; Michel Wensing Journal: J Gen Intern Med Date: 2010-08-12 Impact factor: 5.128