Hospitalist workload influences faculty evaluations by internal medicine clerkship students.

BACKGROUND: The last decade has brought significant changes to internal medicine clerkships through resident work-hour restrictions and the widespread adoption of hospitalists as medical educators. These key medical educators face competing demands for quality teaching and clinical service intensity.
OBJECTIVE: The study reported here was conducted to explore the relationship between clinical service intensity and teaching evaluations of hospitalists by internal medicine clerkship students.
DESIGN: A retrospective correlation analysis of clinical service intensity and teaching evaluations of hospitalists by internal medicine clerkship students during the 2009 to 2013 academic years at Southern Illinois University School of Medicine was conducted. PARTICIPANTS: Internal medicine hospitalists who supervise the third-year inpatient experience for medical students during the 2009 to 2013 academic years participated in the study. MEASURES: Clinical service intensity data in terms of work relative value units (RVUs), patient encounters, and days of inpatient duty were collected for all members of the hospitalist service. Medical students rated hospitalists in the areas of patient rapport, enthusiasm about the profession, clinical skills, sharing knowledge and skills, encouraging the students, probing student knowledge, stimulating independent learning, providing timely feedback, providing constructive criticism, and observing patient encounters with students.
RESULTS: Significant negative correlations between higher work RVU production, total patient encounters, duty days, and learner evaluation scores for enthusiasm about the profession, clinical skills, probing the student for knowledge and judgment, and observing a patient encounter with the student were identified. Higher duty days had a significant negative correlation with sharing knowledge/skills and encouraging student initiative. Higher work RVUs and total patient encounters were negatively correlated with timely feedback and constructive criticism.
CONCLUSION: The results suggest that internal medicine clerkship student evaluations of hospitalist faculty are negatively influenced by high clinical service intensity measured in terms of annual work RVUs, patient encounters, and duty days.

Background

The last decade has brought significant changes to internal medicine clerkships through resident work-hour restrictions and the widespread adoption of hospitalists as medical educators.1 The 2010 annual Clerkship Directors in Internal Medicine survey indicated that 91% of internal medicine clerkships use hospitalists as teaching physicians.1 Observational studies2–4 and a systematic review5 have shown hospitalists to be effective medical-student educators, and nearly two-thirds of academic hospitalists are classified as clinician-educators (CEs).6

The role of the CE seems ideally suited to academic hospitalists, who have intense day-to-day clinical contact and myriad opportunities for clinical bedside teaching.

The majority of US medical schools have CE faculty tracks that emphasize teaching and clinical service over research.7 Surveys of department and promotion committee chairs indicate that teaching skills have the highest priority in promotion decisions regarding CEs,8,9 with the majority of promotion committee chairs (72%) using learner evaluations as a means of assessing teaching skills.8 Department and promotion committee chairs identify learner evaluations as very important, but of low quality, in the promotion decision-making process.9

The quality and utility of learner evaluations of faculty has been an area of extensive and often contradictory research.10,11 A review of larger studies of learner evaluations from a diverse range of educational settings indicates that up to 25% of the variability in learner ratings is due to factors such as class size, workload, difficulty of the subject material, and faculty grading leniency.12 Investigations into medical-student learner evaluations have shown positive influences on scores due to factors such as clinical rotations occurring earlier in the academic year13 and teacher attributes of academic rank,14 recent board certification, attendance at a teacher training program, and prior receipt of a teaching award.15 A greater time commitment to teaching has a positive influence on learner evaluations, but this directly conflicts with the clinical service demands faced by many hospitalist CEs.14–19

Objective

The study reported here was conducted to explore the potential relationship between clinical service intensity and teaching evaluations of hospitalist CEs by internal medicine clerkship students.

Design

The study was a retrospective review of clinical productivity and teaching evaluations of faculty by clerkship students (third-year medical students) prospectively collected during the 2009 to 2013 academic years (July 1 to June 30 of each year) for the hospitalists in the Department of Internal Medicine of the Southern Illinois University School of Medicine (SIU-SOM). After linkage of productivity data to teaching data, the dataset was de-identified prior to analysis.

To protect the anonymity of individual faculty evaluations, no demographic data for the hospitalists were included in the dataset for analysis.

The study was approved as an exempt study by the Springfield Committee for Research Involving Human Subjects, the local institutional review board.

Study participants

SIU-SOM has a total enrollment of 298 medical students who have inpatient rotations at two university-affiliated hospitals. Supervision of the students during internal medicine inpatient rotations is provided by the hospitalists of the internal medicine department at SIU-SOM. All hospitalists are board certified or board eligible in internal medicine, and virtually all inpatients are seen with medical students. A nonacademic hospitalist service does not exist at SIU-SOM for comparison. Non-hospitalist faculty provides occasional weekend coverage, but are not evaluated by medical students for inpatient teaching.

Measures

Clinical service intensity data in terms of work relative value units (wRVUs), the number of patient encounters, and number of days of inpatient duty were collected for all members of the hospitalist service from administrative databases. wRVU values for medical services were determined by the current Center for Medicare and Medicaid Services physician fee schedule. Average wRVUs per day (RVUs/day) and encounters per day (encs/day) were calculated for each hospitalist.

Teaching evaluations of the hospitalist faculty by students were compiled by the clerkship office in an anonymous fashion. Each faculty member was rated on a Likert scale of 1–5 by the students in the areas of patient rapport, enthusiasm about the profession, clinical skills, sharing knowledge and skills, encouraging the students, probing student knowledge, stimulating independent learning, providing timely feedback, providing constructive criticism, and observing patient encounters with students. The average score for the faculty member for each area was used for analysis.

Annual datasets (productivity data matched with teaching scores for a specific hospitalist) from all hospitalists were analyzed with SPSS software (v 20; IBM Corporation, Armonk, NY, USA) for the 2009 to 2013 academic years. A total of 32 datasets were available for analysis. Spearman correlations were used to assess the strength and direction of the association of clinical service intensity variables with teaching scores.

Results

A total of 32 sets of annual learner evaluations and clinical service intensity data were reviewed, representing data for 18 individual hospitalist CEs. The 18 hospitalists included six women (33%), three hospitalists (17%) with 10 or more years of practice after completion of residency, and four hospitalists (22%) with a large percentage of their time spent on administrative duties. Two hospitalists held the rank of associate professor (11%), and the remainder held the rank of assistant professor.

Hospitalists in this study had a median annual productivity of 3,139 wRVUs, had 1,454 patient encounters, and worked in the hospital for 166 days (Table 1). On a daily basis, hospitalists generated 20 wRVUs and had nine patient encounters. The distributions of all evaluated measures of clinical service intensity were skewed toward higher intensity. The greatest degrees of skewing can be seen in duty days and RVUs/day.

Table 1

Clinical service intensity measures

Measure	Median	Range	Skewness
Work RVUs	3,139	1,678–4,512	−0.130
Encounters	1,454	910–2,029	−0.020
Duty days	166	123–175	−0.791
RVUs/day	21	12–26	−0.413
Encs/day	9	6–12	−0.205

Abbreviations: Encs/day, patient encounters per duty day; RVUs, relative value units.

Learner evaluation scores in each area ranged from 3 to 5 out of a maximum possible score of 5, with skewing of the distributions toward higher scores (Table 2). There was greater variability in individual scores, particularly in the areas of clinical skills and sharing knowledge.

Table 2

Annual teaching scores

Score	Median	Range (1–5)	Skewness
Patient rapport	4.73	3.3–5.0	−1.565
Enthusiasm about profession	4.72	4.0–5.0	−1.515
Clinical skills	4.73	4.0–5.0	−0.766
Shares knowledge and skills	4.81	4.0–5.0	−1.657
Encourages student initiative	4.75	4.0–5.0	−0.992
Probes knowledge and judgment	4.73	4.0–5.0	−0.554
Stimulates independent learning	4.67	3.8–5.0	−0.932
Timely feedback	4.50	3.0–5.0	−1.032
Provides constructive criticism	4.50	3.0–5.0	−0.759
Observes patient encounter	4.27	3.0–5.0	−0.644

Correlation analysis (Table 3, Figure 1) showed significant negative correlations between higher wRVUs, total patient encounters, duty days, and learner evaluation scores for enthusiasm about the profession, clinical skills, probing the student for knowledge and judgment, and observing a patient encounter with the student. Higher duty days had a significant negative correlation with sharing knowledge/skills and encouraging student initiative. Higher wRVUs and encounters were negatively correlated with timely feedback and constructive criticism.

Table 3

Spearman correlations between clinical service intensity and teaching scores

Score	Spearman correlations
	wRVUs	Encounters	Duty days	RVUs/day	Encs/day
Patient rapport	−0.064, P=0.728	−0.114, P=0.534	−0.159, P=0.384	−0.063, P=0.731	−0.223, P=0.220
Enthusiasm about profession	−0.587, P<0.001‡	−0.567, P=0.001‡	−0.580, P=0.001‡	−0.348, P=0.051	−0.252, P=0.165
Clinical skills	−0.417, P=0.018‡	−0.384, P=0.030‡	−0.471, P=0.006‡	−0.230, P=0.205	−0.111, P=0.546
Shares knowledge and skills	−0.292, P=0.104	−0.294, P=0.102	−0.407, P=0.021‡	−0.099, P=0.588	−0.064, P=0.729
Encourages student initiative	−0.324, P=0.070	−0.310, P=0.084	−0.409, P=0.020‡	−0.086, P=0.640	−0.033, P=0.858
Probes knowledge and judgment	−0.486, P=0.005‡	−0.471, P=0.007‡	−0.507, P=0.003‡	−0.221, P=0.225	−0.137, P=0.453
Stimulates independent learning	−0.287, P=0.111	−0.305, P=0.090	−0.189, P=0.300	−0.169, P=0.356	−0.196, P=0.282
Timely feedback	−0.397, P=0.024‡	−0.447, P=0.010‡	−0.341, P=0.056	−0.337, P=0.059	−0.365, P=0.040‡
Provides constructive criticism	−0.352, P=0.048‡	−0.396, P=0.025‡	−0.318, P=0.076	−0.291, P=0.106	−0.332, P=0.063
Observes patient encounter	−0.439, P=0.012‡	−0.438, P=0.012‡	−0.600, P<0.001‡	−0.180, P=0.325	−0.113, P=0.537

Note:

P<0.050.

Abbreviations: Encs/day, patient encounters per duty day; wRVUs, work relative value units; RVUs/day, work relative value units per duty day.

Figure 1

Scatterplot analysis of clinical service intensity vs faculty ratings.

Abbreviation: RVUs, relative value units.

In addition to these indicators of clinical service intensity on an annual basis, higher average encs/day was correlated with lower scores on timely feedback. Ratings related to simulating independent learning were insensitive to any of the measures of clinical service intensity.

Discussion

This study showed significant correlations between long-term measures of service intensity such as annual wRVUs, total patient encounters, and duty days, with lower teaching evaluation scores in most areas evaluated. These results differ from data reported for emergency medicine physicians19,20 and anesthesiologists.21 Higher daily wRVUs and daily patient encounters were only correlated with lower teaching scores related to timely feedback.

Potential explanations of the correlation of lower teaching scores with higher clinical service intensity are diverse, including faculty burnout and a local trend for more experienced faculty to spend less time in direct patient care. Emergency medicine and anesthesiology differ significantly from hospital-based medicine because they are geographically organized, procedural, and involve only short periods of patient contact. These factors may attenuate the risk for physician burnout and lower-quality teaching for these specialties.

The SIU-SOM hospitalists’ median wRVUs (3,150) is lower than the national median (4,159) for all hospitalists as reported in the 2012 State of Hospital Medicine Report.22 This is likely due to the academic nature of this service, which is focused on meeting the needs of the learners in the system and has the ability to redirect patients to other independent hospitalist practices. This may allow the hospitalists in this system to spend more time on teaching, which is a known influencer of learner evaluations.14–16

A wide range (median: 166 days, range: 123–175 days) of annual inpatient duty days was also seen for this group of hospitalists, reflecting the diversity of service assignments for members of the hospitalist practice. Four out of 18 hospitalists (22%) in this study had significant administrative responsibilities, which reduced the time spent on inpatient duty and inpatient teaching.

These factors, which may not be present in other academic hospitalist groups, limit the generalizability of the results of this investigation. Furthermore, this study was a retrospective analysis that looked for correlations between multiple variables, thus it is not likely to account for all potential cofounding and difficult to quantify variables such as teaching experience or skill.

Further investigation of academic hospitalists with wRVU productivity closer to the national median and a larger sample size to investigate the influences of other factors such as academic rank, years of teaching experience, and formal teaching training is essential to determine if medical schools should alter clinical service intensity expectations for teaching faculty to maximize the student-rated quality of the inpatient component of the internal medicine clerkship.

Conclusion

The results of this study suggest that internal medicine clerkship student evaluations of hospitalist faculty may be negatively influenced by high clinical service intensity measured in terms of annual wRVUs, patient encounters, and duty days. Further investigation is needed to determine what level of clinical service intensity begins to affect the learner-rated quality of the internal medicine clerkship inpatient experience.