Daniel N Ricotta1,2,3, Brittany L Ranchoff4,5, Christine P Beltran4,5, Andrew J Hale6, Jason A Freed4,7, Grace C Huang4,7,5. 1. Department of Medicine Beth, Israel Deaconess Medical Center, Boston, MA, USA. dricotta@bidmc.harvard.edu. 2. Harvard Medical School, Boston, MA, USA. dricotta@bidmc.harvard.edu. 3. Carl J. Shapiro Institute for Education and Research, Boston, MA, USA. dricotta@bidmc.harvard.edu. 4. Department of Medicine Beth, Israel Deaconess Medical Center, Boston, MA, USA. 5. Carl J. Shapiro Institute for Education and Research, Boston, MA, USA. 6. Larner College of Medicine at the University of Vermont, Burlington, VT, USA. 7. Harvard Medical School, Boston, MA, USA.
Abstract
BACKGROUND: In the context of inpatient general medicine, "rounding" refers to the process of seeing, assessing, and caring for patients as a team. The clinical leadership skills required of residents to lead rounds are essential to inpatient care and clinical education. Assessment of these skills has relevance to developing competent physicians; however, there is an absence of widely accepted tools to specifically measure this competency. OBJECTIVE: To develop and collect validity evidence for a direct observation instrument of internal medicine residents' leadership skills during daily inpatient care rounds for future formative assessment. DESIGN: Prospective observational study. PARTICIPANTS: PGY2 and PGY3 internal medicine residents. MAIN MEASURES: The authors collected inferences of validity evidence according to Kane's validity model. They performed direct observations of PGY2 and PGY3 residents by individual faculty and trained raters and measured inter-rater reliability, using the kappa statistic. Mixed linear regression models were used to compare PGY2 and PGY3 residents. Surveys captured faculty perceptions about value of the instrument. KEY RESULTS: A total of 223 observations were performed in 92 unique individuals. Twenty-four faculty used the observation instrument, of which 18 (75%) completed the post-survey, and 100% agreed that the instrument represented the resident's global leadership abilities. Inter-rater reliability was strong, with an overall kappa statistic equaling 0.82. The mean performance for PGY2 and PGY3 residents was 15.9 (SD 5.1) and 17.7 (SD 4.1), respectively. Adjusting for repeated measures, there was no statistically significant difference between groups. CONCLUSIONS: The authors reported evidence for all four stages of validity and use of the instrument in clinical practice. Their work provides a codification of best practices of rounding leadership, which directly impacts the education of trainees, care of hospitalized patients, and use for formative assessment. The instrument also has the potential to be used for summative assessment.
BACKGROUND: In the context of inpatient general medicine, "rounding" refers to the process of seeing, assessing, and caring for patients as a team. The clinical leadership skills required of residents to lead rounds are essential to inpatient care and clinical education. Assessment of these skills has relevance to developing competent physicians; however, there is an absence of widely accepted tools to specifically measure this competency. OBJECTIVE: To develop and collect validity evidence for a direct observation instrument of internal medicine residents' leadership skills during daily inpatient care rounds for future formative assessment. DESIGN: Prospective observational study. PARTICIPANTS: PGY2 and PGY3 internal medicine residents. MAIN MEASURES: The authors collected inferences of validity evidence according to Kane's validity model. They performed direct observations of PGY2 and PGY3 residents by individual faculty and trained raters and measured inter-rater reliability, using the kappa statistic. Mixed linear regression models were used to compare PGY2 and PGY3 residents. Surveys captured faculty perceptions about value of the instrument. KEY RESULTS: A total of 223 observations were performed in 92 unique individuals. Twenty-four faculty used the observation instrument, of which 18 (75%) completed the post-survey, and 100% agreed that the instrument represented the resident's global leadership abilities. Inter-rater reliability was strong, with an overall kappa statistic equaling 0.82. The mean performance for PGY2 and PGY3 residents was 15.9 (SD 5.1) and 17.7 (SD 4.1), respectively. Adjusting for repeated measures, there was no statistically significant difference between groups. CONCLUSIONS: The authors reported evidence for all four stages of validity and use of the instrument in clinical practice. Their work provides a codification of best practices of rounding leadership, which directly impacts the education of trainees, care of hospitalized patients, and use for formative assessment. The instrument also has the potential to be used for summative assessment.
Entities:
Keywords:
assessment; graduate medical education; leadership; validation
Authors: Sandra K Oza; Sandrijn van Schaik; Christy K Boscardin; Read Pierce; Edna Miao; Tai Lockspeiser; Darlene Tad-Y; Eva Aagaard; Anda K Kuo Journal: J Grad Med Educ Date: 2018-10
Authors: Jan Kiesewetter; Marion Schmidt-Huber; Janine Netzel; Alexandra C Krohn; Matthias Angstwurm; Martin R Fischer Journal: GMS Z Med Ausbild Date: 2013-11-15