Tai M Lockspeiser1, Patricia A Schmitter, J Lindsey Lane, Janice L Hanson, Adam A Rosenberg, Yoon Soo Park. 1. Dr. Lockspeiser is assistant professor, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Ms. Schmitter is program manager for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Lane is professor and vice chair for medical education, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Hanson is professor and director of medical education research and development, Departments of Pediatrics and Family Medicine, University of Colorado School of Medicine, Aurora, Colorado. Dr. Rosenberg is professor and director, Pediatric Residency Program, Department of Pediatrics, University of Colorado School of Medicine, Aurora, Colorado. Dr. Park is assistant professor, Department of Medical Education, University of Illinois at Chicago College of Medicine, Chicago, Illinois.
Abstract
PURPOSE: To provide validity evidence for use of the Learning Goal Scoring Rubric to assess the quality of written learning goals and residents' goal writing skills. METHOD: This two-part study used the rubric to assess University of Colorado third-year pediatric residents' written learning goals to obtain validity evidence. In study 1, five raters independently scored 48 goals written in 2010-2011 and 2011-2012 by 48 residents, who also responded to the Jefferson Scale of Physician Lifelong Learning (JeffSPLL). In study 2, two raters independently scored 48 goals written in 2011-2012 by 12 residents. Intraclass correlation coefficients (ICCs) assessed rater agreement to provide evidence for response process. Generalizability theory assessed internal structure. Independent-samples Mann-Whitney U tests and correlations assessed relationship to other variables. Content was matched to published literature and instructional methods. RESULTS: The ICC was 0.71 for the overall rubric. In study 1, where the generalizability study's (G study's) object of measurement was learning goals, the phi coefficient was 0.867. In study 2, where the G study's object of measurement was the resident (goal writing skill), the phi coefficient was 0.751. The total mean score of residents with goal writing training was significantly higher than that of those without (7.54 versus 4.98, P < .001). Correlation between goal quality and JeffSPLL score was not significant. Investigators agreed that the content matched the published literature and instructional methods. CONCLUSIONS: Preliminary validity evidence indicates that this scoring rubric can assess learning goal quality and goal writing skill.
PURPOSE: To provide validity evidence for use of the Learning Goal Scoring Rubric to assess the quality of written learning goals and residents' goal writing skills. METHOD: This two-part study used the rubric to assess University of Colorado third-year pediatric residents' written learning goals to obtain validity evidence. In study 1, five raters independently scored 48 goals written in 2010-2011 and 2011-2012 by 48 residents, who also responded to the Jefferson Scale of Physician Lifelong Learning (JeffSPLL). In study 2, two raters independently scored 48 goals written in 2011-2012 by 12 residents. Intraclass correlation coefficients (ICCs) assessed rater agreement to provide evidence for response process. Generalizability theory assessed internal structure. Independent-samples Mann-Whitney U tests and correlations assessed relationship to other variables. Content was matched to published literature and instructional methods. RESULTS: The ICC was 0.71 for the overall rubric. In study 1, where the generalizability study's (G study's) object of measurement was learning goals, the phi coefficient was 0.867. In study 2, where the G study's object of measurement was the resident (goal writing skill), the phi coefficient was 0.751. The total mean score of residents with goal writing training was significantly higher than that of those without (7.54 versus 4.98, P < .001). Correlation between goal quality and JeffSPLL score was not significant. Investigators agreed that the content matched the published literature and instructional methods. CONCLUSIONS: Preliminary validity evidence indicates that this scoring rubric can assess learning goal quality and goal writing skill.
Authors: D J Brinkman; J Tichelaar; M Okorie; L Bissell; T Christiaens; R Likic; R Mačìulaitis; J Costa; E J Sanz; B I Tamba; S R Maxwell; M C Richir; M A van Agtmael Journal: Clin Pharmacol Ther Date: 2017-05-30 Impact factor: 6.875
Authors: Amish Aghera; Matt Emery; Richard Bounds; Colleen Bush; R Brent Stansfield; Brian Gillett; Sally A Santen Journal: West J Emerg Med Date: 2017-12-21
Authors: Adam P Sawatsky; Andrew J Halvorsen; Paul R Daniels; Sara L Bonnes; Meltiady Issa; John T Ratelle; Christopher R Stephenson; Thomas J Beckman Journal: Med Educ Online Date: 2020-12