| Literature DB >> 35445932 |
Mary Ellen J Goldhamer1,2,3, Maria Martinez-Lage4,5, W Stephen Black-Schaffer4,5, Jennifer T Huang5,6, John Patrick T Co4,7,5, Debra F Weinstein8,9, Martin V Pusic5,6.
Abstract
Assessing residents and clinical fellows is a high-stakes activity. Effective assessment is important throughout training so that identified areas of strength and weakness can guide educational planning to optimize outcomes. Assessment has historically been underemphasized although medical education oversight organizations have strengthened requirements in recent years. Growing acceptance of competency-based medical education and its logical extension to competency-based time-variable (CB-TV) graduate medical education (GME) further highlights the importance of implementing effective evidence-based approaches to assessment. The Clinical Competency Committee (CCC) has emerged as a key programmatic structure in graduate medical education. In the context of launching a multi-specialty pilot of CB-TV GME in our health system, we have examined several program's CCC processes and reviewed the relevant literature to propose enhancements to CCCs. We recommend that all CCCs fulfill three core goals, regularly applied to every GME trainee: (1) discern and describe the resident's developmental status to individualize education, (2) determine readiness for unsupervised practice, and (3) foster self-assessment ability. We integrate the literature and observations from GME program CCCs in our institutions to evaluate how current CCC processes support or undermine these goals. Obstacles and key enablers are identified. Finally, we recommend ways to achieve the stated goals, including the following: (1) assess and promote the development of competency in all trainees, not just outliers, through a shared model of assessment and competency-based advancement; (2) strengthen CCC assessment processes to determine trainee readiness for independent practice; and (3) promote trainee reflection and informed self-assessment. The importance of coaching for competency, robust workplace-based assessments, feedback, and co-production of individualized learning plans are emphasized. Individual programs and their CCCs must strengthen assessment tools and frameworks to realize the potential of competency-oriented education.Entities:
Keywords: COVID-19; Milestones; clinical competency committee; competency-based advancement; competency-based medical education; individualized learning plan; time-variable graduate medical education
Mesh:
Year: 2022 PMID: 35445932 PMCID: PMC9021365 DOI: 10.1007/s11606-022-07515-3
Source DB: PubMed Journal: J Gen Intern Med ISSN: 0884-8734 Impact factor: 6.473
Current Obstacles and Key Enablers to Advancing the CCC Towards Competency-Based Advancement and Competency-Based Time-Variable Promotion Decisions
| CCC goal | Current landscape | Specific limitations | Proposed improvements | Examples for vignette |
|---|---|---|---|---|
| Discern and describe the developmental status of each resident to optimize education | Lack of a shared mental model of how to conduct trainee developmental assessment[ | Straight-line scoring on the Milestones | Provide faculty development activities aimed at a shared model of assessment and competency-based advancement. CCCs synthesize evaluative feedback for all trainees, whether struggling, average, or exceptional (like Leila), to inform individualized learning plans, co-produced by trainees with program leadership | Data is available that takes into account Leila’s unique journey, allowing individualization |
| Lack of a shared mental model of how to conduct trainee developmental assessment[ | Focus on outlier identification | Discuss EVERY trainee at the CCC meeting with a view to providing forward-oriented recommendations, based on the competency model. | Developmental perspective allows Leila to plan and adjust her training experiences; educational value becomes a criterion for activity scheduling | |
| Failure to address coach-evaluator tension | CCC members often fill both coach and evaluator roles | Diversify CCC membership to include a wide range of stakeholders, including those who do not necessarily have an education role | Clear separation of coach and evaluator increases opportunity for Leila to confide stressors and to adopt growth mindset | |
| CCC may not have sufficient diversity in terms of race, gender, ethnicity, LGBTQ+ | Prone to implicit bias and to counter-productive group dynamics | Ensure diversity of CCC membership, explicit consideration of the group processes | Leila was pleased to see a foreign medical graduate represented on the CCC. | |
| Determine each resident’s readiness for unsupervised practice | Lack of explicit competency-based criteria to determine readiness for graduation and unsupervised practice | Advancement is based on demonstration of specific, observable positive behaviors, rather than absence of problems or sanctions | Utilize explicit criteria for competency-based advancement including achievement of the ACGME Milestones | Leila understands what competencies she needs to demonstrate in order to graduate, and where this has or has not been accomplished |
| Foster each resident’s ability to self-assess | Resident self-evaluation and reflection often only done informally | Informed self-assessment, self-monitoring, and reflective practice are underemphasized by faculty and undervalued by trainees | Ensure that residents practice the skills of informed self-assessment. Incorporate trainee Milestone self-assessment into CCC meeting discussion Utilizing CCC determinations for co-produced individualized learning plans | As Leila learns to self-assess, she understands in which areas she is less strong than others and understands what additional growth is needed to graduate |
| Few data visualizations available, and even fewer that are informed by a competency model | When examinations are the key data point, that sends a message as to what is valued | Adopt a quality improvement mindset for self-improvement, where data visualizations play a key role | Leila works with her program director to make evidence-based decisions to determine which elective rotations or other experiences will enable her to achieve competency |
Figure 1Legend: Bow-Tie Framework for CCC process. A wide range of data are collected and interpreted from three important perspectives: the resident, the coach-advocate, the program advocate. These unique perspectives on the data are kept in balance through data sharing and defined processes validating each perspective. Pre-work leads to an efficient, focused process during the CCC meeting. Conclusions are communicated in the form of both modified learning plans to support the development of each resident and Milestone predictions that promote downstream adjustment of the learner’s path.
Figure 2Legend: Heat map visualization of Milestone competency achievement in one program. An integrated heat map from one residency program’s CCC data, utilizing the system’s independent “Passport” system of Milestone competency assessment which evaluates each Milestone sub-competency. Each column represents one resident; each row, one competency; each cell, the cumulative longitudinal consensus of his or her evaluators. The color corresponds to the ranking, with red scores lower on the developmental progression than blue. White squares indicate missing data. While individual residents vary in their ratings, the program overall is likewise more successful in achieving some competency elements than others. The columns are organized with the more junior residents to the left and the more senior resident to the right. The rows correspond to the ACGME Pathology Milestone sub-competencies, based on the six core competencies. ICS1, Interpersonal and Communication Skills sub-competency 1; MK1, Medical Knowledge sub-competency 1; PBL, Practice-Based Learning and Improvement; PC, Patient Care; PROF, Professionalism; SBP, Systems-Based Practice. This heat map incorporates approximately 5600 datapoints. Figure courtesy of Drs. Emilio Madrigal and Long Phi Le, Department of Pathology, Massachusetts General Hospital.